The state of images on the web is pretty rough. What should be an easy goal, showing a user a picture, is actually complicated. Good websites deliver the right resolution to the right device, perform lazy loading, support ‘retina’ (or high density pixel) devices, serve new image formats like WebP to the correct clients, and show placeholders whilst images load. It’s messy, hard to get right, and requires a lot more effort than a user might realize behind the scenes. Here’s a blow-by-blow account of how I attempt to incorporate those techniques into this blog.

Let’s start with my goals:

1. Devices should download an appropriate image for their resolution, including supporting high pixel density or ‘retina’ images.
2. Images should look good regardless of the device size.
3. Images should be lazy loaded.
4. A placeholder should be displayed whilst images load.
5. Images should work well with or without JavaScript enabled.
6. I shouldn’t need to store an unnecessary amount of images under source control.
7. I shouldn’t have to write raw HTML to make anything work.

Let’s look at these goals in chronological order and see how I get to my solution.

Goal: Devices Should Download an Appropriate Image for Their Resolution

Whilst your high resolution screen might need the highest fidelity image I can offer chances are that readers on their phone can cope with a much smaller image both in terms of file size and dimensions; small screens just don’t need as many pixels per image. Serving a smaller image where possible saves battery life, saves the reader’s bandwidth, and leads to a quicker page load.

The first question to ask is ‘How do I know what size image my reader needs?’. The answer is - it’s difficult to know. The best solution is to provide hints to the browser and let it choose. By using the HTML5 srcset and sizes attributes we specify the options available and the size the image should be displayed at in the browser. Whilst most modern browsers support these attributes img’s that don’t gracefully fall back to the src attribute.

<img 
    sizes="(max-width: 1400px) 100vw, 1400px"
    srcset="castle_c_scale,w_200.jpg 200w, castle_c_scale,w_364.jpg 364w, castle_c_scale,w_1400.jpg 1400w"
    src="castle_c_scale,w_1400.jpg"
    alt="A fancy castle"
>

This technique, also supports high density (or ‘retina’) displays. If the browser does understand the sizes and srcset syntax then it can combine that information with the device’s pixel ratio to request high density images (if available). So far, so good. All we need to do to make this work is produce a variety of images with appropriate dimensions and sizes - the browser will do the rest!

Goal: Images Should Look Good Regardless of the Device Size

Serving appropriately sized images to smaller devices is one half of the battle but sometimes an image’s content won’t translate across devices. Whilst a wide image may look good on a desktop device it’s unlikely that the same image will look as good on a portrait mobile device. It’s possible that the width of the image will constrain the narrowness of the device’s screen to the point that scaling causes the detail of the image to be lost. What we need in this situation is art direction. That is, we want to serve different versions of an image to different sized devices to ensure that the image retains its fidelity.

Whilst generating different size images is easy enough to automate, smartly generating different variants of an image at different aspect ratios is much harder. Fortunately Cloudinary has an open source tool (responsivebreakpoints.com) that takes a single image and produces images of varying quality at chosen aspect ratios per device size, for instance 1:1 images for mobile, 3:2 for tablet, 16:9 for small laptops, and original ratio for desktops. These images are then handily bundled up in a single .zip file, and it provides HTML <picture> markup for the images it generates.

Goal: I Shouldn’t Need to Store an Unnecessary Amount of Images Under Source Control

So now we have some HTML and a zip file of images. Historically I would’ve copied the images into the /static/images/{YEAR}/{MONTH} directory of my blog’s repository, but git isn’t suited to storing a large amount of large binary files - whilst it made sense when I was only storing one version of each image it doesn’t scale to the kind of volume produced by responsivebreakpoints.com. I’ve opted to store my images in an Amazon S3 Bucket. S3’s perfect for this kind of thing as it handles things like reliability and transfer speed for me. All I need to worry about is making sure that images end up in the right place, and it handles the rest. I opted to make my URLs prettier than the default given by AWS, which are truly ugly¹, by using a CloudFront distribution to put the files under my own domain. Images are predictably hosted at files.arranfrance.com/images/{YEAR}/{MONTH}/{SUBJECT}/!

Goal: I Shouldn’t Have to Write Raw HTML to Make Anything Work

Now we theoretically have a variety of appropriately sized art-directed images stored in S3 at a predictable URL and some HTML given to us by responsivebreakpoints.com. We have a couple of problems though - the HTML we have looks like this:

<!--Some srcsets removed for the sake of brevity-->
<picture>
<source
media="(max-width: 767px)"
sizes="(max-width: 1366px) 100vw, 1366px"
srcset="
castle_ar_1_1,c_fill,g_auto__c_scale,w_200.jpg 200w,
castle_ar_1_1,c_fill,g_auto__c_scale,w_333.jpg 333w,
...,
castle_ar_1_1,c_fill,g_auto__c_scale,w_1366.jpg 1366w">
<source
media="(min-width: 768px) and (max-width: 991px)"
sizes="(max-width: 1983px) 70vw, 1388px"
srcset="
...,
castle_ar_4_3,c_fill,g_auto__c_scale,w_1388.jpg 1388w">
<source
media="(min-width: 992px) and (max-width: 1199px)"
sizes="(max-width: 2400px) 60vw, 1440px"
srcset="
...,
castle_ar_16_9,c_fill,g_auto__c_scale,w_1440.jpg 1440w">
<img
sizes="(max-width: 5120px) 40vw, 2048px"
srcset="
...,
castle_c_scale,w_2048.jpg 2048w"
src="castle_c_scale,w_2048.jpg"
alt="">
</picture>

You may notice a few issues at first glance.

1. It’s missing alt attributes
2. There are no captions
3. The images aren’t sourced correctly - they are located them to be located in the same place as the HTML as opposed to stored on a different domain
4. If I ever adjust the template for these images (for example, by adding some styling) I have to change HTML across many blog posts

Additionally, the HTML is large enough to become a distraction whilst writing and a lot of the structure of the markup is boilerplate, this feels like a problem my static site generator, Hugo, should be able to solve.

Hugo Shortcodes: An Initial Attempt

Hugo was thinking along the same lines as us when it comes to the problem of our unwieldy HTML and came up with a solution - shortcodes.

Hugo loves Markdown because of its simple content format, but there are times when Markdown falls short. Often, content authors are forced to add raw HTML (e.g., video <iframes>) to Markdown content. We think this contradicts the beautiful simplicity of Markdown’s syntax.

By writing a Hugo shortcode, we can turn the ugly HTML into a simple snippet we can reuse. Now we have a strategy - we just need to implement it. We can start by figuring out what the repeatable elements of the HTML are.

Each image starts with a <picture> tag and then has many <source> tags which each have a media, sizes, and srcset attribute. There’s also a final <img> tag to allow browsers that don’t support <picture> to fallback. We also want to optionally include a caption for images. An initial semi-pseudo-shortcode solution could look like this.

{{ picture caption="Example Caption" }}
{{ responsiveimage media="..." sizes="..." srcset=".." }}
...
{{ fallbackimage sizes="..." srcset="..." media="..." }}
{{ endpicture }}

In this example we wrap everything in a picture shortcode which allows us to fence content inside it. If the caption is provided we make the picture a <figure> with a <figcaption> otherwise it is just a regular picture element.

{{ if .Get "caption"}}
<figure>
{{ end }}
<picture>
    {{ .Inner }}
</picture>
{{ if .Get "caption"}}
    <figcaption>{{ .Get "caption" }}</figcaption>
</figure>
{{ end }}

The responsive image shortcode is fairly straightforward.

<source media="{{ .Get "media" }}" sizes="{{ .Get "sizes" }}" srcset="{{ .Get "srcset" }}">

As is the fallback image.

<img sizes="{{ .Get "sizes" }}" srcset="{{ .Get "srcset" }}" src="{{ .Get "src" }}"    alt="{{ .Get "caption" }}">

The problem is that whilst the shortcodes are reusable - they aren’t much shorter, and we still don’t have a solution for transforming our HTML into one.

Writing a Generator: A First Pass

Turning HTML into something else in a repeatable way sounds like the job for a program! For the initial proof of concept I wrote a basic ~30 line JavaScript program that takes a string, parses it using cheerio, transforms it, and logs it to console².

var cheerio = require('cheerio'),

// TODO: Get this from the command line
$ = cheerio.load(`<picture ... >`);

// TODO: This should be dynamic
const prefix = 'https://files.arranfrance.com/images/2019/Jan/Witcher2/Prologue/'

function prefixSource(srcset) {
    return srcset
        .split("w,")
        .map(i => prefix + i.trim())
        .reduce((s, v, i) => s + (i > 0 ? 'w,' : '') + v, '');
}

const sources = [];
$('source').each((i, elem) => {
    const source = $(elem);
    const media = source.attr('media');
    const sizes = source.attr('sizes');
    const srcset = prefixSource(source.attr('srcset'))
    sources.push({media, sizes, srcset});
});

const img = $('img');
const sizes = img.attr('sizes');
const srcset = prefixSource(img.attr('srcset'));
const src = 'https://files.arranfrance.com/images/' + img.attr('src');

let s = '{{< picture caption="" >}} \n';
sources.forEach(source => {
    s+= `{{< responsiveimage media="${source.media}" sizes="${source.sizes}" srcset="${source.srcset}" >}} \n`
});
s+= `{{< fallbackimage sizes="${sizes}" srcset="${srcset}" src="${src}" >}} \n`;
s+= '{{< /picture >}}';

console.log(s);

It’s primitive - but it works as a proof of concept. Now we can easily translate responsivebreakpoints.com HTML output into our Hugo shortcodes and make the image sources point to our S3 files instead.

Goal: Images Should be Lazy Loaded

So far we’ve solved the problem of serving the correct image to each device, storing the images outside of source control, and the need to have HTML in our markdown files (although our shortcode falls short of completely eliminating boilerplate). The next step is figuring out how to lazy load our images. Lazy loading is the process of only loading images when necessary as opposed to on page loading. It’s achieved by observing the user’s viewport (and items about to enter the viewport) and then causing them to be fetched using JavaScript. The easiest way to achieve this is the fantastic lazysizes library. All we need to do is to replace the src or srcset attribute with a data attribute (data-srcset for example) and use the class lazyload. The lazyload class allows lazysizes to find images that need to be loaded when they scroll into view using a combination of mutation observers and interaction observers.

All we need to do to make this work is add import lazysizes from "lazysizes"; to our JavaScript and adjust our responsiveimage and fallbackimage shortcodes to look like this.

<source
    media="{{ .Get "media" }}"
    sizes="{{ .Get "sizes" }}"
    data-srcset="{{ .Get "srcset" }}">

<img 
    data-proofer-ignore
    class="lazyload"
    sizes="{{ .Get "sizes" }}"
    data-srcset="{{ .Get "srcset" }}"
    data-src="{{ .Get "src" }}"
    alt="{{ .Get "caption" }}">

I’ve also added a data-proofer-ignore tag to prevent getting htmltest warnings about missing src tags on images.

Goal: Images Should Work Without JavaScript

Now we’re lazyloading our images we have a new problem, if somebody has their JavaScript disabled they won’t see any images! Our only solution to fallback to if a user isn’t using JavaScript is the <noscript> tag which only renders its contents if JavaScript is disabled. Firstly we define a style tag in our theme. This hides all lazyloading images and anything with the class no-js-hidden, which will come in handy later, if JavaScript is disabled.

<noscript>
  <style>
    .no-js-hidden {
      display: none;
    }

    .lazyload {
      display: none;
    }
  </style>
</noscript>

Then we can double up all our images so we have two variants - ones that will not appear unless scripting is disabled which use lazyloading <noscript><source srcset="" ...></noscript> and ones that will be hidden if JavaScript is disabled and that do use lazyloading <picture class="no-js-hidden"><source data-srcset="" ...></picture>.

Shortcode Changes

Unfortunately, producing a noscript and regular version of a source in each individual shortcode breaks the picture tag causing it to render both the img and the source. Not ideal! Instead the solution is to have HTML that looks like the following:

<picture class="no-js-hidden">
 <source>
 <img>
</picture>
<noscript>
 <picture>
  <source>
  <img>
 </picture>
</noscript>

This means we need to rethink our shortcode philosophy and that’s not a bad thing. Our existing solution is ugly and feels more like abuse than use of the shortcode system - mainly because we’re trying to pass too much information in. It’d be great if we could store that information elsewhere. Hugo has a solution for that - data templates! They allow us to store structured data outside of our posts³ but in a way that is accessible from a shortcode.

This allows us to simplify our shortcode to something like {{< picture name="Trial By Fire" caption="I'm sure we'll see this dragon again" >}} because all the details can be stored elsewhere. To make this work our shortcode needs to become more complicated under the hood - but we’ll only need one.

{{ $caption := .Get "caption" }}
{{ range first 1 (where $.Site.Data.images "name" (.Get "name")) }}
  {{ if $caption }}
    <figure>
  {{ end }}
  <picture class="no-js-hidden">
    {{ range .sources}}
      <source
        class="no-js-hidden"
        media="{{ .media }}"
        sizes="{{ .sizes }}"
        data-srcset="{{ .srcset }}"
      >
    {{ end }}
    <img 
      data-proofer-ignore
      class="lazyload"
      sizes="{{ .fallback.sizes }}"
      data-srcset="{{ .fallback.srcset }}"
      alt="{{ $caption }}"
    >
  </picture>
  <noscript>
    <picture>
      {{ range .sources}}
        <source
          media="{{ .media }}"
          sizes="{{ .sizes }}"
          srcset="{{ .srcset }}"
        >
      {{ end }}
      <img
        src="{{.fallback.src}}"
        sizes="{{ .fallback.sizes }}"
        srcset="{{ .fallback.srcset }}"
        alt="{{ $caption }}"
      >
    </picture>
  </noscript>
  {{ if $caption }}
      <figcaption>{{ $caption }}</figcaption>
  </figure>
  {{ end }}
{{ end }}

If you notice carefully you’ll see that the real magic happens on line 2, {{ range first 1 (where $.Site.Data.images "name" (.Get "name")) }} this allows us to grab a single image data where the name in the data file matches the name passed into the shortcode. After that things look very similar, except we’re taking advantage of data structures that allow looping.

Writing a Generator: A Second Attempt

Any change to our markup means we need to change the program that builds it. In fact, in this instance we want to change our program from building a shortcode string to producing the structured data that Hugo uses in our shortcode. Despite the changed goal - the program looks very similar.

#!/usr/bin/env node

const fs = require('fs');
const program = require('commander');
const cheerio = require('cheerio');

function getHtml(program) {
    const input = fs.readFileSync(program.input);
    return cheerio.load(input);
}

function prefixSource(srcset) {
    return srcset
        .split("w,")
        .map(i => prefix + i.trim())
        .reduce((s, v, i) => s + (i > 0 ? 'w,' : '') + v, '');
}

function getFallbackImageData($) {
    const img = $('img');
    const sizes = img.attr('sizes');
    const srcset = prefixSource(img.attr('srcset'));
    const src = 'https://files.arranfrance.com/images/' + prefixSource(img.attr('src'));
    
    return {sizes, srcset, src};
}

function getPrefix(program) {
    const directory = program.directory ? program.directory[program.directory.length - 1] === '/' ? program.directory : program.directory + '/' : '';
    const year = new Date().getFullYear();
    const month = monthNames[new Date().getMonth()];
    return `https://files.arranfrance.com/images/${year}/${month}/${directory}`;
}

program
  .version('0.0.2')
  .option('-n, --name <required>', 'The image name')
  .option('-i, --input <required>','The input file')
  .option('-l, --location [optional]','The location of the data file. Defaults to ./data/images.json')
  .option('-d, --directory [optional]','The directory suffix that S3 files are located in')
  .parse(process.argv);

const $ = getHtml(program);

const monthNames = ["Jan", "Feb", "Mar", "Apr", "May", "Jun",
  "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"
];

const prefix = getPrefix(program);
const fallbackImage = getFallbackImageData($);

const sources = [];
$('source').each((i, elem) => {
    const source = $(elem);
    const media = source.attr('media');
    const sizes = source.attr('sizes');
    const srcset = prefixSource(source.attr('srcset'));
    const split = srcset.split('/')
    const filename = split[split.length - 1].split(' ')[0];
    sources.push({media, sizes, srcset});
});

const data = {
    name: program.name,
    fallback: fallbackImage, 
    sources: sources
};

const outputLocation = program.location ? program.location : './data/images.json'

const images = JSON.parse(fs.readFileSync(outputLocation));
const existingData = images.find(a => a.name === data.name);
if (existingData) {
    existingData = data;
} else {
    images.push(data);
}
fs.writeFileSync(outputLocation, JSON.stringify(existingData));

This script looks a little more grown up than our previous one. Instead of copying and pasting values we’re taking command line arguments using commander and reading/writing to files. The important difference is that instead of building a shortcode string we’re building a JSON data structure.

Goal: A Placeholder Should be Displayed Whilst Images Load

When lazyloading images it’s nice to be able to display a placeholder image for users. There’s been a lot written about this technique elsewhere after it was popularized by Facebook and Medium, so I won’t recap in much detail. The goal is to let the user know something is loading and to avoid the user seeing a shift in content as the image is loaded. The trade-off is the need to load in a placeholder as well as the image, but most placeholders are in the order of 800-1000 bytes, small enough to be irrelevant. SQIP is a technique that uses SVGs as a placeholder. The SVG is made to be a geometric approximation of the image which is then heavily blurred, passed through SVGO, and then encoded in base 64 so it can be inlined as a placeholder. The technique is backed by a robust JavaScript library to produce the SVGs which makes implementing the technique a breeze.

The first step to implementing the technique is to generate a placeholder for each image variant. I’ve opted here to only show the relevant change, the final program in its entirety is available on GitHub.

const sqip = require('sqip');
// <snip>
$('source').each((i, elem) => {
  const placeholder = sqip({
        filename,
        numberOfPrimitives: 10
  });
  sources.push({media, sizes, srcset, placeholder: placeholder.svg_base64encoded});
}

Then we can adjust our shortcode to use the new placeholder property of our JSON.

{{ $caption := .Get "caption" }}
{{ range first 1 (where $.Site.Data.images "name" (.Get "name")) }}
  {{ if $caption }}
    <figure>
  {{ end }}
  <picture class="no-js-hidden">
    {{ range .sources}}
      <source
        class="no-js-hidden"
        media="{{ .media }}"
        sizes="{{ .sizes }}"
        {{ with .placeholder }} srcset="data:image/svg+xml;base64,{{.}}" {{ end }}
        data-srcset="{{ .srcset }}"
      >
    {{ end }}
    <img 
      data-proofer-ignore
      class="lazyload"
      sizes="{{ .fallback.sizes }}"
      data-srcset="{{ .fallback.srcset }}"
      src="data:image/svg+xml;base64,{{ .fallback.placeholder }}"
      alt="{{ $caption }}"
    >
  </picture>
  <noscript>
    <picture>
      {{ range .sources}}
        <source
          media="{{ .media }}"
          sizes="{{ .sizes }}"
          {{ with .placeholder }} style="background-size: cover; background-image: url(data:image/svg+xml;base64,{{.}})" {{ end }}
          srcset="{{ .srcset }}"
        >
      {{ end }}
      <img
        src="{{.fallback.src}}"
        sizes="{{ .fallback.sizes }}"
        srcset="{{ .fallback.srcset }}"
        style="background-size: cover; background-image: url(data:image/svg+xml;base64,{{ .fallback.placeholder }})"
        alt="{{ $caption }}"
      >
    </picture>
  </noscript>
  {{ if $caption }}
      <figcaption>{{ $caption }}</figcaption>
  </figure>
  {{ end }}
{{ end }}

When JavaScript is enabled the placeholder is used as a src that is eventually replaced by the data-src when lazy loading occurs. When JavaScript is disabled, the placeholder is used as a background image whilst the actual image loads in.

Future Goals

Whilst I’ve automated some steps for using an image in this blog there are many smaller steps that are unscripted.

The first step of compressing an image is performed manually by uploading an image to TinyPNG. I also manually upload images to responsivebreakpoints.com, upload image variants to AWS S3, and manually shuffle files around my local file system. All of these steps could be scripted! For now though, the process is good enough.