A Tale of Brotli Compression

Providing a better customer experience has lately become the centre of business strategies and in order to provide better customer experience, it is necessary to load your website in the shortest duration possible.

“According to Google, 53% of mobile site visitors leave a page that takes longer than three seconds to load. The probability of bounce rate reaches a staggering 106% as the page load time increases from 1s to 6s”

Considering this fact you need to use a number of different strategies to load your website faster. There might be many solutions possible one fast and quick way to load a website fast is compression. Compression is the process of reducing the size of assets so they take less time to transmit over the network. Google released a new compression algorithm Brotli, it provides better compression results than the current standard (Gzip) and test results have shown a 20–26% higher compression ratio when comparing Brotli to Zopfli (another modern compression algorithm).

There are two ways to compress your assets:

i) Statically — Generate Brotli and Gzip compressed assets at build time and serve the Brotli assets to all the users who have a browser which supports Brotli otherwise fallback to Gzip.

ii) Dynamically — At the runtime compress the requested assets based on the Browser. If the browser supports Brotli, use Brotli as an algorithm for compression while sending the response otherwise fallback to gzip.

Static Generation

Setup will include the following components

1. Express server for serving the assets.
2. Webpack as module bundler.
3. Docker as our tool to create, deploy and run applications by using containers.
4. AWS EC2.
5. Cloudfront as our CDN.

This is a typical setup which most of us might have, Express as a Frontend server for SSR(Server-Side rendering) which is used to serve a server-side rendered HTML for the first time a user lands on the website. Webpack is the most famous and commonly used module bundler. We will run Express server in the docker container, host these containers in EC2 instances and have CloudFront in front of our EC2 setup to cache our assets.

Setting up Webpack for Brotli

There are two plugins we need to use for the generation of static assets, brotli-webpack-plugin and compression-webpack-plugin. Compression plugin is used to generate gzip assets and Brotli plugin is used to generate Brotli compressed assets.

Once the assets have been generated the next part is to serve them. At runtime, when you are serving a request you need to check if the browser from which the request has originated does it support Brotli compression. Browsers send Accept-Encoding headers which indicate if the browser supports Brotli or not. To simplify the request serving part we will use an express plugin express-static-gzip which will do all this for us.

Now the only part left is to cache our assets, CloudFront is used to cache our assets, it is just a key-value pair where each key is a unique identifier for each asset. For better-caching Cloudfront normalizes the Accept-Encoding header by stripping off all the encoding types except gzip and Express server always serves gzip assets. So irrespective of the type of the browser it will serve all browsers the same gzip assets. This is good but what we want CloudFront to do is to serve Brotliassets to browsers which support Brotli otherwise fallback to gzip . To do this you need to whitelist Accept-Encoding header in CloudFront, doing so will make CloudFront consider the Accept-Encoding header while making a unique key for each asset.

Results

1. Size: Webpack will generate two compressed files for each asset, a gzip version and a brotli version. As you can see brotli compressed version is ~25% smaller than its gzip counterpart.

2. Time: Let’s see the before and after images of how long does it take to load the same assets.

Before Adding Brotli Compression

Assets decreased by ~25% after adding Brolti compression

3. Performance: Lighthouse is the most commonly used tool for measuring the performance of a website, below are the before and after audits done by lighthouse. Just by serving brotli compressed assets we are able to gain 10 points on performance.

Performance before enabling Brotli Compression

Performance increased by 13 points after adding Brotli compression.

Dynamic Generation

Serving brotli compressed assets at runtime requires you to look at the Accept-Encodingrequest header, if brotli is present as encoding format compress the response using brotli otherwise fallback to gzip

For compressing at runtime we will use an express middleware shrink-ray-current it is a maintained version of shrink-ray.

Results: Similar results were observed in terms of Size, Time and Performance.

Gotcha’s around runtime compression

Build Size: Generating brotli compressed assets on runtime leads to a smaller bundle size as number of assets generated are less. Runtime compression is also easy to use with Differential Builds. For serving differential builds, two builds are generated. One for modern browsers and the other for legacy browsers. If we do not use runtime compression then with differential builds the size of build will be huge, as for each file there will be 6 variations. For example — For runtime.js following files will be generated

a) runtime.mjs(For modern browsers)

b) runtime.mjs.br(Brotli compressed version),

c) runtime.mjs.gz(gzipped version)

d) runtime.js(For legacy browsers)

e) runtime.js.br(Brotli compressed version)

f) runtime.mjs.gz(gzipped version).

Increased load on servers: For each resource to being served, server will use some compute power as it will be compressing that resource at runtime. This can be tackled by using a CDN for storing the compressed assets, so effectively the load on the servers is for a certain amount of time after the deployment is made.