Brotli compression is a generic-purpose compression technique widely supported across browsers. It’s comparable to the currently available compression methods as it offers 20-26% better compression ratios. Nevertheless, it’s no good unless the webserver is sending compressed text-based resources with the Brotli algorithm.
In this article, we will learn how compression works in the server and why is it useful? We will also learn to install the Nginx server and get our server to provide Brotli compressed files.
Compression techniques/algorithms improve website performance by reducing the content size. Hence the compressed data takes a low load and transfer time. However, it has a price. Servers utilize a lot of computational resources to provide a better compression rate. Hence, the better, the expensive. So a great deal of effort is added to improve compression formats while utilizing minimum CPU cycles.
By now, the most potential compression format was gzipped. Recently gzip is replaced by a new compression algorithm known as Brotli. It is an advanced compression algorithm composed of Huffman coding, the L77 algorithm, and context modeling. In contrast, Gzip is built on the Deflate algorithm.
The lossless compression format, designed by Google, is closely related to deflate compression format. Both of the compression methods use sliding windows for back referencing. The Brotli sliding window size ranges from 1 KB to 16MB. In contrast, Gzip has a fixed window size of 32KB. That means Brotli’s window is 512 times larger than the deflate window, which isn’t relevant as text files larger than 32 KB are rarely on web servers.
Server Compression Compatibility is Important
Whenever we download a file from the browser, the browser requests the server what kind of compression it supports through a header. For instance, if the browser supports gzip and deflate to decompress. It will add these options in its Accept-Encoding, header, i.e.,
Hence the browsers that don’t support these formats will not include them in the header. When the server responds with the content, it tells the browser about the compression format through a header, Content-Encoding. Hence, if it supports gzip, then the header looks like this:
Headers of the browsers like Firefox that support Brotli compression and the webserver that have a Brotli module installed to look like these:
Accept-Encoding=”deflate, gzip, br”
Hence, if the browser utilizes the best compression format and the web server does not, it’s no good, as the web server won’t send back the files with the preferred compression algorithm. That’s why it is important to install the compression module for the webserver.
Before moving forward with the Brotli configuration, we will set up our Nginx server. Before that sudo apt-get update your Ubuntu distribution and type in the following commands in your bash terminal.
To enable Brotli compression in the Nginx, we will compile our .so modules as per our Nginx version details. As shown, typing the following command will output the Nginx version:
[email protected]:~$ nginx -v
nginx version: nginx/1.18.0 (Ubuntu)
Use the wget command along with your nginx version detail to download the source code from the Nginx website.
[email protected]:~$ wget https://nginx.org/download/nginx-1.18.0.tar.gz
–2021-02-07 02:57:33– https://nginx.org/download/nginx-1.18.0.tar.gz
Resolving nginx.org (nginx.org)… 188.8.131.52, 184.108.40.206, 2a05:d014:edb:5702::6, …
Connecting to nginx.org (nginx.org)|220.127.116.11|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 1039530 (1015K) [application/octet-stream]
Saving to: ‘nginx-1.18.0.tar.gz’
nginx-1.18.0.tar.gz 100%[==================================================================>] 1015K 220KB/s in 4.8s
2021-02-07 02:57:38 (212 KB/s) – ‘nginx-1.18.0.tar.gz’ saved [1039530/1039530]
We will use this source code to compile *.so binaries for Brotli compression. Now extract the file using the following command.
[email protected]:~$ tar xzf nginx-1.18.0.tar.gz
Brotli Module Configuration
Now Google has released the Brotli module for Nginx. We will git-clone the module from the Google repository.
[email protected]:~$ git clone https://github.com/google/ngx_brotli –recursive.
We will cd into the nginx-1.18.0 folder to configure the dynamic Brotli module.
Note: You may receive the following error while configuring
./configure: error: the HTTP rewrite module requires the PCRE library.
You can either disable the module by using –without-http_rewrite_module
option, or install the PCRE library into the system, or build the PCRE library
statically from the source with nginx by using –with-pcre= option.
In that case, run the following command to install the pcre library
[email protected]:~$ sudo apt-get install libpcre3-dev -y
We will use the make command to create a modules folder inside the nginx-1.18.0 directory.
[email protected]:~$ sudo make modules
We use the cp command to copy ngx_http_brotli*.so files from the nginx-1.18.0/objs folder to the modules folder.
Now list the content of the files using the ls command. You will notice that it consists of two different module files, i.e.:
[email protected]:~$ ls ngx_http_brotli*.so
- Regular Brotli Module: The ngx_http_brotli_filter_module.so module compresses all the files on the fly, and hence it requires more computational resources
- Static Brotli Module: The ngx_http_brotli_static_module.so module allows it to serve pre-compressed static files, hence less resource-intensive.
Now use your favorite editor to open the /etc/nginx/nginx.conf file to add Brotli load modules to begin Brotli configuration by including the following lines:
[email protected]:~$ sudo vim /etc/nginx/nginx.conf
# Load module section
We will also include configuration folders paths /etc/nginx/conf.d/*.conf
and /usr/share/nginx/modules/*.conf in the above file such as:
# Include configs folders
To add the Brotli configuration open the /etc/nginx/conf.d/brotli.conf
file in the vim editor and enable Brotli by setting the following configuration directives:
brotli_types application/rss xml application/xhtml xml
The “brotli off|on” value enables or disables dynamic or on the fly compression of the content.
The ‘brotli_ static on’ enables the Nginx server to check if the pre-compressed files with the .br extensions exist or not. We can also turn this setting into an option off or always. The always value allows the server to send pre-compressed content without confirming if the browser supports it or not. Since Brotli is resource-intensive, this module is best suited to reduce the bottleneck situations.
The “brotli_comp_level 6” directive sets the dynamic compression quality level to 6. It can range from 0 to 11.
Lastly, enable dynamic compression for specific MIME types, whereas text/html responses are always compressed. The default syntax for this directive is brotli_types [mime type]. You can find more about the configuration directive on Github.
Save the changes, restart the Nginx service by typing “sudo service restart nginx” and it’s all done.
After the changes, you will notice some obvious improvements in the performance metrics. However, it does come with a slight drawback of increased CPU load at peak times. To avoid such situations keep an eye on CPU usage; if it reaches 100% regularly, we can utilize many options as per our preferences, such as presenting pre-compressed or static content, lowering compression level, and turning off on-the-fly compression, among many.
About the author
<img alt="Usama Azad" data-lazy-src="https://kirelos.com/wp-content/uploads/2021/03/echo/usama-1-150×150.jpg" height="112" src="data:image/svg xml,” width=”112″>
A security enthusiast who loves Terminal and Open Source. My area of expertise is Python, Linux (Debian), Bash, Penetration testing, and Firewalls. I’m born and raised in Wazirabad, Pakistan and currently doing Undergraduation from National University of Science and Technology (NUST). On Twitter i go by @UsamaAzad14