Compression 2021 chapter contents #2574

lvandeve · 2021-11-22T16:46:58Z

src/content/en/2021/compression.md

lvandeve · 2021-11-25T10:14:49Z

Added the bios, PTAL

lvandeve · 2021-11-26T16:54:41Z

I pushed another update to add the png images generated by the script, fix a few image links, and updated the style of the compression level images to match the style of the other charts better

rviscomi

LGTM. @bazzadp any comments before we merge?

tunetheweb · 2021-11-27T17:59:45Z

We're missing a screenshot for this:

{{ figure_markup(
  image="content-encoding.png",
  caption='Use DevTools to check if content encoding is used on your site',
  description="Image showing how to use DevTools to see if content encoding is used.",
  width=591,
  height=939
  )
}}

I presume the intent was to reuse the 2020 image for this so will fix the markup to do that.

Also going to fix some minor issues with the graphs (font sizes, labels and inconsistent decimal places) and will add to this PR before we merge so copy editing can concentrate just on the copy.

lvandeve · 2021-11-27T19:01:13Z

I presume the intent was to reuse the 2020 image for this so will fix the markup to do that.

Indeed it was, thank you for fixing this

tunetheweb · 2021-11-28T11:04:58Z

It is disappointing how much of this content is a straight copy from the 2020 chapter which was already a light chapter.

Most of the chapter (including the opening and the conclusion) is word for word identical to the 2020 chapter. While the new insights on compression levels are very interesting, I do not think the rest of the chapter is sufficiently different to warrant publishing this as part of the 2021 Web Almanac. To do so would reflect badly on the other chapters who have made significant efforts to review, rewrite and reinterpret their topics to give a fresh chapter for this year.

Therefore I am not going to merge this, and given our launch date on this Wednesday the 1st December, I will unfortunately be left with no choice but to close this PR and remove the chapter from the 2021 edition, if the chapter is not significantly differentiated from the 2020 edition by the end of tomorrow (Monday 29th November). Please let me know if you intend to rewrite the duplicated content by then.

lvandeve · 2021-11-28T19:36:14Z

Yes, I will differentiate it. The same (but updated) data as last year is used for many sections to allow comparing with last year. The section about compression levels is new and significantly different.

@tunetheweb please let me know if rewriting the word for word sections, but keeping the content they describe the same (that is, describe similar data as last year but with the updated numbers) since that was reviewed, would be acceptable

tunetheweb · 2021-11-28T20:07:21Z

Yes reviewing and rewriting the identical content, even if the message stays largely the same is acceptable (assuming that is what the data is telling you). Straight copying and pasting with only minor tweaks is not.

rviscomi

Thanks @lvandeve. I hadn't realized that the content was so close to 2020's chapter, so I agree with Barry's suggestion to rewrite it to be more unique this year. If completing it by tomorrow isn't feasible let's find a timeframe that works for everyone.

lvandeve · 2021-11-28T23:45:17Z

Sorry about that.

I have re-worded the sections "introduction", "how to analyze compression on your site" and "conclusion" and the first paragraph of "What type of content should we compress?". I also replaced the chrome developer tools screenshot in the analyze section (content-encoding.png) with a new updated one. The conclusion contains some points that are the same as last year, but this also genuinely is the case.

Still to re-word are: parts of "how to use http compression", "first party vs third party compression", and a few sentences of "How to improve on compression". I'll also ask several people to review the updated text.

"Compression levels" and most of "How to improve on compression" are new and so don't need this, and "Current state of HTTP compression" was also already very different because it now compares the least 3 years.

Re-words the text of the sections Introduction, How to analyze compression on your sites and Conclusion Adds a new updated image for the content encoding in the developer tools

@mo271

Changing the emphasis of several chapters. Done together with Moritz (@mo271)

lvandeve · 2021-11-29T13:04:32Z

Went through the text together with Moritz and changed the emphasis of the affected sections, none of the text is a copy of last year anymore

tunetheweb · 2021-11-29T15:09:21Z

Thanks you very much for actioning this so quickly @lvandeve . It looks a lot better now and we're good to merge. Just a quick copy edit after in a new PR and we'll be done.

src/content/en/2021/compression.md

fischbacher · 2021-11-29T16:31:13Z

src/content/en/2021/compression.md

-## TODO
+## Introduction
+
+When loading websites it is important to make the best use of the available bandwidth. HTTP allows to compress the content that it delivers and this decreases the time needed to transfer the content. Even when taking the time to compress and decompress the content into account, this often leads to significant improvement in the user experience, it can reduce [page weight](./page-weight), improve [web performance](./performance) and boost search rankings, so it's an important part of [search engine optimization](./seo). On the other hand, not using compression, or using it with a suboptimal configuration, can result in longer wait times for the user and can negatively affect the growth of a web service.


Missing comma: "When loading websites, it is ..."

Perhaps better: "When loading web content, ..."

fischbacher

First pass over the text.

fischbacher · 2021-11-29T16:34:01Z

src/content/en/2021/compression.md

+{{ figure_markup(
+  image="compession-methods-by-content-type.png",
+  caption="Compression methods for different content types",
+  description="A stacked bar chart showing the usage rate of different compression algorithms broken down by the content type. The stacked bars divide up the use of Brotli, Gzip, and no compression. `text/html` is the only content type that is compressed less than 50% of the time. `application/json` is approximately 68% compressed, `image/svg+xml` approximately 64%. `text/css` and `application/javascript` are each over 85% compressed. `application/x-javascript` and `text/javascript` are greater than 90% compressed.",


Note that "68% compressed" can be misread as "compressed to 68% of its original size" or "compressed to 32% of its original size", whereas here, the context suggests that this means the percentage of documents compressed.

fischbacher · 2021-11-29T16:34:26Z

src/content/en/2021/compression.md

+  )
+}}
+
+Compared to the other content type, `text/plain` and `text/html` uses the least amount of compression, with merely 12% and 14% using compression at all. This might be because `text/html` is more often dynamically generated than static content such as JavaScript and CSS.


s/content type/content types/

fischbacher · 2021-11-29T16:36:57Z

src/content/en/2021/compression.md

+
+## Server settings for HTTP compression
+
+For HTTP content encoding, the HTTP standard defines the [Accept-Encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding) request header, with which a HTTP client can announce the content encodings it can handle to the server. The server\'s response can then contain a [Content-Encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding) header field that specifies which of the encodings was chosen to transform the data in the response body.


Better:

"with which a HTTP client can announce to the server what content encodings it can handle."

fischbacher · 2021-11-29T16:38:00Z

src/content/en/2021/compression.md

+ - for precompressed (static) content: this content is already compressed beforehand, ideally with the highest level possible, and the web server should be set up to find the appropriate compressed files based on the filename extension, for example.
+ - for dynamic content, which is compressed on the fly for each request by the web server (or a plugin) itself, for dynamically generated text content
+
+When compressing text with brotli or gzip it is possible to select different compression levels. Higher compression levels will result in smaller compressed files, but take a longer time to compress. During decompression CPU usage tends not to be higher for more heavily compressed files, rather files that are compressed with a higher compression level are slightly faster to decode.


Comma here: "During decompression, CPU usage tends..."

Also, "Rather, ..." should start a new sentence.

fischbacher · 2021-11-29T16:41:17Z

src/content/en/2021/compression.md

+
+Compression level is a parameter given to the encoder adjusting the amount of effort the encoder is applying to find redundancy in the input, to consequently achieve higher compression density.
+
+Brotli encoding allows compression levels from 0 to 11, while Gzip uses levels 1 to 9 as well as higher optimal compression such as with Zopfli, indicated `opt` in the graph below. A higher compression level results in slower compression, but does not substantially affect the decompression speed, even making it slightly faster.


Can we split this sentence and move "Zopfli" to its own sentence, such as perhaps:

"On top of that, the Zopfli compressor..."

fischbacher · 2021-11-29T16:46:11Z

src/content/en/2021/compression.md

+
+ A different tool that can analyze compression on your site is Google's <a hreflang="en" href="https://developers.google.com/web/tools/lighthouse">Lighthouse</a> tool. It runs a series of audits, including the <a hreflang="en" href="https://web.dev/uses-text-compression/">"Enable text compression" audit</a>. This audit attemps to compress resources to check if they reduced by at least 10% and 1,400 bytes. Depending on the score, it can show a compression recommendation in the results, with a list of the resources that can be compressed to benefit your website.
+
+The HTTP Archive [runs Lighthouse audits](./methodology#lighthouse) for each mobile page, and from this data we can see that 72% of websites pass this audit. This is 2% less than [last year's](../2020/compression#identifying-compression-opportunities) 74% which is, despite more usage of text compression overall compared to last year, a slight drop.


s/for each/for every/ - but technically, both are permissible here.

fischbacher · 2021-11-29T16:47:29Z

src/content/en/2021/compression.md

+
+After having the minimal form of the content you want to transmit, the next step is to ensure compression is enabled. You can verify it is enabled as highlighted in the previous section, and configure your web server if needed.
+
+When using only Gzip compression (also known as Deflate or Zlib), you can add support for Brotli. Brotli compresses to <a hreflang="en" href="https://quixdb.github.io/squash-benchmark/">smaller files at the same speed</a>, decompresses at the same speed, and is widely supported: <a hreflang="en" href="https://caniuse.com/?search=brotli">can I use Brotli</a>.


s/Brotli compresses to/In comparison to Gzip, Brotli compresses to/

fischbacher · 2021-11-29T16:49:02Z

src/content/en/2021/compression.md

+
+You can choose a well-tuned compression level. What compression level is right for your application might depend on multiple factors, but keep in mind that a more heavily compressed text file does not need more CPU when decoding, so for precompressed assets there's no drawback from the user's perspective to set the compression levels as high as possible. For dynamic compression, we have to make sure that the user doesn't have to wait longer for a more heavily compressed file, taking both the time it takes to compress as well as the potentially decreased transmission time into account. This difference is borne out when looking at compression level recommendations for both methods.
+
+When using Gzip compression for precompressed resources, consider using [Zopfli](https://en.wikipedia.org/wiki/Zopfli), which generates smaller Gzip compatible files. Zopfli uses an iterative approach to find an optimal parsing, leading to 3--8% denser output, but taking substantially longer to compute, whereas Gzip uses a more straightforward but less effective approach. See <a hreflang="en" href="https://cran.r-project.org/web/packages/brotli/vignettes/brotli-2015-09-22.pdf">this comparison between multiple compressors</a>, and this <a hreflang="en" href="https://blog.codinghorror.com/zopfli-optimization-literally-free-bandwidth/">comparison between Gzip and Zopfli</a> that takes into account different compression levels for Gzip.


Strictly speaking, "Zopfli" compression is not optimal, so let's perhaps say "to find a very compact parsing"

fischbacher · 2021-11-29T16:50:10Z

src/content/en/2021/compression.md

+
+The analysis on compression levels reveals that about 6% of Gzip-compressed content uses more advanced compressors such as Zopfli, while a similar \'optimal parsing\' approach is used for 17% of Brotli-compressed content. This indicates that when much slower but slightly more efficient methods are available, a significant amount of users will deploy these methods for their static content such as JavaScript.
+
+Usage of HTTP compression continous to grow, and especially Brotli has increased significantly compared to the [previous year's Almanac](../2020/compression). The number of HTTP responses using any text compression increased 2%, while Brotli increased over 4%. Despite the increase, we still see opportunities to use more HTTP compression by tweaking compression settings of servers. You can benefit from taking a closer look at your own website's responses and your server configuration for the compression methods and their level. Tuning the default compression settings for the most popular HTTP servers could have a great impact for users.


Typo: s/continuous/continues/

fischbacher · 2021-11-29T16:51:11Z

src/content/en/2021/compression.md

+
+The analysis on compression levels reveals that about 6% of Gzip-compressed content uses more advanced compressors such as Zopfli, while a similar \'optimal parsing\' approach is used for 17% of Brotli-compressed content. This indicates that when much slower but slightly more efficient methods are available, a significant amount of users will deploy these methods for their static content such as JavaScript.
+
+Usage of HTTP compression continous to grow, and especially Brotli has increased significantly compared to the [previous year's Almanac](../2020/compression). The number of HTTP responses using any text compression increased 2%, while Brotli increased over 4%. Despite the increase, we still see opportunities to use more HTTP compression by tweaking compression settings of servers. You can benefit from taking a closer look at your own website's responses and your server configuration for the compression methods and their level. Tuning the default compression settings for the most popular HTTP servers could have a great impact for users.


"increased x%" is ambiguous. It's better to say something like "increased by so-and-so-many percentage points".

lvandeve requested a review from rviscomi November 22, 2021 16:47

lvandeve force-pushed the compression branch from ed50a73 to 3e06417 Compare November 22, 2021 16:57

rviscomi added the writing Related to wording and content label Nov 23, 2021

rviscomi added this to the 2021 Content Writing milestone Nov 23, 2021

rviscomi requested changes Nov 23, 2021

View reviewed changes

src/content/en/2021/compression.md Show resolved Hide resolved

rviscomi assigned mo271, lvandeve and jyrkialakuijala Nov 23, 2021

lvandeve force-pushed the compression branch from 3e06417 to ce22883 Compare November 23, 2021 17:01

lvandeve force-pushed the compression branch from ce22883 to 7ba7b74 Compare November 25, 2021 11:34

Compression 2021 chapter contents

747b283

lvandeve force-pushed the compression branch from 7ba7b74 to 747b283 Compare November 26, 2021 16:53

lvandeve requested a review from rviscomi November 26, 2021 20:14

rviscomi approved these changes Nov 27, 2021

View reviewed changes

Merge remote-tracking branch 'upstream/main' into compression

110fbdf

Figure fix ups

559d9d9

tunetheweb added 2 commits November 27, 2021 22:21

Add unedited label

aaf52f9

Line endings

7897ed2

rviscomi requested changes Nov 28, 2021

View reviewed changes

Update introduction, analyze and conclusion text

071db03

Re-words the text of the sections Introduction, How to analyze compression on your sites and Conclusion Adds a new updated image for the content encoding in the developer tools

lvandeve force-pushed the compression branch from d4b0355 to 071db03 Compare November 28, 2021 23:55

Restructuring

ba65a93

Changing the emphasis of several chapters. Done together with Moritz (@mo271)

tunetheweb added the ASAP This issue is blocking progress label Nov 29, 2021

tunetheweb approved these changes Nov 29, 2021

View reviewed changes

tunetheweb merged commit 645df53 into HTTPArchive:main Nov 29, 2021

fischbacher reviewed Nov 29, 2021

View reviewed changes

src/content/en/2021/compression.md Show resolved Hide resolved

fischbacher reviewed Nov 29, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compression 2021 chapter contents #2574

Compression 2021 chapter contents #2574

lvandeve commented Nov 22, 2021 •

edited by rviscomi

Loading

lvandeve commented Nov 25, 2021

lvandeve commented Nov 26, 2021

rviscomi left a comment

tunetheweb commented Nov 27, 2021

lvandeve commented Nov 27, 2021

tunetheweb commented Nov 28, 2021

lvandeve commented Nov 28, 2021 •

edited

Loading

tunetheweb commented Nov 28, 2021

rviscomi left a comment

lvandeve commented Nov 28, 2021 •

edited

Loading

lvandeve commented Nov 29, 2021

tunetheweb commented Nov 29, 2021

fischbacher Nov 29, 2021 •

edited

Loading

fischbacher left a comment

fischbacher Nov 29, 2021

fischbacher Nov 29, 2021

fischbacher Nov 29, 2021

fischbacher Nov 29, 2021

fischbacher Nov 29, 2021

fischbacher Nov 29, 2021

fischbacher Nov 29, 2021

fischbacher Nov 29, 2021

fischbacher Nov 29, 2021

fischbacher Nov 29, 2021


		## Server settings for HTTP compression

		For HTTP content encoding, the HTTP standard defines the [Accept-Encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding) request header, with which a HTTP client can announce the content encodings it can handle to the server. The server\'s response can then contain a [Content-Encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding) header field that specifies which of the encodings was chosen to transform the data in the response body.


		Compression level is a parameter given to the encoder adjusting the amount of effort the encoder is applying to find redundancy in the input, to consequently achieve higher compression density.

		Brotli encoding allows compression levels from 0 to 11, while Gzip uses levels 1 to 9 as well as higher optimal compression such as with Zopfli, indicated `opt` in the graph below. A higher compression level results in slower compression, but does not substantially affect the decompression speed, even making it slightly faster.


		A different tool that can analyze compression on your site is Google's <a hreflang="en" href="https://developers.google.com/web/tools/lighthouse">Lighthouse</a> tool. It runs a series of audits, including the <a hreflang="en" href="https://web.dev/uses-text-compression/">"Enable text compression" audit</a>. This audit attemps to compress resources to check if they reduced by at least 10% and 1,400 bytes. Depending on the score, it can show a compression recommendation in the results, with a list of the resources that can be compressed to benefit your website.

		The HTTP Archive [runs Lighthouse audits](./methodology#lighthouse) for each mobile page, and from this data we can see that 72% of websites pass this audit. This is 2% less than [last year's](../2020/compression#identifying-compression-opportunities) 74% which is, despite more usage of text compression overall compared to last year, a slight drop.


		After having the minimal form of the content you want to transmit, the next step is to ensure compression is enabled. You can verify it is enabled as highlighted in the previous section, and configure your web server if needed.

		When using only Gzip compression (also known as Deflate or Zlib), you can add support for Brotli. Brotli compresses to <a hreflang="en" href="https://quixdb.github.io/squash-benchmark/">smaller files at the same speed</a>, decompresses at the same speed, and is widely supported: <a hreflang="en" href="https://caniuse.com/?search=brotli">can I use Brotli</a>.


		You can choose a well-tuned compression level. What compression level is right for your application might depend on multiple factors, but keep in mind that a more heavily compressed text file does not need more CPU when decoding, so for precompressed assets there's no drawback from the user's perspective to set the compression levels as high as possible. For dynamic compression, we have to make sure that the user doesn't have to wait longer for a more heavily compressed file, taking both the time it takes to compress as well as the potentially decreased transmission time into account. This difference is borne out when looking at compression level recommendations for both methods.

		When using Gzip compression for precompressed resources, consider using [Zopfli](https://en.wikipedia.org/wiki/Zopfli), which generates smaller Gzip compatible files. Zopfli uses an iterative approach to find an optimal parsing, leading to 3--8% denser output, but taking substantially longer to compute, whereas Gzip uses a more straightforward but less effective approach. See <a hreflang="en" href="https://cran.r-project.org/web/packages/brotli/vignettes/brotli-2015-09-22.pdf">this comparison between multiple compressors</a>, and this <a hreflang="en" href="https://blog.codinghorror.com/zopfli-optimization-literally-free-bandwidth/">comparison between Gzip and Zopfli</a> that takes into account different compression levels for Gzip.


		The analysis on compression levels reveals that about 6% of Gzip-compressed content uses more advanced compressors such as Zopfli, while a similar \'optimal parsing\' approach is used for 17% of Brotli-compressed content. This indicates that when much slower but slightly more efficient methods are available, a significant amount of users will deploy these methods for their static content such as JavaScript.

		Usage of HTTP compression continous to grow, and especially Brotli has increased significantly compared to the [previous year's Almanac](../2020/compression). The number of HTTP responses using any text compression increased 2%, while Brotli increased over 4%. Despite the increase, we still see opportunities to use more HTTP compression by tweaking compression settings of servers. You can benefit from taking a closer look at your own website's responses and your server configuration for the compression methods and their level. Tuning the default compression settings for the most popular HTTP servers could have a great impact for users.

Compression 2021 chapter contents #2574

Compression 2021 chapter contents #2574

Conversation

lvandeve commented Nov 22, 2021 • edited by rviscomi Loading

lvandeve commented Nov 25, 2021

lvandeve commented Nov 26, 2021

rviscomi left a comment

Choose a reason for hiding this comment

tunetheweb commented Nov 27, 2021

lvandeve commented Nov 27, 2021

tunetheweb commented Nov 28, 2021

lvandeve commented Nov 28, 2021 • edited Loading

tunetheweb commented Nov 28, 2021

rviscomi left a comment

Choose a reason for hiding this comment

lvandeve commented Nov 28, 2021 • edited Loading

lvandeve commented Nov 29, 2021

tunetheweb commented Nov 29, 2021

fischbacher Nov 29, 2021 • edited Loading

Choose a reason for hiding this comment

fischbacher left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lvandeve commented Nov 22, 2021 •

edited by rviscomi

Loading

lvandeve commented Nov 28, 2021 •

edited

Loading

lvandeve commented Nov 28, 2021 •

edited

Loading

fischbacher Nov 29, 2021 •

edited

Loading