Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use identity as default value for Accept-Encoding if it is blank or not provided #110

Merged
merged 1 commit into from
Jul 6, 2020

Conversation

dobesv
Copy link
Contributor

@dobesv dobesv commented May 14, 2020

I noticed that the existing test for "no Accept-Encoding" header was not valid because the library used to test was adding a default value for it. The test in this PR fails because of that.

I have a PR on superagent to allow unsetting the Accept-Encoding header, I suppose once that is fixed this PR could be updated.

Or there might be another hack to make the test work properly.

@jonathanong jonathanong self-assigned this May 14, 2020
@@ -61,7 +61,7 @@ module.exports = (options = {}) => {
const encodings = new Encodings({
preferredEncodings
})
encodings.parseAcceptEncoding(ctx.request.headers['accept-encoding'] || undefined)
encodings.parseAcceptEncoding(ctx.request.headers['accept-encoding'] || 'identity')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the default is not actually identity but *. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding#Directives

* Matches any content encoding not already listed in the header. This is the default value if the header is not present. It doesn't mean that any algorithm is supported; merely that no preference is expressed.

Copy link
Member

@jonathanong jonathanong May 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm happy to make an option to set a default value though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found it really confusing when I used curl to hit my server and got back compressed responses even though I didn't ask for them.

Even if the spec says that the default is "don't care" I think on the server side it's better to play it safe and only use compression formats explicitly supported by the client.

I think HTTP clients that support compression will almost always send the header, and ones that do not support compression will probably almost never send it, so it makes more sense to me that way (assume identity).

Anyway, I've made my case :) Do with it what you will.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i agree, that spec is confusing

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonathanong

FWIW, this does seem to be a different behavior than was present in v3 of this library. Since the previous version of the library used koa's ctx.acceptsEncodings, it means this behavior differs from koa (and for that matter from express). So I would assume the confusion largely comes from that.

The previous version ultimately used negotiator, which always adds identity to the list of accepted encodings: https://github.com/jshttp/negotiator/blob/master/lib/encoding.js#L46-L56

If the new behavior is more correct according to the spec, maybe its worth explicitly calling that out since its a departure from previous versions?

@dobesv
Copy link
Contributor Author

dobesv commented May 14, 2020

Since this might be the wrong solution, I filed an issue to track this:

#112

Then any new solution can refer back to that issue.

@uhop
Copy link
Contributor

uhop commented Jun 17, 2020

My 2c: the spec is confusing. From my experience, all browsers send Accept-Encoding, and when nothing is sent (curl, wget) users usually assume that the content is unencoded.

A sidenote:

Moreover we cannot even specify/modify Accept-Encoding in a browser. It is always lists what browser accepts. To wit: https://developer.mozilla.org/en-US/docs/Glossary/Forbidden_header_name
The only time I needed to implement it myself was in the library, which makes server-to-server HTTP calls.

I suggest to implement two things:

  • The default for when Accept-Encoding is missing.
  • Set the default to identity and document that if somebody wants to do it "by the book" they should set it to *.

I think it breaks nothing and satisfies expectations of most users.

If this is acceptable to @jonathanong and other participants I can update the PR or create a new one with the fix.

@zacanger
Copy link
Contributor

zacanger commented Jul 1, 2020

Is there anything I can do to help this along? This change broke some APIs for us and we had to downgrade in a hurry (the use-case in that situation was not browser to Node, it was .NET Framework APIs calling a Node microservice).

@niftylettuce
Copy link
Contributor

Can you please update the PR per your notes @uhop ? I just reproduced same issue on curl.

@uhop uhop mentioned this pull request Jul 5, 2020
@niftylettuce niftylettuce merged commit 3d4c0ac into koajs:master Jul 6, 2020
niftylettuce added a commit that referenced this pull request Jul 6, 2020
@niftylettuce
Copy link
Contributor

v5.0.0 published to npm and GitHub

https://github.com/koajs/compress/releases/tag/v5.0.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants