server: compress large response payloads if client supports it

[anderseknert]

For certain types of payloads, or just exploratory browsing of /data, the main time spent for these queries is not inside of OPA, but waiting for data in transit. If the client supports content encoding like gzip, compressing the data may provide quite a dramatic reduction of payload size and hence total response time.

1. If an integrating HTTP client sends an Accept-Encoding header including gzip...
2. ...and the payload is larger than some arbitrary number, like 1 kB...
3. Respond with a Content-Encoding: gzip header and a gzip compressed payload.

[srenatus]

https://pkg.go.dev/github.com/gorilla/handlers#CompressHandler we should perhaps just use something like that. 💭

[sanjeev98kumar]

Could you please assign this issue to me? I am looking for my first PR.
Also wanted to know how much time will it require for me to set up opa on my local?

[anderseknert]

Of course, @sanjeev98kumar 👍

I would not expect much time to be spent setting anything up — just clone the repo and get hacking :)
But do see https://www.openpolicyagent.org/docs/latest/contrib-code/ and https://www.openpolicyagent.org/docs/latest/contrib-development/ for some useful information on contributing. And feel free to ask any questions whenever you want.

[margelatu]

+1 for supporting compression, we're also in need of this feature.

[anderseknert]

@sanjeev98kumar any progress to report on this issue? 🙂

[burnerlee]

@anderseknert @srenatus can I pick this issue up if it's okay?

[srenatus]

@burnerlee since there's no response from @sanjeev98kumar for a while, I guess that's OK 👍

[burnerlee]

cool, I'll start working on the issue

[burnerlee]

@srenatus can you help by giving me some insight into the issue? Like in which section of the project we're making this change and what do we exactly want to do?

[burnerlee]

I've opened a draft PR from what I could understand from the issue description. Do we need to add this handler somewhere else too?

[srenatus]

Ideally, tests would tell us, I think.

[irwanshofwan]

can I pick this issue or pair for this ? @anderseknert @srenatus

[anderseknert]

Hey @irwanshofwan! Perhaps we could get an update from @burnerlee first? He has an open PR for this here, but as far as I can see, there's a few unaddressed comments there. An unforseen event here is how the gorilla library was archived just recently, so whoever works on this issue will likely want to await this PR to be merged.

[anderseknert]

But either way, thanks for stepping up! 😃

[burnerlee]

hey @irwanshofwan please go ahead. I'm stuck in some other things rn and not able to contribute. Sorry for holding you guys back

[burnerlee]

i would suggest continuing the same PR. we just need to handle the tests right now

[irwanshofwan]

sure, will try continue from existing PR

[anderseknert]

Whatever works best for you, but do see my note about gorilla above.

[irwanshofwan]

sure, I believe we can do it parallel for this issue and rebase after the gorilla mux solved.

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

[margelatu]

@irwanshofwan any news on this ?

Context: we use OPA heavily at Adobe and in some scenarios need gzip support on both requests and responses as they get rather large.

[anderseknert]

@margelatu I believe this was put on hold while waiting for the router library situation to be resolved. But this is probably simple enough to pull off without having to use a helper from there, should anyone want to step up and do it :)