[Test Proxy] Support matching GZip request content

[nisha-bhatia]

To be completed by 2nd Preview in mid September
Explore if .Net gzip is deterministic?
When running tests in Record vs. Playback it seems like the length of the gzipped requests vary slightly. The test framework needs the a test's RequestBody's to match when running in Playback.
Possible solution: add a boolean property in TextProxy that for gzipped libraries that when true does not compare the Request Body in a Recording

[azure-sdk]

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Storage:0.20091417,Event Hubs:0.07821843,Search:0.068059646'

[JoshLove-msft]

Test Proxy might be able to detect if request is Gzip based on content-encoding, and use that as a signal to decompress the bodies before comparing.

[pallavit]

This is an ask on the test framework. Gzip by nature is non-deterministic so we need a way to check if the content encoding is Gzip and compare the uncompressed request body instead in the proxy.
/cc:@nisha-bhatia Is this a blocker? Can we make the Gzip comparison tests live-only for preview and make this a GA criteria?

[JoshLove-msft]

This should be handled in the Test Proxy itself, as it will be an issue for all languages.

[nisha-bhatia]

This is currently not a blocker because we have told the tests to not compare the text bodies of requests. It is something we should address soon though

[JoshLove-msft]

/cc @scbedd

[jsquire]

@JoshLove-msft: If this was handled in the test proxy, would that let Monitor just re-record the tests and things work - or would we need additional adjustments in the test framework to accommodate playback mode?

[JoshLove-msft]

@JoshLove-msft: If this was handled in the test proxy, would that let Monitor just re-record the tests and things work - or would we need additional adjustments in the test framework to accommodate playback mode?

Ideally, the monitor tests would not need to be re-recorded. The Test Proxy should check the content-encoding of a request, and if it is Gzip, then it needs to decompress the request body and the recorded request body before comparing them.

[jsquire]

Ah, gotcha. So the proxy does the comparison, not the framework. That's where I wasn't aware.

[JoshLove-msft]

Ah, gotcha. So the proxy does the comparison, not the framework. That's where I wasn't aware.

Yep, the playback matching happens within the test proxy.

[scbedd]

Can I please see what the requests would look like in json form?

We absolutely already automatically un-gzip request bodies when it is indicated by the header values. I suspect that the body differences are actually real.

[JoshLove-msft]

Can I please see what the requests would look like in json form?

We absolutely already automatically un-gzip request bodies when it is indicated by the header values. I suspect that the body differences are actually real.

I think this is one example - https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/monitor/Azure.Monitor.Ingestion/tests/SessionRecords/MonitorIngestionLiveTest/ValidInputFromArrayAsJsonWithSingleBatchWithGzip.json#L17-L22

So the body is decompressed from the request when recording, but what about during playback? I don't see where the incoming request body is decompressed there. Without doing this, we end up comparing the compressed body with the decompressed one, so even without the issue of determinism (which may turn out to have not really been an issue here), the bodies won't match if the request is zipped.

[scbedd]

which may turn out to have not really been an issue here

I'm really thinking this is the case. We absolutely re-compress as we should, but it looks like there is a bug with decompressing the incoming. I'll add this to the list of "get these done ASAP" bugs for test-proxy.

The only other item on that list is for the further multipart/mixed fixes. I'm grinding super hard trying to get asset-sync stuff ready to go before MQ, but I'll definitely dedicate some time to this bug in the very near future.

[JoshLove-msft]

I looked into this more and the proxy is storing decompressed Gzip for response bodies. This issue is specifically about the need to decompress request bodies when matching, and potentially when recording. If you look at the linked recording files, you can see that the data that was recorded contains a bunch of \uFFFD which is the unicode replacement character. This is because the binary Gzipped data is being translated to UTF-8 and stored as a string. So we will need to re-record these as the data is corrupted after the proxy is updated to handle storing Gzipped content (right now it assumes it is UTF-8 because the content type is JSON). Since we seem to be decompressing the response body when writing to the session files, it makes sense to do the same for request bodies.