Add Live TestMode and Skip Recording Concepts #6134
Conversation
… teardown if test was skipped
|
/azp run java - storage - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run java - storage - tests |
|
Azure Pipelines failed to run 1 pipeline(s). |
|
/azp run java - storage - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
alzimmermsft
requested review from
jianghaolu,
mssfang,
rickle-msft,
samvaity and
sima-zhu
as code owners
|
/azp run java - appconfiguration - tests |
|
/azp run java - storage - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
1 similar comment
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run java - storage - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
| return Objects.requireNonNull(client); | ||
| if (testMode == TestMode.PLAYBACK && !testRunVerifier.doNotRecordTest()) { | ||
| clientBuilder.httpClient(interceptorManager.getPlaybackClient()); | ||
| } else if (testMode == TestMode.RECORD && !testRunVerifier.doNotRecordTest()) { |
There was a problem hiding this comment.
What's the reason to check test testRunVerifier.doNotRecordTest()?
If we don't want record, can we simply setup env to LIVE
There was a problem hiding this comment.
LIVE and DoNotRecord serve slightly different but related purposes. LIVE won't record anything but the DoNotRecord annotation is meant for tests that shouldn't have any data recorded.
| if (testMode == TestMode.PLAYBACK && !testRunVerifier.doNotRecordTest()) { | ||
| clientBuilder.httpClient(interceptorManager.getPlaybackClient()); | ||
| } else if (testMode == TestMode.RECORD && !testRunVerifier.doNotRecordTest()) { | ||
| clientBuilder.httpClient(new NettyAsyncHttpClientBuilder().wiretap(true).build()) |
There was a problem hiding this comment.
This one applies to LIVE mode. Better to put in both RECORD and LIVE
There was a problem hiding this comment.
LIVE won't need Netty wiretap set to true as it doesn't need to inspect the request or response for recording purposes.
|
/azp run java - storage - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
| */ | ||
| @Retention(RUNTIME) | ||
| @Target({METHOD}) | ||
| public @interface DoNotRecord { |
There was a problem hiding this comment.
Just for my understanding, why would any test run only in live mode but shouldn't be recorded?
There was a problem hiding this comment.
Tests that contain information that cannot be recorded into a playback JSON such as random UUIDs generated by methods called.
|
Closing in favor of #6671 |
Implements #5174
This PR introduces two new concepts to the Azure Core testing framework.
A new value
LIVEwas added to theTestModeenum. This state is used to indicate that tests should be ran against a live service without recording the network calls, this is meant to help reduce the overhead of testing by removing the recording process. The use case for this change is to remove recording tests during the nightly validation and in cases where sensitive information that hasn't been masked is being sent during testing.A new test annotation,
DoNotRecord, was added. This annotation is meant to be used with tests that don't want their network calls record or they make no network calls but surrounding testing infrastructure does. This annotation has an optional parameter ofskipInPlaybackwhich is meant to be used with tests that make network calls but won't be recorded due to the inability to redact sensitive data or they call into code paths that cannot be mocked.