Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add recorded tests for Storage #6609

Merged
merged 7 commits into from
Jun 19, 2019
Merged

Add recorded tests for Storage #6609

merged 7 commits into from
Jun 19, 2019

Conversation

tg-msft
Copy link
Member

@tg-msft tg-msft commented Jun 17, 2019

  • Switch from MSTest to NUnit 3
  • Azure.Core testing framework tweaks (recording related)
  • Add Storage recorded tests
  • Replay test execution is ~20s across all Blobs, Files, and Queues tests

Most of the changes are tweaking the Storage tests to be less stateful. They used a lot of random values and DateTimeOffset.Now which needed to be isolated to individual test methods so replaying works correctly. The static TestHelper class, in particular, had to be turned into base classes that can access the current test Recording member variable.

You can set the AZURE_TEST_MODE environment variable to Live for running tests against the live service, Record for updating recordings, Playback to run the recordings, or leave it empty to default to Playback. Additionally, all of the Storage test classes allow you to pass the RecordedTestMode enum into their ctor if you want to override the environment variable (only intended for use while developing).

/cc @kfarmer-msft and @seanmcc-msft

@tg-msft
Copy link
Member Author

tg-msft commented Jun 18, 2019

/azp run net - storage - tests

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@tg-msft
Copy link
Member Author

tg-msft commented Jun 18, 2019

/azp run net - storage - tests

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@tg-msft
Copy link
Member Author

tg-msft commented Jun 18, 2019

/azp run net - storage - tests

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@maririos
Copy link
Member

Changes to Azure.Core testing framework look good!

@tg-msft
Copy link
Member Author

tg-msft commented Jun 18, 2019

/azp run net - storage - tests

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@tg-msft
Copy link
Member Author

tg-msft commented Jun 18, 2019

/azp run net - storage - tests

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@@ -118,17 +118,17 @@ public async Task CreateAsync_Metadata()
}

//TODO
[Ignore]
[TestMethod]
[Ignore("Not yet implemented")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have an issue for this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't been filing issues for the Storage Team's TODOs that were already in the code. We should chat tomorrow about whether we want to do this and possibly add a work item in the next milestone for me to add issues for all TODOs.

Copy link
Member

@jsquire jsquire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, though I lack a thorough understanding of the contexts and our recorded test infrastructure. You'll probably want at least one other approval rather than relying on just me.

tg-msft added 7 commits June 19, 2019 14:33
- Improve the StorageRequestFailedException error message
- Have Response.Headers.Date return x-ms-date if Date isn't present
- Make tests more reliable with MemoryStream.ToArray() instead .GetBuffer()

Here's an example of an error message with additional info specified:

Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:6b6d5a57-201e-007b-2291-250f92000000
Time:2019-06-18T04:53:02.9793056Z
Status: 403 (Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.)

ErrorCode: AuthenticationFailed

Additional Information:
AuthenticationErrorDetail: The MAC signature found in the HTTP request 'T8bvpwE2Fp9cKr9yAfo1kblARYwUoRLovmH1nIlPrpQ=' is not the same as any computed signature. Server used following string to sign: 'GET

x-ms-client-request-id:097eb32f-12eb-476b-8fe2-e912b96e6974
x-ms-date:Tue, 18 Jun 2019 04:53:01 GMT
x-ms-return-client-request-id:true
x-ms-version:2018-11-09
/storageteglazatesting/
comp:properties
restype:account'.

Headers:
Server: Microsoft-HTTPAPI/2.0
x-ms-request-id: 6b6d5a57-201e-007b-2291-250f92000000
x-ms-error-code: AuthenticationFailed
Date: Tue, 18 Jun 2019 04:53:02 GMT
Content-Length: 779
Content-Type: application/xml
@tg-msft tg-msft merged commit 2fd9944 into Azure:master Jun 19, 2019
@tg-msft tg-msft deleted the testing branch June 19, 2019 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants