Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(test): improve test recording docs #33744

Merged
merged 1 commit into from
Jan 31, 2023
Merged

docs(test): improve test recording docs #33744

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 14 additions & 12 deletions sdk/core/Azure.Core.TestFramework/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ public class ConfigurationLiveTests: ClientTestBase
private ConfigurationClient GetClient() =>
InstrumentClient(
new ConfigurationClient(
...,
...,
InstrumentClientOptions(
new ConfigurationClientClientOptions())));
}
Expand Down Expand Up @@ -137,7 +137,7 @@ public class AppConfigurationTestEnvironment : TestEnvironment
try
{
await service.GetConfigurationSettingAsync("Setting");
}
}
catch (RequestFailedException e) when (e.Status == 403)
{
return false;
Expand Down Expand Up @@ -191,7 +191,7 @@ public class ConfigurationLiveTests: RecordedTestBase<AppConfigurationTestEnviro
private ConfigurationClient GetClient() =>
InstrumentClient(
new ConfigurationClient(
...,
...,
InstrumentClientOptions(
new ConfigurationClientClientOptions())));
}
Expand Down Expand Up @@ -375,8 +375,8 @@ How it looks it the test explorer:

## Support for an additional test parameter

The `ClientTestFixture` attribute also supports specifying an additional array of parameter values to send to the test class.
Similar to the service versions, this results in the creation of a permutation of each test for each parameter value specified.
The `ClientTestFixture` attribute also supports specifying an additional array of parameter values to send to the test class.
Similar to the service versions, this results in the creation of a permutation of each test for each parameter value specified.
Example usage is shown below:

```c#
Expand All @@ -388,7 +388,7 @@ public class TableServiceLiveTestsBase : RecordedTestBase<TablesTestEnvironment>
{
protected readonly TableEndpointType _endpointType;

public TableServiceLiveTestsBase(bool isAsync, TableEndpointType endpointType, RecordedTestMode recordedTestMode)
public TableServiceLiveTestsBase(bool isAsync, TableEndpointType endpointType, RecordedTestMode recordedTestMode)
: base(isAsync, recordedTestMode)
{
_endpointType = endpointType;
Expand All @@ -405,15 +405,15 @@ public class TableServiceLiveTestsBase : RecordedTestBase<TablesTestEnvironment>
protected readonly TableEndpointType _endpointType;
TableClientOptions.ServiceVersion _serviceVersion

public TableServiceLiveTestsBase(bool isAsync, TableClientOptions.ServiceVersion serviceVersion, TableEndpointType endpointType, RecordedTestMode recordedTestMode)
public TableServiceLiveTestsBase(bool isAsync, TableClientOptions.ServiceVersion serviceVersion, TableEndpointType endpointType, RecordedTestMode recordedTestMode)
: base(isAsync, recordedTestMode)
{
_serviceVersion = serviceVersion;
_endpointType = endpointType;
}
```

**Note:** Additional parameter options work with test recordings and will create differentiated SessionRecords test class directory names for each additional parameter option.
**Note:** Additional parameter options work with test recordings and will create differentiated SessionRecords test class directory names for each additional parameter option.
For example:

`/SessionRecords/TableClientLiveTests(CosmosTable)/CreatedCustomEntitiesCanBeQueriedWithFiltersAsync.json`
Expand All @@ -423,7 +423,9 @@ For example:

Testing of management libraries uses the Test Framework and should generally be very similar to tests that you write for data plane libraries. There is an intermediate test class that you will likely want to derive from that lives within the management code base - [ManagementRecordedTestBase](https://github.com/Azure/azure-sdk-for-net/blob/babee31b3151e4512ac5a77a55c426c136335fbb/common/ManagementTestShared/ManagementRecordedTestBase.cs). To see examples of Track 2 Management tests using the Test Framework, take a look at the [Storage tests](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.ResourceManager.Storage/tests/Tests).

## Recording tests on CI
For details about testing management libraries, see [Test .NET management plane SDK](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/resourcemanager/Azure.ResourceManager/docs/TestGuide.md).

## Recording data plane tests on CI

Test framework provides an ability to re-record tests remotely using an Azure DevOps test pipeline. To re-record tests you need to have an open GitHub pull request.

Expand All @@ -445,7 +447,7 @@ To download and unpack all artifacts use the `Download-DevOpsRecordings.ps1` scr

The `Download-DevOpsRecordings.ps1` would wait for active runs to finish before retrieving artifacts unless `-NoWait` switch is used.

**NOTE:** these scripts require being signed in with Azure CLI (https://docs.microsoft.com/cli/azure/authenticate-azure-cli?view=azure-cli-latest) and access to the internal DevOps project (https://dev.azure.com/azure-sdk/internal/)
**NOTE:** these scripts require being signed in with Azure CLI (https://docs.microsoft.com/cli/azure/authenticate-azure-cli?view=azure-cli-latest) and access to the internal DevOps project (https://dev.azure.com/azure-sdk/internal/)

### Note on private/non-virtual fields in your clients (such as _clientDiagnostics) and InternalsVisibleTo

Expand Down Expand Up @@ -522,10 +524,10 @@ var isSet = AppContext.TryGetSwitch("Azure.Core.Pipeline.DisableHttpWebRequestTr
```

### AsyncAssert
This type contains static helper methods that cover some of the gaps in NUnit when it comes to async assertions. For instance, attempting to assert that a specific exception is thrown using Assert.That, Assert.Throws, or Assert.ThrowsAsync all result in sync over async code, which can lead to test flakiness.
This type contains static helper methods that cover some of the gaps in NUnit when it comes to async assertions. For instance, attempting to assert that a specific exception is thrown using Assert.That, Assert.Throws, or Assert.ThrowsAsync all result in sync over async code, which can lead to test flakiness.

#### Example usage
```c#
```c#
ServiceBusException exception = await AsyncAssert.ThrowsAsync<ServiceBusException>(
async () => await args.CompleteMessageAsync(message, args.CancellationToken));
Assert.AreEqual(ServiceBusFailureReason.MessageLockLost, exception.Reason);
Expand Down