Skip to content

Commit

Permalink
docs(test): improve test recording docs (Azure#33744)
Browse files Browse the repository at this point in the history
  • Loading branch information
archerzz authored Jan 31, 2023
1 parent d0d83a5 commit 9bf595d
Showing 1 changed file with 14 additions and 12 deletions.
26 changes: 14 additions & 12 deletions sdk/core/Azure.Core.TestFramework/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ public class ConfigurationLiveTests: ClientTestBase
private ConfigurationClient GetClient() =>
InstrumentClient(
new ConfigurationClient(
...,
...,
InstrumentClientOptions(
new ConfigurationClientClientOptions())));
}
Expand Down Expand Up @@ -137,7 +137,7 @@ public class AppConfigurationTestEnvironment : TestEnvironment
try
{
await service.GetConfigurationSettingAsync("Setting");
}
}
catch (RequestFailedException e) when (e.Status == 403)
{
return false;
Expand Down Expand Up @@ -191,7 +191,7 @@ public class ConfigurationLiveTests: RecordedTestBase<AppConfigurationTestEnviro
private ConfigurationClient GetClient() =>
InstrumentClient(
new ConfigurationClient(
...,
...,
InstrumentClientOptions(
new ConfigurationClientClientOptions())));
}
Expand Down Expand Up @@ -375,8 +375,8 @@ How it looks it the test explorer:

## Support for an additional test parameter

The `ClientTestFixture` attribute also supports specifying an additional array of parameter values to send to the test class.
Similar to the service versions, this results in the creation of a permutation of each test for each parameter value specified.
The `ClientTestFixture` attribute also supports specifying an additional array of parameter values to send to the test class.
Similar to the service versions, this results in the creation of a permutation of each test for each parameter value specified.
Example usage is shown below:

```c#
Expand All @@ -388,7 +388,7 @@ public class TableServiceLiveTestsBase : RecordedTestBase<TablesTestEnvironment>
{
protected readonly TableEndpointType _endpointType;

public TableServiceLiveTestsBase(bool isAsync, TableEndpointType endpointType, RecordedTestMode recordedTestMode)
public TableServiceLiveTestsBase(bool isAsync, TableEndpointType endpointType, RecordedTestMode recordedTestMode)
: base(isAsync, recordedTestMode)
{
_endpointType = endpointType;
Expand All @@ -405,15 +405,15 @@ public class TableServiceLiveTestsBase : RecordedTestBase<TablesTestEnvironment>
protected readonly TableEndpointType _endpointType;
TableClientOptions.ServiceVersion _serviceVersion

public TableServiceLiveTestsBase(bool isAsync, TableClientOptions.ServiceVersion serviceVersion, TableEndpointType endpointType, RecordedTestMode recordedTestMode)
public TableServiceLiveTestsBase(bool isAsync, TableClientOptions.ServiceVersion serviceVersion, TableEndpointType endpointType, RecordedTestMode recordedTestMode)
: base(isAsync, recordedTestMode)
{
_serviceVersion = serviceVersion;
_endpointType = endpointType;
}
```

**Note:** Additional parameter options work with test recordings and will create differentiated SessionRecords test class directory names for each additional parameter option.
**Note:** Additional parameter options work with test recordings and will create differentiated SessionRecords test class directory names for each additional parameter option.
For example:

`/SessionRecords/TableClientLiveTests(CosmosTable)/CreatedCustomEntitiesCanBeQueriedWithFiltersAsync.json`
Expand All @@ -423,7 +423,9 @@ For example:

Testing of management libraries uses the Test Framework and should generally be very similar to tests that you write for data plane libraries. There is an intermediate test class that you will likely want to derive from that lives within the management code base - [ManagementRecordedTestBase](https://github.com/Azure/azure-sdk-for-net/blob/babee31b3151e4512ac5a77a55c426c136335fbb/common/ManagementTestShared/ManagementRecordedTestBase.cs). To see examples of Track 2 Management tests using the Test Framework, take a look at the [Storage tests](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.ResourceManager.Storage/tests/Tests).

## Recording tests on CI
For details about testing management libraries, see [Test .NET management plane SDK](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/resourcemanager/Azure.ResourceManager/docs/TestGuide.md).

## Recording data plane tests on CI

Test framework provides an ability to re-record tests remotely using an Azure DevOps test pipeline. To re-record tests you need to have an open GitHub pull request.

Expand All @@ -445,7 +447,7 @@ To download and unpack all artifacts use the `Download-DevOpsRecordings.ps1` scr

The `Download-DevOpsRecordings.ps1` would wait for active runs to finish before retrieving artifacts unless `-NoWait` switch is used.

**NOTE:** these scripts require being signed in with Azure CLI (https://docs.microsoft.com/cli/azure/authenticate-azure-cli?view=azure-cli-latest) and access to the internal DevOps project (https://dev.azure.com/azure-sdk/internal/)
**NOTE:** these scripts require being signed in with Azure CLI (https://docs.microsoft.com/cli/azure/authenticate-azure-cli?view=azure-cli-latest) and access to the internal DevOps project (https://dev.azure.com/azure-sdk/internal/)

### Note on private/non-virtual fields in your clients (such as _clientDiagnostics) and InternalsVisibleTo

Expand Down Expand Up @@ -522,10 +524,10 @@ var isSet = AppContext.TryGetSwitch("Azure.Core.Pipeline.DisableHttpWebRequestTr
```

### AsyncAssert
This type contains static helper methods that cover some of the gaps in NUnit when it comes to async assertions. For instance, attempting to assert that a specific exception is thrown using Assert.That, Assert.Throws, or Assert.ThrowsAsync all result in sync over async code, which can lead to test flakiness.
This type contains static helper methods that cover some of the gaps in NUnit when it comes to async assertions. For instance, attempting to assert that a specific exception is thrown using Assert.That, Assert.Throws, or Assert.ThrowsAsync all result in sync over async code, which can lead to test flakiness.

#### Example usage
```c#
```c#
ServiceBusException exception = await AsyncAssert.ThrowsAsync<ServiceBusException>(
async () => await args.CompleteMessageAsync(message, args.CancellationToken));
Assert.AreEqual(ServiceBusFailureReason.MessageLockLost, exception.Reason);
Expand Down

0 comments on commit 9bf595d

Please sign in to comment.