Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Storage][DataMovement] Updated perf test readme, remove perf from base sln #47129

Merged
merged 2 commits into from
Nov 12, 2024
Merged

[Storage][DataMovement] Updated perf test readme, remove perf from base sln #47129

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c65b5a8
Updated perf test readme, remove from base sln
jalauzon-msft Nov 12, 2024
1d683c2
Fix size
jalauzon-msft Nov 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions sdk/storage/Azure.Storage.DataMovement.Blobs/perf-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
Service: storage-datamovement-blob

Project: sdk/storage/Azure.Storage.DataMovement.Blobs/perf/Azure.Storage.DataMovement.Blobs.Perf/Azure.Storage.DataMovement.Blobs.Perf.csproj
# For Track1, comment above line, uncomment next line
# Project: sdk/storage/Azure.Storage.DataMovement.Blobs/perf/Microsoft.Azure.Storage.DataMovement.Perf/Microsoft.Azure.Storage.DataMovement.Perf.csproj

PackageVersions:
- Azure.Storage.DataMovement.Blobs: source
# For Track1, comment above line, uncomment next line
# - Microsoft.Azure.Storage.DataMovement: 2.0.5

Tests:
- Test: download
Expand Down
28 changes: 23 additions & 5 deletions sdk/storage/Azure.Storage.DataMovement.Blobs/perf/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,39 @@
The assets in this area comprise a set of performance tests for the [Azure Storage Blobs Data Movement library for .NET](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.Storage.DataMovement.Blobs) and its associated ecosystem. The artifacts in this library are intended to be used primarily with the Azure SDK engineering system's testing infrastructure, but may also be run as stand-alone applications from the command line.

## Running tests locally
First, build the project either via Visual Studio or the command line. Be sure to build the `Release` build.
There are two perf test projects:
- Track2 (default): sdk\storage\Azure.Storage.DataMovement.Blobs\perf\Azure.Storage.DataMovement.Blobs.Perf\Azure.Storage.DataMovement.Blobs.Perf.csproj
- Track1: sdk\storage\Azure.Storage.DataMovement.Blobs\perf\Microsoft.Azure.Storage.DataMovement.Perf\Microsoft.Azure.Storage.DataMovement.Perf.csproj

First, build the project either via Visual Studio or the command line. For best results, build the `Release` build but the `Debug` will also work and allow for debugging.
```
dotnet build -c Release -f <supported-framework> sdk\storage\Azure.Storage.DataMovement.Blobs\perf\Azure.Storage.DataMovement.Blobs.Perf\Azure.Storage.DataMovement.Blobs.Perf.csproj
dotnet build -c Release -f <supported-framework> <project>
```

Setup the account you want to use for testing. Note these tests use OAuth using a DefaultAzureCredential, meaning you will need
tot he Storage account as well as a way to authenticate (Visual Studio, Azure CLI, etc.)
Setup the account you want to use for testing. Note Track2 tests use OAuth using a DefaultAzureCredential, meaning you will need permission to the Storage account as well as a way to authenticate (Visual Studio, Azure CLI, etc.)
```
set AZURE_STORAGE_ACCOUNT_NAME=<account-name>
# Only needed for Track1 tests
set AZURE_STORAGE_ACCOUNT_KEY=<account-key>
```

Then run the desired test via the command line.
```
dotnet run -c Release -f <supported-framework> --no-build --project sdk\storage\Azure.Storage.DataMovement.Blobs\perf\Azure.Storage.DataMovement.Blobs.Perf\Azure.Storage.DataMovement.Blobs.Perf.csproj -- <test-name> <test-options>
dotnet run -c Release -f <supported-framework> --no-build --project sdk\storage\Azure.Storage.DataMovement.Blobs\perf\Azure.Storage.DataMovement.Blobs.Perf\Azure.Storage.DataMovement.Blobs.Perf.csproj -- (UploadDirectory|DownloadDirectory|CopyDirectory) <test-options>
```

Current test options (non-exahustive list):
- `-c`, `--count`: The number of files in each transfer (default: 10)
- `-s`, `--size`: The size of each file (default: 1024)
- `-d`, `--duration`: The duration of the test (default: 10)
- `--councurrency`: The concurrency limit for Data Movement
- `--initial-transfer-size`: The initial transfer size of the transfer (Track2 only)
- `--chunk-size`: The chunk size to use during the transfer
- `--disable-checkpointer`: Disable the checkpointer (Track2 only)
- `--help`: See full list of options for that particular test

### Running in Visual Studio
You can also run these tests directly through Visual Studio which can be helpful for debugging or profiling. You just need to setup the run profile for either project to include the command line arguments `(UploadDirectory|DownloadDirectory|CopyDirectory) <test-options>` and the enviorment variables.

## Contributing

Expand Down
7 changes: 0 additions & 7 deletions sdk/storage/Azure.Storage.sln
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,8 +159,6 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Storage.DataMovement.
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.ResourceManager.Storage", "Azure.ResourceManager.Storage\src\Azure.ResourceManager.Storage.csproj", "{6EA8939A-537F-475E-B16C-F9E80825C6AD}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Storage.DataMovement.Blobs.Perf", "Azure.Storage.DataMovement.Blobs\perf\Azure.Storage.DataMovement.Blobs.Perf\Azure.Storage.DataMovement.Blobs.Perf.csproj", "{45EF1203-EAFF-4BB7-97F1-58DA0C310D97}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand Down Expand Up @@ -347,10 +345,6 @@ Global
{6EA8939A-537F-475E-B16C-F9E80825C6AD}.Debug|Any CPU.Build.0 = Debug|Any CPU
{6EA8939A-537F-475E-B16C-F9E80825C6AD}.Release|Any CPU.ActiveCfg = Release|Any CPU
{6EA8939A-537F-475E-B16C-F9E80825C6AD}.Release|Any CPU.Build.0 = Release|Any CPU
{45EF1203-EAFF-4BB7-97F1-58DA0C310D97}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{45EF1203-EAFF-4BB7-97F1-58DA0C310D97}.Debug|Any CPU.Build.0 = Debug|Any CPU
{45EF1203-EAFF-4BB7-97F1-58DA0C310D97}.Release|Any CPU.ActiveCfg = Release|Any CPU
{45EF1203-EAFF-4BB7-97F1-58DA0C310D97}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand All @@ -367,7 +361,6 @@ Global
{CC33E052-9737-42A3-BEB5-060CA659F408} = {FBC7A205-B57A-44E7-A7CA-2E82F79A4D9C}
{2FB0A438-7DCB-4748-8E95-BA495865AD1D} = {FBC7A205-B57A-44E7-A7CA-2E82F79A4D9C}
{0323A01E-E360-4D22-B1F4-15FD2DB39437} = {FBC7A205-B57A-44E7-A7CA-2E82F79A4D9C}
{45EF1203-EAFF-4BB7-97F1-58DA0C310D97} = {FBC7A205-B57A-44E7-A7CA-2E82F79A4D9C}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {9F236F49-3FED-4E35-AF93-48BA7EB0D42A}
Expand Down