Merge main into feature/amqp. (#4491)

* Add GetCredentialName() (#4428) * Add GetCredentialName() * Update * Undo accidental change * Clang-format * Call GetCredentialName() instead of using constant; Return in-place constructed name; Explicit tests for GetCredentialName() * PR feedback * constructor parameter + non-virtual GetCredentialName() * Update sdk/core/azure-core/CMakeLists.txt * Update sdk/identity/azure-identity/test/ut/client_secret_credential_test.cpp * Update sdk/identity/azure-identity/test/ut/client_certificate_credential_test.cpp * GCC and Clang warnings * Promote ThrowIfNotSafeCmdLineInput() to private member; avoid copies when calling GetCredentialName() * Spelling * Fix deprecated usage * Fix iteration * Clang-format --------- Co-authored-by: Anton Kolesnyk <[email protected]> * Sync eng/common directory with azure-sdk-tools for PR 5702 (#4453) * add healthinsights to Test-SampleMetadata script * spacing * update productSlug to azure-health-insights --------- Co-authored-by: Asaf Levi <[email protected]> * Use aka.ms link to Identity troubleshooting (#4449) * Use aka.ms link to Identity troubleshooting * Update default_azure_credential.cpp * Update default_azure_credential.cpp --------- Co-authored-by: Anton Kolesnyk <[email protected]> * Undocument ChainedCred usage by DefaultAzCred & remove friend and private ctor (#4447) * Undocument ChainedCred usage by DefaultAzCred & remove friend and private ctor * Clang warning fix --------- Co-authored-by: Anton Kolesnyk <[email protected]> * Sync eng/common directory with azure-sdk-tools for PR 5691 (#4450) * typespec renaming * add back scripts for cadl * support .cadl and .tsp * rename * add newline at the end of file --------- Co-authored-by: FAREAST\chunyu <[email protected]> * fix cspell for readme.ms in libcurl sterss test (#4441) * fix cspell * capitalize Valgrind and ubuntu * sdada * fix2 * Simpler identity logging (#4455) * Simpler identity logging * Even simpler * Remove refactoring artifact * Cosmetic change * foreach --------- Co-authored-by: Anton Kolesnyk <[email protected]> * 3rd Party Libs in Samples (#4408) Adding guidance on the proper usage of 3rd Party Libraries in our samples. * Update changelog with issue link (#4458) Co-authored-by: Anton Kolesnyk <[email protected]> * Tests: replace most `EXPECT_TRUE(a OP b)` with `EXPECT_OP(a, b)` (#4457) * Tests: replace most `EXPECT_TRUE(a OP b)` with `EXPECT_OP(a, b)` * Undo unnecessary change --------- Co-authored-by: Anton Kolesnyk <[email protected]> * Revert "[check-spelling] Temporarily pin Node 18 to 18.13.0 (#5537)" (#4456) This reverts commit 8a02e02adfc0d213509fce2764132afa74bd4ba4. Co-authored-by: Mike Harder <[email protected]> * Fix potentially high CPU usage on Windows (#4448) * Fix potentially high CPU usage on Windows * Undo unnecessary formatting * Undo unnecessary changelog * Undo unnecessary formatting * Undo unnecessary formatting * Uninclude locale * Add issue link to changelog * EXPECT_TRUE(a == b) => EXPECT_EQ(a, b) * Update second changelog with link as well --------- Co-authored-by: Anton Kolesnyk <[email protected]> * Ensure the comparison is unsigned to unsigned (#4464) * Ensure the comparison is unsigned to unsigned * Remove cast * Sync eng/common directory with azure-sdk-tools for PR 5742 (#4465) * add some default output to see about minimizing any occurrence of the task failing for no reason. perhaps having some output will allow devops to have an easier time with the invocation * update message * Update eng/common/scripts/trust-proxy-certificate.ps1 Co-authored-by: Wes Haggard <[email protected]> * Update CODEOWNERS (#4461) Fixup an invalid user * Organize applying Identity log prefix (#4459) * Organize applying Identity log prefix * logLevel * Cosmetic changes --------- Co-authored-by: Anton Kolesnyk <[email protected]> * Explicitly set PSNativeCommandArgumentPassing to Legacy for git push script (#4481) https://learn.microsoft.com/en-us/powershell/scripting/learn/experimental-features?view=powershell-7.3#psnativecommandargumentpassing Do to that breaking change in PS 7.3 we need to opt into the legacy arg parsing. Co-authored-by: Wes Haggard <[email protected]> * Fix unmatched parenthesis in doc (#4482) * fix concurrent upload failures (#4484) * Sync eng/common directory with azure-sdk-tools for PR 5726 (#4488) * rerun flag * rerun failed stress test * naming & commenting * update * function and var renaming for better readability * readability & exit on error --------- Co-authored-by: Albert Cheng <[email protected]> --------- Co-authored-by: Anton Kolesnyk <[email protected]> Co-authored-by: Anton Kolesnyk <[email protected]> Co-authored-by: Azure SDK Bot <[email protected]> Co-authored-by: Asaf Levi <[email protected]> Co-authored-by: FAREAST\chunyu <[email protected]> Co-authored-by: George Arama <[email protected]> Co-authored-by: Ronnie Geraghty <[email protected]> Co-authored-by: Mike Harder <[email protected]> Co-authored-by: Rick Winter <[email protected]> Co-authored-by: Wes Haggard <[email protected]> Co-authored-by: Wes Haggard <[email protected]> Co-authored-by: JinmingHu <[email protected]> Co-authored-by: Albert Cheng <[email protected]>
Azure · Mar 30, 2023 · e23be62 · e23be62
1 parent f0a26fd
commit e23be62
Show file tree

Hide file tree

Showing 63 changed files with 1,100 additions and 549 deletions.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -23,10 +23,10 @@
 ###########
 
 # PRLabel: %Attestation
-/sdk/attestation/        @LarryOsterman @gkostal @anilba06 @kroshkina-ms @ahmadmsft @rickwinter @ahsonkhan @antkmsft @vhvb1989 @gearama
+/sdk/attestation/        @LarryOsterman @gkostal @anilba06 @ahmadmsft @rickwinter @ahsonkhan @antkmsft @vhvb1989 @gearama
 
 # PRLabel: %KeyVault
-/sdk/keyvault/           @vhvb1989 @gearama @antkmsft @rickwinter
+/sdk/keyvault/           @gearama @antkmsft @rickwinter @LarryOsterman
 
 # PRLabel: %Storage
 /sdk/storage/            @vinjiang @Jinming-Hu @EmmaZhu @antkmsft @vhvb1989 @gearama @LarryOsterman @microzchang

diff --git a/.vscode/cspell.json b/.vscode/cspell.json
@@ -12,7 +12,6 @@
     "*.a",
     "*.lib",
     "*.yaml",
-    "**/libcurl-stress-test/README.md",
     ".github/CODEOWNERS",
     ".gitignore",
     ".vscode/cspell.json",

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -257,3 +257,23 @@ If the coverage data has been previously generated (for example, if you manually
 ### Visual Studio 2019
 
 You can also build the project by simply opening the repo directory in Visual Studio. Visual Studio will detect the `CMake` file and will configure itself to generate, build and run tests.
+
+## Samples
+
+### Third-party dependencies
+
+Third party libraries should only be included in samples when necessary to demonstrate usage of an Azure SDK package; they should not be suggested or endorsed as alternatives to the Azure SDK.
+
+When code samples take dependencies, readers should be able to use the material without significant license burden or research on terms. This goal requires restricting dependencies to certain types of open source or commercial licenses.
+
+Samples may take the following categories of dependencies:
+
+- **Open-source** : Open source offerings that use an [Open Source Initiative (OSI) approved license](https://opensource.org/licenses). Any component whose license isn't OSI-approved is considered a commercial offering. Prefer OSS projects that are members of any of the [OSS foundations that Microsoft is part of](https://opensource.microsoft.com/ecosystem/). Prefer permissive licenses for libraries, like [MIT](https://opensource.org/licenses/MIT) and [Apache 2](https://opensource.org/licenses/Apache-2.0). Copy-left licenses like [GPL](https://opensource.org/licenses/gpl-license) are acceptable for tools, and OSs. [Kubernetes](https://github.com/kubernetes/kubernetes), [Linux](https://github.com/torvalds/linux), and [Newtonsoft.Json](https://github.com/JamesNK/Newtonsoft.Json) are examples of this license type. Links to open source components should be to where the source is hosted, including any applicable license, such as a GitHub repository (or similar).
+
+- **Commercial**: Commercial offerings that enable readers to learn from our content without unnecessary extra costs. Typically, the offering has some form of a community edition, or a free trial sufficient for its use in content. A commercial license may be a form of dual-license, or tiered license. Links to commercial components should be to the commercial site for the software, even if the source software is hosted publicly on GitHub (or similar).
+
+- **Dual licensed**: Commercial offerings that enable readers to choose either license based on their needs. For example, if the offering has an OSS and commercial license, readers can  choose between them. [MySql](https://github.com/mysql/mysql-server) is an example of this license type.
+
+- **Tiered licensed**: Offerings that enable readers to use the license tier that corresponds to their characteristics. For example, tiers may be available for students, hobbyists, or companies with defined revenue  thresholds. For offerings with tiered licenses, strive to limit our use in tutorials to the features available in the lowest tier. This policy enables the widest audience for the article. [Docker](https://www.docker.com/), [IdentityServer](https://duendesoftware.com/products/identityserver), [ImageSharp](https://sixlabors.com/products/imagesharp/), and [Visual Studio](https://visualstudio.com) are examples of this license type.
+
+In general, we prefer taking dependencies on licensed components in the order of the listed categories. In cases where the category may not be well known, we'll document the category so that readers understand the choice that they're making by using that dependency.
diff --git a/eng/common/pipelines/templates/steps/check-spelling.yml b/eng/common/pipelines/templates/steps/check-spelling.yml
@@ -18,8 +18,8 @@ steps:
     - task: NodeTool@0
       condition: and(succeededOrFailed(), ne(variables['Skip.SpellCheck'],'true'))
       inputs:
-        versionSpec: 18.13.0
-      displayName: Use Node.js 18.13.0
+        versionSpec: 18.x
+      displayName: Use Node.js 18.x
 
     - task: PowerShell@2
       displayName: Check spelling (cspell)

diff --git a/eng/common/scripts/Test-SampleMetadata.ps1 b/eng/common/scripts/Test-SampleMetadata.ps1
@@ -202,6 +202,7 @@ begin {
         "azure-genomics",
         "azure-hdinsight",
         "azure-hdinsight-rserver",
+        "azure-health-insights",
         "azure-hpc-cache",
         "azure-immersive-reader",
         "azure-information-protection",

diff --git a/eng/common/scripts/TypeSpec-Project-Generate.ps1 b/eng/common/scripts/TypeSpec-Project-Generate.ps1
@@ -0,0 +1,101 @@
+# For details see https://github.com/Azure/azure-sdk-tools/blob/main/doc/common/TypeSpec-Project-Scripts.md
+
+[CmdletBinding()]
+param (
+    [Parameter(Position=0)]
+    [ValidateNotNullOrEmpty()]
+    [string] $ProjectDirectory,
+    [Parameter(Position=1)]
+    [string] $typespecAdditionalOptions ## addtional typespec emitter options, separated by semicolon if more than one, e.g. option1=value1;option2=value2
+)
+
+$ErrorActionPreference = "Stop"
+. $PSScriptRoot/Helpers/PSModule-Helpers.ps1
+. $PSScriptRoot/common.ps1
+Install-ModuleIfNotInstalled "powershell-yaml" "0.4.1" | Import-Module
+
+function NpmInstallForProject([string]$workingDirectory) {
+    Push-Location $workingDirectory
+    try {
+        $currentDur = Resolve-Path "."
+        Write-Host "Generating from $currentDur"
+
+        if (Test-Path "package.json") {
+            Remove-Item -Path "package.json" -Force
+        }
+
+        if (Test-Path ".npmrc") {
+            Remove-Item -Path ".npmrc" -Force
+        }
+
+        if (Test-Path "node_modules") {
+            Remove-Item -Path "node_modules" -Force -Recurse
+        }
+
+        if (Test-Path "package-lock.json") {
+            Remove-Item -Path "package-lock.json" -Force
+        }
+
+        #default to root/eng/emitter-package.json but you can override by writing
+        #Get-${Language}-EmitterPackageJsonPath in your Language-Settings.ps1
+        $replacementPackageJson = "$PSScriptRoot/../../emitter-package.json"
+        if (Test-Path "Function:$GetEmitterPackageJsonPathFn") {
+            $replacementPackageJson = &$GetEmitterPackageJsonPathFn
+        }
+
+        Write-Host("Copying package.json from $replacementPackageJson")
+        Copy-Item -Path $replacementPackageJson -Destination "package.json" -Force
+        npm install --no-lock-file
+        if ($LASTEXITCODE) { exit $LASTEXITCODE }
+    }
+    finally {
+        Pop-Location
+    }
+}
+
+$resolvedProjectDirectory = Resolve-Path $ProjectDirectory
+$emitterName = &$GetEmitterNameFn
+$typespecConfigurationFile = Resolve-Path "$ProjectDirectory/tsp-location.yaml"
+
+Write-Host "Reading configuration from $typespecConfigurationFile"
+$configuration = Get-Content -Path $typespecConfigurationFile -Raw | ConvertFrom-Yaml
+
+$specSubDirectory = $configuration["directory"]
+$innerFolder = Split-Path $specSubDirectory -Leaf
+
+$tempFolder = "$ProjectDirectory/TempTypeSpecFiles"
+$npmWorkingDir = Resolve-Path $tempFolder/$innerFolder
+$mainTypeSpecFile = If (Test-Path "$npmWorkingDir/client.*") { Resolve-Path "$npmWorkingDir/client.*" } Else { Resolve-Path "$npmWorkingDir/main.*"}
+
+try {
+    Push-Location $npmWorkingDir
+    NpmInstallForProject $npmWorkingDir
+
+    if ($LASTEXITCODE) { exit $LASTEXITCODE }
+
+    if (Test-Path "Function:$GetEmitterAdditionalOptionsFn") {
+        $emitterAdditionalOptions = &$GetEmitterAdditionalOptionsFn $resolvedProjectDirectory
+        if ($emitterAdditionalOptions.Length -gt 0) {
+            $emitterAdditionalOptions = " $emitterAdditionalOptions"
+        }
+    }
+    $typespecCompileCommand = "npx tsp compile $mainTypeSpecFile --emit $emitterName$emitterAdditionalOptions"
+    if ($typespecAdditionalOptions) {
+        $options = $typespecAdditionalOptions.Split(";");
+        foreach ($option in $options) {
+            $typespecCompileCommand += " --option $emitterName.$option"
+        }
+    }
+    Write-Host($typespecCompileCommand)
+    Invoke-Expression $typespecCompileCommand
+
+    if ($LASTEXITCODE) { exit $LASTEXITCODE }
+}
+finally {
+    Pop-Location
+}
+
+$shouldCleanUp = $configuration["cleanup"] ?? $true
+if ($shouldCleanUp) {
+    Remove-Item $tempFolder -Recurse -Force
+}
diff --git a/eng/common/scripts/TypeSpec-Project-Sync.ps1 b/eng/common/scripts/TypeSpec-Project-Sync.ps1
@@ -0,0 +1,127 @@
+# For details see https://github.com/Azure/azure-sdk-tools/blob/main/doc/common/TypeSpec-Project-Scripts.md
+
+[CmdletBinding()]
+param (
+    [Parameter(Position=0)]
+    [ValidateNotNullOrEmpty()]
+    [string] $ProjectDirectory
+)
+
+$ErrorActionPreference = "Stop"
+. $PSScriptRoot/Helpers/PSModule-Helpers.ps1
+Install-ModuleIfNotInstalled "powershell-yaml" "0.4.1" | Import-Module
+$sparseCheckoutFile = ".git/info/sparse-checkout"
+
+function AddSparseCheckoutPath([string]$subDirectory) {
+    if (!(Test-Path $sparseCheckoutFile) -or !((Get-Content $sparseCheckoutFile).Contains($subDirectory))) {
+        Write-Output $subDirectory >> .git/info/sparse-checkout
+    }
+}
+
+function CopySpecToProjectIfNeeded([string]$specCloneRoot, [string]$mainSpecDir, [string]$dest, [string[]]$specAdditionalSubDirectories) {
+    $source = "$specCloneRoot/$mainSpecDir"
+    Copy-Item -Path $source -Destination $dest -Recurse -Force
+    Write-Host "Copying spec from $source to $dest"
+
+    foreach ($additionalDir in $specAdditionalSubDirectories) {
+        $source = "$specCloneRoot/$additionalDir"
+        Write-Host "Copying spec from $source to $dest"
+        Copy-Item -Path $source -Destination $dest -Recurse -Force
+    }
+}
+
+function UpdateSparseCheckoutFile([string]$mainSpecDir, [string[]]$specAdditionalSubDirectories) {
+    AddSparseCheckoutPath $mainSpecDir
+    foreach ($subDir in $specAdditionalSubDirectories) {
+        AddSparseCheckoutPath $subDir
+    }
+}
+
+function GetGitRemoteValue([string]$repo) {
+    Push-Location $ProjectDirectory
+    $result = ""
+    try {
+        $gitRemotes = (git remote -v)
+        foreach ($remote in $gitRemotes) {
+            if ($remote.StartsWith("origin")) {
+                if ($remote -match 'https://github.com/\S+') {
+                    $result = "https://github.com/$repo.git"
+                    break
+                } elseif ($remote -match "[email protected]:\S+"){
+                    $result = "[email protected]:$repo.git"
+                    break
+                } else {
+                    throw "Unknown git remote format found: $remote"
+                }
+            }
+        }
+    }
+    finally {
+        Pop-Location
+    }
+
+    return $result
+}
+
+function InitializeSparseGitClone([string]$repo) {
+    git clone --no-checkout --filter=tree:0 $repo .
+    if ($LASTEXITCODE) { exit $LASTEXITCODE }
+    git sparse-checkout init
+    if ($LASTEXITCODE) { exit $LASTEXITCODE }
+    Remove-Item $sparseCheckoutFile -Force
+}
+
+function GetSpecCloneDir([string]$projectName) {
+    Push-Location $ProjectDirectory
+    try {
+        $root = git rev-parse --show-toplevel
+    }
+    finally {
+        Pop-Location
+    }
+
+    $sparseSpecCloneDir = "$root/../sparse-spec/$projectName"
+    New-Item $sparseSpecCloneDir -Type Directory -Force | Out-Null
+    $createResult = Resolve-Path $sparseSpecCloneDir
+    return $createResult
+}
+
+$typespecConfigurationFile = Resolve-Path "$ProjectDirectory/tsp-location.yaml"
+Write-Host "Reading configuration from $typespecConfigurationFile"
+$configuration = Get-Content -Path $typespecConfigurationFile -Raw | ConvertFrom-Yaml
+
+$pieces = $typespecConfigurationFile.Path.Replace("\","/").Split("/")
+$projectName = $pieces[$pieces.Count - 2]
+
+$specSubDirectory = $configuration["directory"]
+
+if ( $configuration["repo"] -and $configuration["commit"]) {
+    $specCloneDir = GetSpecCloneDir $projectName
+    $gitRemoteValue = GetGitRemoteValue $configuration["repo"]
+
+    Write-Host "Setting up sparse clone for $projectName at $specCloneDir"
+
+    Push-Location $specCloneDir.Path
+    try {
+        if (!(Test-Path ".git")) {
+            InitializeSparseGitClone $gitRemoteValue
+            UpdateSparseCheckoutFile $specSubDirectory $configuration["additionalDirectories"]
+        }
+        git checkout $configuration["commit"]
+        if ($LASTEXITCODE) { exit $LASTEXITCODE }
+    }
+    finally {
+        Pop-Location
+    }
+} elseif ( $configuration["spec-root-dir"] ) {
+    $specCloneDir = $configuration["spec-root-dir"]
+}
+
+
+$tempTypeSpecDir = "$ProjectDirectory/TempTypeSpecFiles"
+New-Item $tempTypeSpecDir -Type Directory -Force | Out-Null
+CopySpecToProjectIfNeeded `
+    -specCloneRoot $specCloneDir `
+    -mainSpecDir $specSubDirectory `
+    -dest $tempTypeSpecDir `
+    -specAdditionalSubDirectories $configuration["additionalDirectories"]
diff --git a/eng/common/scripts/git-branch-push.ps1 b/eng/common/scripts/git-branch-push.ps1
@@ -37,6 +37,10 @@ param(
     [boolean] $AmendCommit = $false
 )
 
+# Explicit set arg parsing to Legacy mode because some of the git calls in this script depend on empty strings being empty and not passing a "" git.
+# more info https://learn.microsoft.com/en-us/powershell/scripting/learn/experimental-features?view=powershell-7.3#psnativecommandargumentpassing
+$PSNativeCommandArgumentPassing = "Legacy"
+
 # This is necessary because of the git command output writing to stderr.
 # Without explicitly setting the ErrorActionPreference to continue the script
 # would fail the first time git wrote command output.

diff --git a/eng/common/scripts/stress-testing/deploy-stress-tests.ps1 b/eng/common/scripts/stress-testing/deploy-stress-tests.ps1
@@ -25,6 +25,8 @@ param(
     # Renders chart templates locally without deployment
     [Parameter(Mandatory=$False)][switch]$Template,
 
+    [Parameter(Mandatory=$False)][switch]$RetryFailedTests,
+
     # Matrix generation parameters
     [Parameter(Mandatory=$False)][string]$MatrixFileName,
     [Parameter(Mandatory=$False)][string]$MatrixSelection,