Automate Azure Arc testing

[joshfree]

Tracking issue for step-by-step instructions for how to manually run e2d tests for Azure Arc hosted environments for each of the azure-sdk-for-* languages. This issue will be used by our Vendor team for manually running tests in this scenario, and will eventually be referenced by Central-EngSys for adding this environment to our automated test runs.

Assigning to @catalinaperalta who will create the initial instructions specific to Azure SDK for Go. Other SDK languages will be appended to this issue as comments.

[catalinaperalta]

Testing Azure Identity in Azure Arc Environments

Prerequisites

1. A non-Azure Windows VM with Azure Arc installed on it. (See Installing Azure Arc on the VM on steps to follow)
2. Administrator privileges on the VM.

Installing Azure Arc on the VM

Please note, you must be in your VM to install Azure Arc.

1. Create an Azure Arc service instance on Azure Portal.
2. Click on your Azure Arc service instance and on the main page, under the "Getting Started" tab select Add in the "Add your existing infrastructure" section.
3. On the next page click on Add under "Servers".
4. When configuring the Arc server, on the "Resource Details" tab select the right resource group, region and operating system for your specific VM.
5. No other configuration is necessary. You can go to the "Download and run script tab" and click on Download. This will download the installation script.
6. Once the script has been downloaded, run the script on your machine with administrator privileges.
7. If using a Linux VM, run the following commands (using your user name for <user>) to gain necessary privileges:

sudo usermod -a -G himds <user>
sudo setfacl -m "g:himds:r-x" /var/opt/azcmagent/tokens/
sudo setfacl -m "g::r-x" /var/opt/azcmagent/tokens/

8. Installation should be complete at this point. You should restart your computer to finalize the new configuration.
9. Check your installation by checking for the IDENTITY_ENDPOINT. (In Powershell on Windows: echo $env:IDENTITY_ENDPOINT). The output should be similar to: http://localhost:40342/metadata/identity/oauth2/token.

Run the Azure Identity Test on the Azure Arc VM

The following steps are specific to Go

Install Go

1. Download and run the Go installer for Windows at https://golang.org/dl/.
2. In a Powershell window run go version to check Go installation.

Download the repo and run the test

1. In a Powershell window run: git clone https://github.com/Azure/azure-sdk-for-go.git.
2. Change directory to the azidentity package: cd azure-sdk-for-go/sdk/azidentity.
3. Open a new Powershell window as Administrator.
4. Run the test: go test -run TestManagedIdentityCredential_GetTokenInAzureArcLive
5. Expected output: PASS

[mccoyp]

Testing azure-identity in Azure Arc (in Python)

Prerequisite tools

1. A non-Azure Windows or Linux VM.
2. Administrator privileges on the VM.
3. An Azure Key Vault.
4. Python 2.7 and 3.5+

Install Azure Arc on the VM

Note: You must be in your VM to install Azure Arc.

1. Create an Azure Arc server resource on the Azure Portal (at the time of writing, the
   resource is named "Servers - Azure Arc").
2. Choose to add an existing server using an interactive script.
3. When creating the resource, fill in your desired subscription, resource group, and region for the VM. Choose the
   operating system of your existing VM.
4. No other configuration is necessary. You can go to the "Download and run script" tab and download the script shown.
5. Once the script has been downloaded, run the script on your machine with administrator privileges.
6. If using a Linux VM, run the following commands (using your user name for <user>) to gain necessary privileges:

sudo usermod -a -G himds <user>
sudo setfacl -m "g:himds:r-x" /var/opt/azcmagent/tokens/
sudo setfacl -m "g::r-x" /var/opt/azcmagent/tokens/

7. Arc setup should now be complete. Restart your VM to finalize your environment setup.
8. After restarting, check your environment by searching for environment variables named IDENTITY_ENDPOINT and
   IMDS_ENDPOINT. If they are not present, or don't resemble http://localhost:40342/metadata/identity/oauth2/token and
   http://localhost:40342 respectively, you may need to wait a short while or try restarting the VM again.

Give the Azure Arc VM access to the key vault

For the tests to pass, the VM will need secret management permissions in your key vault.

1. Go to your key vault resource in the Azure Portal.
2. Go to the vault's "Access policies" page, and click "Add Access Policy".
3. Using the secret management template, select your Arc VM resource as the principal.
4. Click "Add".
5. Don't forget to click "Save" at the top of the access policies page after the policy is added.

Run the azure-identity Tests on the Azure Arc VM

Note: The following steps are specific to Python.

In a terminal window, run:

git clone https://github.com/Azure/azure-sdk-for-python --single-branch --branch master --depth 1
cd azure-sdk-for-python/sdk/identity/azure-identity/tests/managed-identity-live

Set the environment variable AZURE_IDENTITY_TEST_VAULT_URL to the vault URI of your key vault.

Install requirements.txt:

pip install -r requirements.txt

Run the managed identity tests, using the below command once with Python 2.7 and once with Python 3.5+:

pytest -k managed_identity_live

Expected output for each: passed for all tests run.

[JosueJoshua]

Hi @catalinaperalta ,the branch addAzureArcAuthentication cannot be found in the repo 'azure-sdk-for-go'.

[catalinaperalta]

@JosueJoshua I fixed the instructions, you can simply clone the repo and use the content that's in master now. Thanks!

[sadasant]

Testing azure-identity in Azure Arc (in JavaScript)

Prerequisite tools

1. A non-Azure Windows or Linux VM.
2. Administrator privileges on the VM.
3. An Azure Key Vault.
4. NodeJS 8 or higher.

Install Azure Arc on the VM

Note: You must be in your VM to install Azure Arc.

1. Create an Azure Arc server resource on the Azure Portal (at the time of writing, the
   resource is named "Servers - Azure Arc").
2. Choose to add an existing server using an interactive script.
3. When creating the resource, fill in your desired subscription, resource group, and region for the VM. Choose the
   operating system of your existing VM.
4. No other configuration is necessary. You can go to the "Download and run script" tab and download the script shown.
5. Once the script has been downloaded, run the script on your machine with administrator privileges.
6. If using a Linux VM, run the following commands (using your user name for <user>) to gain necessary privileges:

sudo usermod -a -G himds <user>
sudo setfacl -m "g:himds:r-x" /var/opt/azcmagent/tokens/
sudo setfacl -m "g::r-x" /var/opt/azcmagent/tokens/

7. Arc setup should now be complete. Restart your VM to finalize your environment setup.
8. After restarting, check your environment by searching for environment variables named IDENTITY_ENDPOINT and
   IMDS_ENDPOINT. If they are not present, or don't resemble http://localhost:40342/metadata/identity/oauth2/token and http://localhost:40342 respectively, you may need to wait a short while or try restarting the VM again.

Give the Azure Arc VM access to the key vault

For the tests to pass, the VM will need key management permissions in your key vault.

1. Go to your key vault resource in the Azure Portal.
2. Go to the vault's "Access policies" page, and click "Add Access Policy".
3. Using the key management template, select your Arc VM resource as the principal.
4. Click "Add".
5. Don't forget to click "Save" at the top of the access policies page after the policy is added.

Run the azure-identity Tests on the Azure Arc VM

Note: The following steps are specific to JavaScript.

Create a new folder for the identity test:

mkdir identity-test

Change directory to that folder with cd identity-test, then copy the following content in a file named package.json:

{
  "name": "identity-test",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "start": "node index.js"
  },
  "keywords": [],
  "author": "",
  "license": "MIT",
  "dependencies": {
    "@azure/identity": "1.2.0",
    "@azure/keyvault-keys": "4.1.0",
  }
}

Then, copy the following code in a file named index.js (Make sure to replace key-vault-name with the name of your Key Vault):

const { ManagedIdentityCredential } = require("@azure/identity");
const { KeyClient } = require("@azure/keyvault-keys");

async function main() {
  const credential = new ManagedIdentityCredential();
  const keyVaultUrl = `https://key-vault-name.vault.azure.net`;
  const client = new KeyClient(keyVaultUrl, credential);

  // Retrieving the properties of the existing keys in that specific Key Vault.
  // This might log empty if no keys have been created in the given Key Vault.
  console.log(await client.listPropertiesOfKeys().next());

  console.log("Test succeeded!");
}
main().then(console.log).catch((e) => console.error(e));

With the package.json and the index.js, you'll be able to install the dependencies with: npm install in that directory, then run the test by executing node index.js. If you see Test succeeded! in the console, then Identity has successfully authenticated for your VM through Azure Arc.

[JosueJoshua]

Hi @sadasant , it's using secret management template to add access policy in the third step, perhaps it should use key management template.

[sadasant]

@JosueJoshua that is true! thank you!

[sadasant]

I updated it!

[maorleger]

Testing Identity in Azure Arc (Latest JavaScript instructions)

These instructions are also available here

Prerequisites

• A non-Azure Windows or Linux virtual machine with NodeJS, NPM, and git installed.
• Administrator privileges on the VM.
• An Azure Key Vault.

Install Azure Arc on the VM

Note: You must be in your VM to install Azure Arc.

1. Create an Azure Arc server resource on the Azure Portal (at the time of writing, the
   resource is named "Azure Arc").
2. Choose to add an existing server using an interactive script.
3. When creating the resource, fill in your desired subscription, resource group, and region for the VM. Choose the
   operating system of your existing VM.
4. No other configuration is necessary. You can go to the "Download and run script" tab and download the script shown.
5. Once the script has been downloaded, run the script on your machine with administrator privileges.
6. If using a Linux VM, run the following commands (using your user name for <user>) to gain necessary privileges:

sudo usermod -a -G himds <user>
sudo setfacl -m "g:himds:r-x" /var/opt/azcmagent/tokens/
sudo setfacl -m "g::r-x" /var/opt/azcmagent/tokens/

7. The Azure Arc setup should now be complete. Restart your VM to finalize your environment setup.
8. After restarting, check your environment by searching for environment variables named IDENTITY_ENDPOINT and
   IMDS_ENDPOINT. If they are not present, or don't resemble http://localhost:40342/metadata/identity/oauth2/token and
   http://localhost:40342 respectively, you may need to wait a short while or try restarting the VM again.

Give the Azure Arc VM access to the key vault

For the tests to pass, the VM will need secret management permissions in your key vault.

1. Go to your key vault resource in the Azure Portal.
2. Go to the vault's "Access policies" page, and click "Add Access Policy".
3. Using the secret management template, select your Arc VM resource as the principal.
4. Click "Add".
5. Don't forget to click "Save" at the top of the access policies page after the policy is added.

Run the azure-identity Tests on the Azure Arc VM

Note: The following steps are specific to JavaScript.

In a terminal window, run:

git clone https://github.com/Azure/azure-sdk-for-js --single-branch --depth 1
cd azure-sdk-for-js/sdk/identity/identity/test/manual-integration/AzureArc

Set the environment variable KEYVAULT_URI to the vault URI of your key vault.

Install dependencies:

npm install

Compile the test file using TypeScript:

npm run build

Run the test file:

node dist/index.js

Expected output: Successfully authenticated with Key Vault!

[antkmsft]

Testing ManagedIdentityCredential in Azure Arc (in C++)

Set up a VM, Azure Resources, and Azure Arc just as you would for Go, Python, or JavaScript.

run Visual Studio Installer
Press Modify
Select "Desktop development with C++"
Go to Individual Components tab
search for "git"
Select "Git for Windows"
Press Modify, wait for installation to finish (usually takes about 10 minutes), close the Installer

Open x64 Native Tools Commond Prompt

cd /D %USERPROFILE%
git clone https://github.com/microsoft/vcpkg.git
cd vcpkg
bootstrap-vcpkg.bat
vcpkg install curl:x86-windows openssl:x86-windows libxml2:x86-windows curl:x64-windows openssl:x64-windows libxml2:x64-windows curl:x64-windows-static openssl:x64-windows-static libxml2:x64-windows-static

Wait for the installation to finish (about 20 minutes)

cd ..
git clone https://github.com/Azure/azure-sdk-for-cpp.git
cd azure-sdk-for-cpp
cmake -B build -S . -DCMAKE_TOOLCHAIN_FILE=%USERPROFILE%\vcpkg\scripts\buildsystems\vcpkg.cmake -DBUILD_TESTING=ON
cmake --build build

The build usually takes up to 5 minutes.

build\sdk\identity\azure-identity\test\e2e\Debug\azure-identity-e2e-test.exe

Watch the output. It will either print "OK" + print "SUCCESS" in large letters, or it will print error details + the relevant environment.

Successful executions are usually instant, in case of failures the test may wait for up to a minute before printing the error details. This is caused by the credential going through retries before finally giving up.

[AFengKK]

Hi, @mccoyp
When test the Testing azure-identity in Azure Arc (in Python) and run the pytest -k managed_identity_live command, it displays the error :

Through the investigation , the token cannot be obtained. I tried to delete the 'tenant_id' parameter of the code self.authorize_request(request, scope, tenant_id=challenge.tenant_id) in the challenge_auth_policy.py, and it works normally. Can you provide some help on this issue?

[mccoyp]

Hi @AFengKK, sorry for the late reply! You're absolutely right about this, thank you for finding this bug. The latest preview versions of Key Vault packages include a tenant_id in challenge auth requests since azure-identity has started supporting this feature, but ManagedIdentityCredential is broken in this scenario because it passes the keyword argument too far down the chain.

We're working on a fix for this now, but in the meantime, using the latest stable version of azure-keyvault-secrets (4.3.0) -- or removing tenant_id from the token request you mention -- should succeed.

[zzkaii]

Hi,@joshfree
When test the Testing azure-identity in Azure Arc(in go) nd run the go test -run TestManagedIdentityCredential_AzureArc command,it display the error:

Through the investigation , I tried to use internal v0.8.2, it works normally. But use internal v0.8.3 and later version, it can't work. Can you provide some help on this issue?

[AFengKK]

Hi, @mccoyp, Thank you for your help. The test can pass with the stable version of azure-keyvault-secrets (4.3.0). In addition, if the bug is fixed, please let me know.

[joshfree]

@zzkaii I filed Azure/azure-sdk-for-go#17143 in the Go repo to track the issue / question.

[mccoyp]

Hi @AFengKK, the latest stable release of azure-identity (1.8.0) is now available and has a fix for the ManagedIdentityCredential issue you discovered -- thank you again! This new version of azure-identity should be compatible with any version of azure-keyvault-secrets. Please let me know if you run into any other issues!

[v-xuto]

@mccoyp We have run the tests use azure-identity (1.8.0), and its result is pass. Thanks for your help.

[chlowell]

Testing azidentity on Azure Arc (latest Go instructions)

Prerequisites

• A non-Azure Windows or Linux VM with Go and git installed
• Administrator privileges on the VM

Install Azure Arc on the VM

1. Create an Azure Arc server resource on the Azure Portal (at the time of writing, the resource is named "Servers - Azure Arc")
2. Choose to generate a script for a single server
3. Run the script on the VM with administrator privileges
4. If using a Linux VM, run the following commands (substituting your user name for <user>) to gain necessary privileges:

sudo usermod -a -G himds <user>
sudo setfacl -m "g:himds:r-x" /var/opt/azcmagent/tokens/
sudo setfacl -m "g::r-x" /var/opt/azcmagent/tokens/

5. Restart your VM to finalize setup
6. Check your environment variables IDENTITY_ENDPOINT and IMDS_ENDPOINT. If they aren't set, you may need to wait a short while or restart the VM again.

Run the test on the Azure Arc VM

In a terminal window, run:

git clone https://github.com/Azure/azure-sdk-for-go --single-branch --depth 1
cd azure-sdk-for-go/sdk/samples/azidentity/manual-tests/managed-identity/general
go run main.go

Expected output: Success! Token received.

[v-jiaodi]

Hi, @mccoyp. We have two problems when test the Testing azure-identity in Azure Arc (in Python).

1.When test the Testing azure-identity in Azure Arc (in Python) and run the pytest -k managed_identity_live command, it displays the error :

Through the investigation, we need to install tools azure-devtools and azure-sdk-tools, so we need to add code as follow in requirements.txt.

../../../../../tools/azure-devtools
../../../../../tools/azure-sdk-tools

Maybe we can create a PR to fix this issue.

2.Rerun pytest -k managed_identity_live and get a new error as follow：

Through the investigation, we need to configure the test-proxy to pass the test. So, it is suggested to add relevant descriptions in the test process.

[mccoyp]

Hi @v-jiaodi, thank you for the summary! I'm sorry about the delayed response.

Getting the test proxy set up in the test environment may be possible, but it would likely be more difficult than is worth attempting. I created a tracking issue (Azure/azure-sdk-for-python#26237) to address this and can update this thread when it's completed.

[Menghua1]

Hi @antkmsft. Following the steps above, while testing ManagedIdentityCredential in Azure Arc (in C++), we encountered an error after executing the command:

build\sdk\identity\azure-identity\test\e2e\Debug\azure-identity-e2e-test.exe

The detailed error information is as follows:

Could you help to check and resolve the error?

[github-actions]

Hi @joshfree, we deeply appreciate your input into this project. Regrettably, this issue has remained inactive for over 2 years, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.

[v-xuto]

@joshfree This issue has been closed. Do we need to continue testing this Automate Azure Arc testing?