Key Vault integration in local dev environment

[ChristianWeyer]

I am playing around with the preview of the Key Vault integration as outlined here:
https://azure.microsoft.com/en-us/blog/simplifying-security-for-serverless-and-web-apps-with-azure-functions-and-app-service/
and here:
https://docs.microsoft.com/en-us/azure/app-service/app-service-key-vault-references

This all looks fine.

But what is the story for local development in such cases?
Currently, I just get back the original string value of the app setting, like @Microsoft.KeyVault(SecretUri=https://...

Thanks!

[FrankHoogmans]

I'm questioning the same thing. The Key Vault references don't seem to work from a local dev environment, or I'm doing something wrong.

[zmarty]

For now this feature does not seem to be supported, as confirmed by Azure Functions team. We implemented an incomplete local workaround like so:

• Instead of calling Environment.GetEnvironmentVariable, we call our own function called RetrieveVerifyConfigValue
• Our function detects if the code runs locally. We do that with this line: var isLocal = string.IsNullOrEmpty(Environment.GetEnvironmentVariable("WEBSITE_INSTANCE_ID"));
• If the code does NOT run locally, just return what we get from Environment.GetEnvironmentVariable
• If the code DOES run locally, perform certificate based authentication to Azure Key Vault, then return the requested secret.

Caveat: this workaround does not work if your Azure Function uses the [StorageAccount("...")] attribute, because that one bypasses our function.

[Zenuka]

Is this feature on the roadmap? I really would like that all developers just use integrated security with RBAC to access resources instead of having access keys at developers machines to our servicebus and storage accounts. Since we use those as triggers.

[davidbarrows]

I'm experiencing this problem also. What it means is, if the developer if forbidden to access the key vault in the deployed azure environment, there's no way to actually test locally whether your code is working. The "get environment variable" is a hack, and means that the way you get the value locally is different from the way it actually works in the environment. Total PITA.

[kashimiz]

cc @mattchenderson

[mattchenderson]

No ETA for this, but still something we are tracking. Some of the core underpinnings require some revision, but we agree this is an important scenario.

[TroyWitthoeft]

Just ran into this myself. Having the ability for local development to effortlessly use a remote key vault would be a boon to development speed, security, and would encourage the use of Microsoft's KMS.

[JosXa]

• If the code DOES run locally, perform certificate based authentication to Azure Key Vault, then return the requested secret.

@zmarty How do you parse away the @Microsoft.KeyVault(SecretUri=...) part? Is there a builtin for that, or can you just pass this whole string to some method of one of the Microsoft Key Vault libraries?
Right now I'll hack this together using a regex, but for obvious reasons that's not how it should be done.

[zmarty]

@JosXa

No, there is no built-in function that I know of.

I would not use regular expressions for such a simple text parsing task.

I use something like this:

private const string KeyVaultValuePrefix = "@Microsoft.KeyVault(SecretUri=";
private const string KeyVaultValuePostfix = ")";

...

// How to check Azure function is running on local environment? - https://stackoverflow.com/questions/45026215/how-to-check-azure-function-is-running-on-local-environment-roleenvironment-i
var isLocal = string.IsNullOrEmpty(Environment.GetEnvironmentVariable("WEBSITE_INSTANCE_ID"));

if (isLocal
    && configValue.StartsWith(KeyVaultValuePrefix)
    && configValue.EndsWith(KeyVaultValuePostfix))
{
    var secretUri = configValue.Substring(KeyVaultValuePrefix.Length, configValue.Length - KeyVaultValuePrefix.Length - KeyVaultValuePostfix.Length);

    configValue = await GetVaultValue(secretUri);

    if (string.IsNullOrWhiteSpace(configValue))
    {
        throw new Exception($"After reading from Key Vault, config value for key {keyName} had a null or empty value");
    }
}

[JosXa]

@zmarty Alright, this works, but sounds really flaky to me. Will this expression ever change? Are there more arguments possible than just SecretUri? Not the most robust solution we've got going on here.

Here's my solution to this (including support for certificates), maybe someone finds it useful as a base:
https://gist.github.com/JosXa/2e1925c58077fe97871bb56697128355

I'd urge the functions team to include local key vault refs out of the box :)

[k-rush]

@mattchenderson we'd love to see this feature prioritized, it will absolutely make life easier for developing functions locally in a distributed team.

[robinmanuelthiel]

This is not only important for local development, but also for running Azure Functions in containers, where the KeyVault references also don’t work!

[merfolk]

It seem strange that Azure Keyvault is not supported for local development of Azure Functions.

Being able to use the keyvault for local development would make development both easier and more secure.

[JiCi]

I am developing apps with KeyVault, including Azure Functions, and able to access/test on my local box. I did this some time ago and am just revisiting it for a new app. So, these are the general steps that should get it running. The short of it is you can give yourself permissions for keyvault, and then setup Visual Studio to use your user credential for Azure Auth as a developer.

First you need to give yourself permissions in KeyVault. Make sure you (your VS login) show up in Access Control and Access Policies. If not, add your login. Once this is done your login name should be a link.. click on it to get your user "Object ID".

This next step may be redundant with the last.. I have not done this in a while.

Get into the CLI. Make sure you are logged in via 'az login' etc.
az keyvault set-policy --name <yourKeyVaultName> --object-id <yourUserObjectId> --secret-permissions get list

You should get back a lot of info about KeyVault which includes your user ObjectId in 'accessPolicies' with proper permissions.

Now setup VS on dev box (PC anyway):

Create an environment variable. It should look like this when done if you type 'set' from cmd prompt:
AzureServicesAuthConnectionString=RunAs=Developer; DeveloperTool=VisualStudio

In Visual Studio go to: Tools -> Options -> Azure Service Authentication
Select your username

NOTE: I have found it probable that you need to logout and back into VS to make this work right... but only once for each new solution.

This should allow you to run locally with the same KeyVault access code as you would deploy with.
Hope this helps someone.

[CasperWSchmidt]

Any news on this feature?

[Hao-Qian]

It's interesting that azure web apps support using key vault in local development but Function app does not support. Really appreciate if it can be prioritised. It will save a lot of dev/testing time.

[MrTantum]

Caveat: this workaround does not work if your Azure Function uses the [StorageAccount("...")] attribute, because that one bypasses our function.

The same problem occurs for the ServiceBusAccount property.

We have moved all of our secrets to azure key vault and are quite happy that nearly everything azure key vault related can use Azure.Core.TokenCredential (e.g. the data protection apis). Would be nice to have a similar extension logic for the azure function trigger attributes (StorageAccountAttribute and ServiceBusAccountAttribute).

[ThomasPiskol]

So this issue is open since almost 3 years and there's a PR (Azure/azure-functions-core-tools#2258) available since almost one year. It seems that a solution is already available but not yet integrated.

What is blocking you from fixing this issue?

[mithunshanbhag]

@ThomasPiskol: I think, the originally submitted PR needs to be rebased against the latest main branch updates.

CC: @anthonychu who submitted the original PR.

[uda832]

Any update on this?

[aegpoadan]

I've created a new PR that adds the desired functionality. It is up-to-date with the v3.x branch. You can check it out here: Azure/azure-functions-core-tools#2806
CC @anthonychu

[CasperWSchmidt]

Any news on this?

[anthonychu]

@michaelpeng36 Do you have a chance to review Azure/azure-functions-core-tools#2806?

[aegpoadan]

The v3.x PR has been merged and a port for v4.x has been created here: Azure/azure-functions-core-tools#2924

[aegpoadan]

The v4.x PR has been merged.

[agravity-philipp]

Since this is still open I am curious if this includes the Connection parameter of BlobTrigger?

Because I receive:
Microsoft.Azure.WebJobs.Host: Error indexing method 'FunctionBlob'. System.Private.CoreLib: Exception has been thrown by the target of an invocation. Azure.Storage.Blobs: No valid combination of account information found.
on startup.

My local.settings.json on my local dev machine contains the Key Vault reference:
"StorageConnectionString": "@Microsoft.KeyVault(SecretUri=https://xxxxxxx.vault.azure.net/secrets/StorageConnectionString/<keyversion>)"

My blob trigger looks like this:
[FunctionName("FunctionBlob")] public void Run([BlobTrigger("testcontainer/{name}", Connection = "StorageConnectionString")] Stream myBlob, string name, ILogger log) { log.LogInformation($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {myBlob.Length} Bytes"); }

[CodyLeeWhite]

can we get documentation on this feature? i find it really useful but my company is skittish on using it since it's undocumented. also finding it in the first place was hard. this seems to be the only place on the internet that tells you use can use the keyvault reference syntax locally for azure functions local.setting.json.

[jstarkadf]

Has this been implemented? Is there documentation for this? It still does not work for me. When I run locally with a local.settings.json value of "MyConnectionString": "@Microsoft.KeyVault(...)" and a service bus trigger defined as [ServiceBusTrigger(topicName: "mytopic", subscriptionName: "mysubscription", Connection = "MyConnectionString")], I get the error on startup:

The listener for function 'Functions.MyFunction' was unable to start. Azure.Messaging.ServiceBus: The connection string used for an Service Bus client must specify the Service Bus namespace host and either a Shared Access Key (both the name and value) OR a Shared Access Signature to be valid. (Parameter 'connectionString').

If I replace the @Microsoft.KeyVault(...) with the actual connection string it works fine. I'm on .NET 8 and Visual Studio 2022.

EDIT:
Actually, I just tried again and my issue was I needed to reauth Visual Studio w/ my domain account. So it works but the error message when auth fails could be better.

[TroyWitthoeft]

@jstarkadf - FWIW, I was curious and just tried this locally and it worked. However, my setup was different. Using python and http trigger, and I was simply setting an internal variable from key-vault referenced env var in my local.settings.json.

Something that stands out to me is you are trying to use the key vault reference inside of the ServiceBusTrigger attribute.
[ServiceBusTrigger(topicName: "mytopic", subscriptionName: "mysubscription", Connection = "MyConnectionString")]

I know you probably NEED it there, but just to test, have you tried accessing a key-vault referenced in the body of the service bus trigger? Try accessing the var outside of the attribute to test if that works?

string myConnectionString = Environment.GetEnvironmentVariable("MyConnectionString");

Does it fail there too?

It may be that key-vault references have a limitation and can't be used inside of trigger attributes? My thinking is that those attributes are pretty early in the function app setup.

[jstarkadf]

@TroyWitthoeft Thanks for your quick reply. I made an edit to my post and it does indeed work. It was an authorization issue where Visual Studio needed reauthorized to Azure. It would be helpful if an auth error would provide a useful error message.