nuget not prompting for credentials when querying Azure Artifacts feed on WSL

[DSMitten]

Environment:
Ubuntu 22.04 or 24.04
mono 6.8.0.105
nuget 6.10.1.5
credential provider installed at:
~/.nuget/plugins/netcore/CredentialProvider.Microsoft
~/.nuget/plugins/netfx/CredentialProvider.Microsoft
Also installed at /usr/share/dotnet/8.0.303/Plugins/netcore|netfx

Repro:

mono /usr/local/bin/nuget.exe list ClientNativeHostLoc -Source https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json

Result:

WARNING: 'NuGet list' is deprecated. Use 'NuGet search' instead.
MSBuild auto-detection: using msbuild version '' from '/usr/bin'.
WARNING:     [CredentialProvider]Warning: Cannot persist Microsoft authentication token cache securely!
WARNING:     [CredentialProvider]Warning: Using plain-text fallback token cache
Please provide credentials for: https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json
UserName:

Expect:
IIRC this command used to put a URL at the command line, with a code, and you would navigate to that URL, authenticate in the browser, and then the nuget operation would complete.

When run with increased verbosity, this is the log:
/usr/local/bin/nuget list ClientNativeHostLoc -Source https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json -Verbosity detailed
[CredentialProvider]Sending response: 'Request' 'SetLogLevel'. Time elapsed in ms: 1
[CredentialProvider]Time elapsed in milliseconds after sending response 'Request' 'SetLogLevel': 4
[CredentialProvider]Handling 'Request' 'GetAuthenticationCredentials'. Time elapsed in ms: 8 - Payload: {"Uri":"https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json","IsRetry":false,"IsNonInteractive":false,"CanShowDialog":true}
[CredentialProvider]Creating a progress reporter with interval: 00:00:02
[CredentialProvider]Handling auth request, Uri: https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json, IsRetry: False, IsNonInteractive: False, CanShowDialog: True
[CredentialProvider]URI: https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json
[CredentialProvider]VstsBuildTaskServiceEndpointCredentialProvider - This credential provider must be run under the Team Build tasks for NuGet with external endpoint credentials. Appropriate environment variable needs to be set.
[CredentialProvider]Skipping NuGetCredentialProvider.CredentialProviders.VstsBuildTaskServiceEndpoint.VstsBuildTaskServiceEndpointCredentialProvider, cannot provide credentials for https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json
[CredentialProvider]VstsBuildTaskCredentialProvider - This credential provider must be run under the Team Build tasks for NuGet. Appropriate environment variables must be set.
[CredentialProvider]Skipping NuGetCredentialProvider.CredentialProviders.VstsBuildTask.VstsBuildTaskCredentialProvider, cannot provide credentials for https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json
[CredentialProvider]VstsCredentialProvider - Matched well-known Azure DevOps Service hostname: outlookweb.pkgs.visualstudio.com
[CredentialProvider]Using NuGetCredentialProvider.CredentialProviders.Vsts.VstsCredentialProvider to try to get credentials for https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json.
[CredentialProvider]IsRetry: False
[CredentialProvider]Could not find cached SessionToken for https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json
[CredentialProvider]GET https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json
[CredentialProvider]Found AAD Authority from 401 headers: https://login.windows.net/72f988bf-86f1-41af-91ab-2d7cd011db47
[CredentialProvider]VstsCredentialProvider - Using AAD authority: https://login.windows.net/72f988bf-86f1-41af-91ab-2d7cd011db47
[CredentialProvider]Using MSAL cache at /home/smitten/.local/.IdentityService/msal.cache
WARNING: [CredentialProvider]Warning: Cannot persist Microsoft authentication token cache securely!
[CredentialProvider]Microsoft.Identity.Client.Extensions.Msal.MsalCachePersistenceException: Persistence check failed. Inspect inner exception for details ---> System.TypeInitializationException: The type initializer for 'Microsoft.Identity.Extensions.Mac.SecurityFramework' threw an exception. ---> System.DllNotFoundException: /usr/lib/libSystem.dylib assembly: type: member:(null)
WARNING: [CredentialProvider]Warning: Using plain-text fallback token cache
[CredentialProvider]VstsCredentialProvider - Attempting to acquire bearer token using provider 'MSAL Silent'
UserName: [CredentialProvider]VstsCredentialProvider - Bearer token provider 'MSAL Silent' failed with exception:\nSystem.NullReferenceException: Object reference not set to an instance of an object
[CredentialProvider]VstsCredentialProvider - Attempting to acquire bearer token using provider 'MSAL Interactive'
[CredentialProvider]VstsCredentialProvider - Bearer token provider 'MSAL Interactive' failed with exception:\nSystem.NullReferenceException: Object reference not set to an instance of an object
[CredentialProvider]VstsCredentialProvider - Attempting to acquire bearer token using provider 'MSAL Device Code'
[CredentialProvider]VstsCredentialProvider - Bearer token provider 'MSAL Device Code' failed with exception:\nSystem.NullReferenceException: Object reference not set to an instance of an object
[CredentialProvider]VstsCredentialProvider - Could not obtain credentials for https://outlookweb.pkgs.visualstudio.com/_packaging/owa-nuget/nuget/v3/index.json
[CredentialProvider]Unable to acquire credentials.
[CredentialProvider]Sending response: 'Request' 'GetAuthenticationCredentials'. Time elapsed in ms: 537
[CredentialProvider]Time elapsed in milliseconds after sending response 'Request' 'GetAuthenticationCredentials': 538

[embetten]

Can you confirm the version of the credential provider is the latest version. Additionally, can you provide the dotnet runtimes/SDKs on the machine (you can use the command dotnet --list-runtimes and dotnet --list-sdks) - is it just 8.0.303 ? thanks

[DSMitten]

Multiple versions of the credential manager were attempted.
We were using release 1.0.9, but when that didn't work I tried 1.1.1 and then downloaded from https://github.com/microsoft/artifacts-credprovider/releases/download/v1.2.0-alpha

I meant to include the versions of .NET that are installed:

This is from the Ubuntu 22.04 WSL VM:

PS /home/smitten/git/client-native-host/external/nuget> dotnet --list-runtimes
Microsoft.AspNetCore.App 8.0.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 8.0.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

PS /home/smitten/git/client-native-host/external/nuget> dotnet --list-sdks    
8.0.303 [/usr/share/dotnet/sdk]

PS /home/smitten/git/client-native-host/external/nuget> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      7.4.3
PSEdition                      Core
GitCommitId                    7.4.3
OS                             Ubuntu 22.04.3 LTS

This is from the Ubuntu 24.04 machine:

PS /home/smitten/git/client-native-host> dotnet --list-runtimes   
Microsoft.AspNetCore.App 8.0.7 [/usr/lib/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 8.0.7 [/usr/lib/dotnet/shared/Microsoft.NETCore.App]

PS /home/smitten/git/client-native-host> dotnet --list-sdks    
8.0.107 [/usr/lib/dotnet/sdk]

PS /home/smitten/git/client-native-host> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      7.4.3
PSEdition                      Core
GitCommitId                    7.4.3
OS                             Ubuntu 24.04 LTS

[embetten]

I was able to reproduce this issue locally. The netfx cred provider is being called because we are using nuget.exe and the logs show the error is in MSAL. Looks like MSAL has a known issue with netfx and mono AzureAD/microsoft-authentication-library-for-dotnet#3138 that they are not planning to fix and need to use .net core to avoid. If nuget is invoked via dotnet, it will use the .net core credprovider. So, the only known workaround would be to invoke the equivalent dotnet command :

dotnet package search ClientNativeHostLoc --source <packageSource> --interactive

We will need to pull in NuGet to discuss long term direction for netfx.

[DSMitten]

Is there a way to download a package with dotnet that doesn't involve creating an msbuild project? We need to run nuget install to get packages locally into specific directories. If we just needed to install packages for msbuild projects, we would be using dotnet restore already.

As far as I can tell, nuget install is the only option that allows you to simply install a package in the specified location.

[embetten]

@DSMitten, I am not aware of a dotnet install command that does install without the project or solution file.

I was able to locally unblock the cred provider by manually setting the registry referenced in this comment to set the windows registry value on WSL. To do this manually, I created the follow file
/etc/mono/registry/LocalMachine/software/microsoft/windows nt/currentversion/values.xml with the contents:

<values>
<value name="ProductName"
type="string">Mono</value>
</values>

Then I had to change the permissions to the file give the cred provider access to read. Disclaimer - I have not looked into the ramifications of adding this reg value to mono - I was just testing that this was technically possible and unblocked the cred provider here.

For a code fix to the cred provider, we can look into adding something to the program file for netfx that is similar to:

#if NETFRAMEWORK
                using (var reg = Microsoft.Win32.Registry.LocalMachine.CreateSubKey(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion", true))
                {
                    reg.SetValue("ProductName", "Mono");
                }
#endif

But we will need more time to fully understand this addition and figure out the permissions for the cred provider to write to the above file. It looks like we are blocked from the above code fix as we cannot run the cred provider with sudo to write to the file due to this issue.

[roklenardic]

I am also not able to execute "dotnet restore" with a Azure Devops secured artefacts source, despite having used:

• dotnet 8.0.110
• latest credential provider 1.3.0
• installed libsecret-1-0
• manually added the source with the PAT using "dotnet nuget add source" command
• tried Ubuntu 22.04 or 24.04, both on WSL 1 and 2

Consequently we cannot execute the build on Github actions at all.

[embetten]

@roklenardic this might deserve its own separate issue. But I think you want to either use the dotnet nuget add source command to put the PAT in your NuGet config file OR use the credential provider.

If you want the cred provider and you have completed the steps above to add mono correctly:
option 1: I think you should try setting PAT token as one of the supported pipeline build env variables (from the readme):

Build Provider URI Prefixes
    VSS_NUGET_URI_PREFIXES
        Semi-colon separated list of hosts the build provider supports.

Build Provider Access Token
    VSS_NUGET_ACCESSTOKEN
        The Personal Access Token that will be returned as credentials by
        the build provider.

option 2: If github actions allows interactive login, and you want to use that please ensure there is a browser or interactive window available. Additionally, providing more detailed logs here would be helpful.

[roklenardic]

@roklenardic this might deserve its own separate issue. But I think you want to either use the dotnet nuget add source command to put the PAT in your NuGet config file OR use the credential provider.

If you want the cred provider and you have completed the steps above to add mono correctly: option 1: I think you should try setting PAT token as one of the supported pipeline build env variables (from the readme):

Build Provider URI Prefixes
    VSS_NUGET_URI_PREFIXES
        Semi-colon separated list of hosts the build provider supports.

Build Provider Access Token
    VSS_NUGET_ACCESSTOKEN
        The Personal Access Token that will be returned as credentials by
        the build provider.

option 2: If github actions allows interactive login, and you want to use that please ensure there is a browser or interactive window available. Additionally, providing more detailed logs here would be helpful.

Might be slightly different since I am not aiming to use interactive mode. But essentially the resulting problem I believe is the same, issue with secret storage in Linux, is it not?

I tried all methods for setting the PAT token apart from encoding it in the nuget.config which is cleartext and not acceptable for us.