Android: Installed X509 certificates can't be used with SSL authentication

[dotMorten]

Description

On Android when using the SocketsHttpHandler with X509 Client certificates to PKI authenticate with a server, you can only use certificates with a private key coming from a file. When using the more correct and secure way of only using certificates installed in the keychain authentication fails, because the SocketsHttpHandler wants direct access to the private key instead of using the Java APIs to authenticate.

Reproduction Steps

Reproduction steps are a little tricky since they require a server that requires PKI authentication, and a certificate installed on the users device. I'm going to assume you have this here.

           Uri uri = new Uri("https://mypkiserver/index.html");
            var handler = new SocketsHttpHandler();
            var tcs = new TaskCompletionSource<X509Certificate2>();
            Android.Security.KeyChain.ChoosePrivateKeyAlias(Platform.CurrentActivity!, response: new KeyChainAliasCallback(tcs),
                new String[] { Android.Security.Keystore.KeyProperties.KeyAlgorithmRsa, "DSA" }, // List of acceptable key types. null for any
                null,                        // issuer, null for any
                null,                        // host name of server requesting the cert, null if unavailable
                -1,                          // port of server requesting the cert, -1 if unavailable
                "");
            certificate = await tcs.Task;
            if (certificate != null)
                handler.SslOptions.ClientCertificates = new X509Certificate2Collection(certificate); 

            using var client = new HttpClient(handler);
            var response = await client.GetAsync(infoUri); //will usually throw if certificate can't be used
            response = response.EnsureSuccessStatusCode();

KeyChainAliasCallback.cs responsible for letting the user pick an installed certificate:

    public class KeyChainAliasCallback : Java.Lang.Object, Android.Security.IKeyChainAliasCallback
    {
        private readonly TaskCompletionSource<X509Certificate2?> _tcs;
        public KeyChainAliasCallback(TaskCompletionSource<X509Certificate2?> tcs)
        {
            _tcs = tcs;
        }

        void Android.Security.IKeyChainAliasCallback.Alias(string? alias)
        {
            if (alias != null)
            {
                var certificateChain = Android.Security.KeyChain.GetCertificateChain(Platform.CurrentActivity!, alias!);
                if (certificateChain != null && certificateChain.Length > 0)
                {
                    X509Certificate2 certificate = new(certificateChain[0].Handle);
                    _tcs.TrySetResult(certificate);
                }
            }
            _tcs.TrySetResult(null);
        }
    }

Expected behavior

Certificates used from the keychain where the private key isn't directly accessible will work.

Actual behavior

Fails to authenticate.

Regression?

No response

Known Workarounds

None

Configuration

No response

Other information

Related issue: #45741 (comment)

This is a blocker for larger enterprises since they can't use their secured services with the level of security they require. Using file-based certificates in-app with the full exportable private key just isn't acceptable to them.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[dotMorten]

@simonrozsival Any update on this? We have quite a few customers who are not able to use our products because they can't authenticate with their servers using device-installed certificates.

[simonrozsival]

@dotMorten I haven't had time to look into this issue yet, but I will triage it later this week.

[simonrozsival]

I looked into the current capabilities and if I understand it correctly, the way we implement client certificates in the SocketsHttpHandler/SslStream doesn't support this workflow with aliases. Maybe @elinor-fung knows more.

I also looked into how one could implement this with the AndroidMessageHandler by overriding the ConfigureKeyStore or ConfigureKeyManagerFactory and I wasn't able to get it working either. It shouldn't be hard to add support for this flow into AndroidMessageHandler though. Maybe we could do that while adding fixing dotnet/android#7274.

[dotMorten]

The AndroidMessageHandler has a number of limitations that makes it really inconvenient to use, like weird pre-authenticate issues, frequent crashes in the send request when it suddenly decides to close the socket, and based on an old Java implementation.

I would expect the certificate handling sits all the way down at the socket level which is also using the java apis, so I would expect to be able to use the installed certificates: https://github.com/dotnet/runtime/blob/b194416ea68e2d1c93a91fc7abf80eb2607b4831/src/native/libs/System.Security.Cryptography.Native.Android/pal_sslstream.c

[simonrozsival]

The AndroidMessageHandler has a number of limitations that makes it really inconvenient to use, like weird pre-authenticate issues, frequent crashes in the send request when it suddenly decides to close the socket, and based on an old Java implementation.

Yes, I understand why you would prefer SocketsHttpHandler over the native one.

I would expect the certificate handling sits all the way down at the socket level which is also using the java apis, so I would expect to be able to use the installed certificates: ...

You're right, there's no reason why we couldn't implement it in the Java/JNI code. The problem I see at the moment is that there isn't a good way to pass the alias all the way down to pal_sslstream.c through existing .NET APIs. I'm currently looking into ways which we could extend the implementation without introducing a new (platform specific) API.

[jonathanpeppers]

Looking at the code example above, I don't think passing a Java.Lang.Object.Handle into here would work:

runtime/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/X509Certificate2.cs

Line 118 in e377f16

public X509Certificate2(IntPtr handle)

Maybe it doesn't throw an exception? But this would be a handle to a Java object instance, and X509Certificate2 wouldn't know what to do with that.

Is there a way to use the X509Certificate2 byte[] or string fileName constructors instead?

[dotMorten]

@jonathanpeppers it does work. I get all the properties reporting the correct values so seems fine.
Problem with using byte[] is you'll not have the private key. That's the entire reason why we need the platform certificate which is able to handle the encryption since the certificates by design aren't exportable

[simonrozsival]

I was thinking if it might make sense to allow this constructor to be a "file path or alias" on Android. While technically it would be definitely possible, it's probably not a good idea:

runtime/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/X509Certificate2.cs

Line 129 in e377f16

public X509Certificate2(string fileName)

It might be better to explore if some of the X509Store APIs map to the android KeyChain more closely:

https://learn.microsoft.com/en-us/dotnet/api/system.security.cryptography.x509certificates.x509store.-ctor?view=net-8.0#system-security-cryptography-x509certificates-x509store-ctor(system-string-system-security-cryptography-x509certificates-storelocation)

[simonrozsival]

@filipnavara do you have any insight?

[filipnavara]

do you have any insight?

Unfortunately, I am not particularly familiar with the Android Keychain APIs. There were certainly cases where I had a use case on iOS / Android to convert a native object (SecIdentityRef/SecCertificateRef for iOS, java.security.cert.X509Certificate for Android) into X509Certificate2 and back. I certainly feel it would be warrantied to provide extension methods into the Android/iOS SDKs to do such conversion (without going through export and reimport). iOS is rather straightforward when it comes to the mapping but I vaguely remember that the backing implementation of X509Certificate2 actually mapped to more than one Java object (feel free to correct me) and it gets tricky.

The alias mapping sound like something that could be mapped to X509Store(string storeName) constructor. Not sure if that's the best way though. Maybe using the primary store and some store.Certificates.Find(...) overload would be a better idea.

I'll try to sleep on it and perhaps I will come up with a better answer.

[simonrozsival]

So to do this on Windows, you need to install the System.Windows.Extensions NuGet package and use the X509Certificate2UI class that will show a Windows dialog (similar to KeyChain.ChoosePrivateKeyAlias) and it will give you a certificate with a non-exportable private key back.

I wonder if we should add a similar public API to xamarin-android and corresponding internal APIs to the Android PAL to work with certificates via aliases in the runtime (I don't think we want to create and maintain System.Android.Extensions NuGet). I think we would need something like:

// xamarin/xamarin-android
public static class X509Certificate2Factory
{
    public static X509Certificate2 CreateFromAlias(Android.Content.Context context, string alias) { ... }
}

// dotnet/runtime
internal partial class AndroidCertificatePal
{
    internal static X509Certificate2 FromAlias(IntPtr context, string alias) { ... }
}

Unfortunately, unlike System.Windows.Extensions, xamari-android can't see the internals directly, but I don't think that would be a major problem.

[dotMorten]

I'm not quite getting the argument for providing these extensions - sure they are helpful to avoid writing the code in the issue details, but at the end of the day we can still create the X502Certificate2 from what KeyChain.ChoosePrivateKeyAlias returns.

[simonrozsival]

but at the end of the day we can still create the X502Certificate2 from what KeyChain.ChoosePrivateKeyAlias returns

We can create the certificate, but AFAIK we're not able to include the private key information (KeyChain.GetPrivateKey) in the current X509Certificate2 class. Or are we? I might be missing something.

[filipnavara]

We can create the certificate, but AFAIK we're not able to include the private key information (KeyChain.GetPrivateKey) in the current X509Certificate2 class. Or are we? I might be missing something.

You are correct. AndroidCertificatePal (the backing type for X509Certificate2) uses two handles:

runtime/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/AndroidCertificatePal.cs

Lines 18 to 19 in 1ded19e

	private SafeX509Handle _cert;
	private SafeKeyHandle? _privateKey;

Calling X509Certificate2.FromHandle initializes only the first one and not the private key. The handle-based constructor for RSAAndroid implementation is not public. You could probably get the handle through KeyChain.GetPrivateKey, then create RSAAndroid/ECDsaAndroid/ECDiffieHellmanAndroid from the handle through reflection / UnsafeAccessor, and finally bind it back through X509Certificate2.CopyWithPrivateKey(...). Then again, that's very convoluted way to work around the lack of native conversion API.

[dotMorten]

Aaaah I think that's where the confusion in this thread comes from. I do not think it's possible to ever get the private key. That's the entire point of using the keychain to store it, so that it can't be "stolen" and taking out of the device. It's the reason my customers are insisting on using installed certificates, over just a file-based one.

What I'm hoping for is that we're able to push this certificate all the way down to the socket level where Android is again responsible for communicating with the other end and use platform APIs to ensure the encryption is handled.

[zli-programer]

It isn't about getting private key. Private key is in the device.
It is about getting SslStream to use it trough native API so we could use certificate stored in the device as X509Certificate2 from X509Store for HttpClient.

[simonrozsival]

I've looked into the Apple platforms APIs and I noticed that the X509Certificate2(IntPtr) constructor allows passing a handle of an SecCertificate object or of an SecIdentity. It might not be a bad idea to do something similar on Android and allow passing in not only a handle of java.security.cert.X509Certificate, but also of java.security.KeyStore.PrivateKeyEntry.

This approach wouldn't require any changes to the public APIs and the internal implementation of AndroidCertificatePal and pal_sslstream.c shouldn't be too complicated. I'm just not sure where we keep documentation of these platform specific details (it certainly isn't in the X509Certificate2(IntPtr) docs) if we even have such place.

The usage would look like this:

var privateKey = Android.Security.KeyChain.GetPrivateKey(Platform.CurrentActivity!, alias);
var certificateChain = Android.Security.KeyChain.GetCertificateChain(Platform.CurrentActivity!, alias);
var privateKeyEntry = new Java.Security.KeyStore.PrivateKeyEntry(privateKey, certificateChain);

var certificate = new X509Certificate2(privateKeyEntry.Handle);
handler.SslOptions.ClientCertificates = new X509Certificate2Collection(certificate);

[dotMorten]

@simonrozsival That sounds great, but will you be able to tell the difference what kind of handle you're getting?
Ie the second line of this is calling the same overload:

Java.Security.Cert.X509Certificate javacert = GetCert();
var dotnetcert = new X509Certificate2(javacert.Handle);

as when using the entry handle:

Java.Security.KeyStore.PrivateKeyEntry entry = GetEntry();
var dotnetcert = new X509Certificate2(entry.Handle);

Since it's just a pointer, you can't really tell the difference which kind of handle you're getting. Is there some magic internal secret sauce to find this?

[simonrozsival]

@dotMorten yes, that should be straightforward IMO. We can check if it is a java object reference and check the type of the java object using JNI.

[dotMorten]

Ship it!