Leverage External Key, such as AWS KMS

[andrewjroth]

Hello,

I am trying to figure out how I can use the cryptography module to create and sign x509 certificates using a key stored in an external service, such as AWS KMS. AWS KMS has functions to send a message to the key service to be signed (ref: KMS.Client.sign from boto3). Cryptography has all the functionality to construct the x509 certificate, but it seems like it has to have access to the private key to add the CA signature. Is there any way that I can use an external process get the signature and insert it into the certificate?

I thought that I might be able to build a class based on ec.EllipticCurvePrivateKey to represent the key, and override the sign method to send the message to the external system to be signed. Unfortunately, it seems that the openssl backend seems to take over and require the private key to hold data. The class I wrote is here: awskms.py

I invoke x509.CertificateBuilder.sign like this:

        builder = x509.CertificateBuilder().issuer_name(
            x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, u'Test CA')])
        ).subject_name(
            csr.subject
        ).public_key(
            csr.public_key()
        ).serial_number(
            1
        ).not_valid_before(
            datetime.today()
        ).not_valid_after(
            datetime.today() + timedelta(days=30)
        )
        cert = builder.sign(AWSKMSEllipticCurvePrivateKey(KEYID), hashes.SHA384())

This is the error I get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "handler.py", line 76, in notify
    cert = builder.sign(AWSKMSEllipticCurvePrivateKey(KEYID), hashes.SHA384())
  File "/home/ec2-user/.local/lib/python3.8/site-packages/cryptography/x509/base.py", line 723, in sign
    return backend.create_x509_certificate(self, private_key, algorithm)
  File "/home/ec2-user/.local/lib/python3.8/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1050, in create_x509_certificate
    res = self._lib.X509_sign(x509_cert, private_key._evp_pkey, evp_md)
TypeError: initializer for ctype 'EVP_PKEY *' must be a cdata pointer, not NoneType

Is there an easier way for me to do this? I am hoping to leverage the cryptography library to construct the x509 structure, but want to use an external service for the real cryptography (signing). While I am asking about AWS KMS, this could be use for any external key management system that has functionality to sign messages.

Thank you!

[alex]

At the moment, no, there's no way to sign an X.509 cert with an externally held key. I think we'd like to support this, but we don't currently.

[saper]

Maybe something like tbs_certrequest_bytes that is already available for CSR would be feasible?

[reaperhulk]

@saper You actually need to create a tbs_certificate_bytes. Since cryptography currently uses OpenSSL to create this ASN.1 DER serialization the workaround is that you build the certificate using our APIs, sign it with a throwaway key, obtain the tbs_certificate_bytes from the resulting object, hash that, sign that hash, then build the final ASN.1 structure yourself:

   Certificate  ::=  SEQUENCE  {
        tbsCertificate       TBSCertificate,
        signatureAlgorithm   AlgorithmIdentifier,
        signatureValue       BIT STRING  }

This isn't hugely difficult, but it's a gross hack that requires quite a bit of domain knowledge. It would be nice to support something better.

[saper]

@reaperhulk I fully agree that this is difficult and hard to do correctly. This is something that can be done now until a better solution is available. I wouldn't recommend it either.

[space88man]

If you have an OpenSSL engine wrapper around KMS — you could take a look at my POC: https://github.com/space88man/cryptography_engine

cryptography has reinstated some of the low-level ENGINE APIs to make this work.

[andrewjroth]

@saper @reaperhulk @space88man
Please checkout the PR I submitted for the certbuilder package: wbond/certbuilder#10

I think that modification hits the need for now.

[NicolaF]

@andrewjroth I have the exact same use case. Solved it using @space88man 's cryptography engine + https://github.com/nakedible/openssl-engine-kms

Minor annoyances

• One needs to build cryptography manually (--no-binary) so that it links dynamically against libcrypto
• Link explicitly against libcrypto nakedible/openssl-engine-kms#3 : make the engine explicitly link against libcrypto

It works like a charm on AWS lambda

[alex]

Good news, this is going to be very easy to do with our next release: We'll no longer be using OpenSSL's x509 layer, so anything that implements our RSA/EC/etc PrivateKey interfaces will work for signing certificates. So all you'll need to do is write an implementation of RSAPrivateKey using KMS and you'll be golden!

[saper]

@alex this is great, let's try if NSS is up to the task and could be integrated as well!

[reaperhulk]

You could definitely write a library that implements our asymmetric interfaces using NSS. I look forward to seeing what everyone produces 😄

[reaperhulk]

I have implemented an alternate asymmetric provider to prove it's possible to myself. 😄 In this case I implemented a CloudRSAPrivateKey (sorry PKCS11 fans, I didn't feel like doing that work) that conforms to the cryptography.hazmat.primitives.asymmetric.rsa.RSAPrivateKey interface. If you only need signing support this really just means implement sign and declare all the other functions as raising NotImplementedError and you're done. You can then "load" a key and pass that object to the sign function on the x509 builder functions and it will work exactly as expected. This will work for all private key types our x509 layer is aware of : ed25519, ed448, EC, RSA, and even DSA if you're some kind of monster.

Once 36.0 is released we'll likely close this out and let the community build these solutions, but if you implement one please let us know and we will consider adding it to our downstream smoke tests (depending on quality of testing, popularity, etc)!

Edit: The original awskms.py in this issue is exactly the sort of approach that will work as of 36.0

[alex]

For the purposes of release tracking, this task is complete now and will ship in cryptography 36.0, so I'm going to close.