PKCS#7 / CMS Signature validation failure with unordered attributes #1365

dhensby · 2023-03-30T10:24:19Z

Firstly, apologies if this issue is known or has been raised and closed. I had a look through existing issues and found a couple of things that could be related, but don't explicitly get to the bottom of it. #286 #761

I have a service producing detached CMS payloads, these payloads validate successfully when using openssl, but when using Bouncy Castle, the signature fails to validate. NB: I am not the one using BC, so apologies that I can't provide full errors.

After much debugging, it was found that the issue is the service generating the CMS payloads is not strictly compliant with the specification which says that the authenticated attributes must be a DER encoded SET OF, the DER spec says that a SET OF must be ordered:

11 Restrictions on BER employed by both CER and DER
…
11.6 Set-of components
The encodings of the component values of a set-of value shall appear in ascending order, the encodings being compared as octet strings with the shorter components being padded at their trailing end with 0-octets.
NOTE – The padding octets are for comparison purposes only and do not appear in the encodings.

I wondered why this was an issue in BC but not in openssl and after a bit of digging, I think the issue comes down to the fact that BC is parsing and then reconstructing what it thinks the signed ASN.1 structure should be and not what it actually is. There is a decode/encode step that causes information (order) loss and therefore the signature is not calculated over the bytes that are supplied causing an invalid signature to be calculated.

I can see that there are two valid viewpoints on this:

It's a non-compliant structure, so failure is understandable; and
BC should be validating signatures over the actual bytes received, not over reconstructed data.

I don't think these are mutually exclusive and both are true; BC should not be calculating the signatures except from the actual supplied bytes. Ultimately, the signature over the bytes supplied is valid, the bytes supplied just don't adhere strictly to the spec.

Version: org.bouncycastle:bcpkix-jdk15on:1.70

Example - here is a certificate and signature that passes with openssl but should fail with bouncy castle:

certificate.pem:

pkcs7-bad.pem:

openssl command to verify:

$ openssl smime -verify -attime 1332945777 -in ./pkcs7-bad.pem -signer ./certificate.pem -inform PEM -CAfi
le ./certificate.pem

Example - pkcs7 that passes with both openssl and BC:

pkcs7-good.pem:

openssl command to verify:

$ openssl smime -verify -attime 1332945777 -in ./pkcs7-good.pem -signer ./certificate.pem -inform PEM -CAfi
le ./certificate.pem

Example failing code:

fun verifySignature(): Boolean{
    return try {
        val jsonBytes = Files.readAllBytes(Paths.get("path/to/json"));
        val sigBytes = Files.readAllBytes(Paths.get("path/to/p7s"));
        // Create certificate from byte array
        val cert = System.getenv("PINNED_CERT")
        val certBytes: ByteArray = Base64.getDecoder().decode(cert)

        val certificateFactory = CertificateFactory.getInstance("X.509")
        println(certBytes.size)
        val certificate = certificateFactory.generateCertificate(certBytes.inputStream()) as X509Certificate

        // Verify the signature of the content byte array using CMS
        val cms = CMSSignedData(sigBytes)
        val signers: SignerInformationStore = cms.signerInfos
        val signer: SignerInformation = signers.signers.first()

        // Create a new Signature object
        val signature = Signature.getInstance(certificate.sigAlgName)
        signature.initVerify(certificate.publicKey)
        signature.update(jsonBytes)

        val sig = signer.signature
        val isValidSignature = signature.verify(sig) // <-- this always fails

        isValidSignature
    } catch (e: GeneralSecurityException) {
        println("Exception: ${e.message}")
        false
    }
}

The text was updated successfully, but these errors were encountered:

dghgit · 2023-03-30T22:36:40Z

So it's 1, and 2 is fair enough. Have a look at

bc-java/pkix/src/test/java/org/bouncycastle/cms/test/NewSignedDataTest.java

Line 1320 in fd44df9

public void testSignerInformationExtension()

MyRightSignerInformation will show you what to do. Keep in mind if you are accepting messages like this people can reorder the data and add extras as it suits them - the purpose of the DER encoding is to ensure only one encoding of data can be used to verify the signature. It's better to get people to correct what they are doing if they are producing invalid messages.

dhensby · 2023-03-30T23:18:05Z

Keep in mind if you are accepting messages like this people can reorder the data and add extras as it suits them

Only the authenticated attributes are signed, so modification of the other parts of the message is always possible (except the content).

At the moment it would be possible for someone to sign the ordered attributes and then a third party could theoretically reorder them and BC would validate the signature (despite the signed attributes being tampered with), when I suspect openssl would fail as the actual bytes wouldn't verify.

However, this is academic and, I'd suggest, meaningless as the data would still have been signed, just someone had arbitrarily (but not meaningfully) interfered with the order of the authenticated attributes.

I have submitted a fix for this issue in the upstream library too: digitalbazaar/forge#1025

dhensby · 2023-03-30T23:42:35Z

In fact, I think this is an example of a PKCS#7 message that would fail validation in openssl but pass in BC because the signature is calculated across sorted attributes but the message contains unsorted attributes.

That means any library verifying signatures over the actual bytes in the message will fail, but any calculating it over expected bytes will validate it.

Bad PKCS#7 may pass in BC

dghgit · 2023-03-31T12:32:18Z

Frankly, I'm surprised OpenSSL would validate an invalid message. Likewise I am extremely surprised that it would not validate a valid message with disordered signed attributes.

If it's any help, RFC 5652, section 5.4 (and it's predecessors) is quite explicit about the need to DER encode signed attributes, the message with unsorted attributes is valid providing the signature is calculated over the DER encoding. As I rule we would only write things out in DER encoding as it means the message will validate even if the parser checking it is broken, but there are occasions where people do not save the attributes in DER ordering.

Glad to hear it will be fixed.

dhensby · 2023-03-31T13:24:51Z

Interesting, thanks. So then the BC behaviour appears to be correct (calculate over theoretical bytes not supplied bytes) and OpenSSL is wrong.

Thanks for the feedback on this and hopefully this will help anyone else who stumbles upon this problem.

dghgit self-assigned this Mar 30, 2023

dghgit closed this as completed Mar 31, 2023

aloopkin mentioned this issue Jul 24, 2024

Make message decoding more interoperable for signature verification jscep/jscep#309

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PKCS#7 / CMS Signature validation failure with unordered attributes #1365

PKCS#7 / CMS Signature validation failure with unordered attributes #1365

dhensby commented Mar 30, 2023 •

edited

Loading

dghgit commented Mar 30, 2023

dhensby commented Mar 30, 2023

dhensby commented Mar 30, 2023

dghgit commented Mar 31, 2023

dhensby commented Mar 31, 2023

PKCS#7 / CMS Signature validation failure with unordered attributes #1365

PKCS#7 / CMS Signature validation failure with unordered attributes #1365

Comments

dhensby commented Mar 30, 2023 • edited Loading

dghgit commented Mar 30, 2023

dhensby commented Mar 30, 2023

dhensby commented Mar 30, 2023

dghgit commented Mar 31, 2023

dhensby commented Mar 31, 2023

dhensby commented Mar 30, 2023 •

edited

Loading