OpenSSL error in ASN1_get_object: too long

[sdanbury]

I am getting the error when the jwt_decode_sig function calls the signature_verify function. Any ideas on where to look at first?

If I copy the jwt_split code and pass the sig through it, it works fine and I can get all of the information out of the JWT signature. However, when I use the jwt_decode_sig function directly, it fails with the error, even though the public key is valid, and I assume the sig is valid.

[sdanbury]

I am trying to do the following Python, in R:

import jwt
import requests
import base64
import json

# Step 1: Get the key id from JWT headers (the kid field)
encoded_jwt = headers.dict['x-amzn-oidc-data']
jwt_headers = encoded_jwt.split('.')[0]
decoded_jwt_headers = base64.b64decode(jwt_headers)
decoded_json = json.loads(decoded_jwt_headers)
kid = decoded_json['kid']

# Step 2: Get the public key from regional endpoint
url = 'https://public-keys.auth.elb.' + region + '.amazonaws.com/' + kid
req = requests.get(url)
pub_key = req.text

# Step 3: Get the payload
payload = jwt.decode(encoded_jwt, pub_key, algorithms=['ES256'])

This is so that I can verify the JWT sent back from the AWS ALB and in turn safely use the information stored within it.

[jeroen]

Do you have an example of the code that is giving this error?

[sdanbury]

library(openssl)
library(jose)

jwt <- "<jwt-here-as-string>"

pubkeystring <- "-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEkWo7tYMmQslyqwidviEsTQYfU6So
+Osx64z2wGRKzMpxxxxxxxxxxxxxxxxxxxxxxxxxx4g49Xz9A==
-----END PUBLIC KEY-----"

pubkey = read_pubkey(pubkeystring)

jwt_decode_sig(jwt, pubkey = pubkey)

The public key I got from passing the kid to the AWS URL as mentioned in the previous comments.

I would give you the JWT I was using exactly, but it contains sensitive information, apologies. I will try and recreate my setup in a throwaway AWS account in the meantime. Here is the basic structure of the header and payload of the JWT if it helps:

Header
{
  "typ": "JWT",
  "kid": "xxxxxxxxxxxxxxxxxxxxxx",
  "alg": "ES256",
  "iss": "https://mydomain.eu.auth0.com/",
  "client": "Xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "signer": "arn:aws:elasticloadbalancing:eu-west-1:123456765434:loadbalancer/app/my-alb/xxxxxxxxxxxxxx",
  "exp": 1544016026
}

Payload
{
  "sub": "hello|world|helloworldhelloworld",
  "https://example.com/roles": "role1,role2",
  "https://example.com/permissions": []
}

[jeroen]

Are you sure your jwt string contains a signature? Does this example work for you?

library(jose)

pubkeystring <- "-----BEGIN PUBLIC KEY-----
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDdlatRjRjogo3WojgGHFHYLugd
UWAY9iR3fy4arWNA1KoS8kVw33cJibXr8bvwUAUparCwlvdbH6dvEOfou0/gCFQs
HUfQrSDv+MuSUMAe8jzKE4qW+jK+xQU9a03GUnKHkkle+Q0pX/g6jXZ7r1/xAK5D
o2kQ+X5xK9cipRgEKwIDAQAB
-----END PUBLIC KEY-----"
sig <- "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.EkN-DOsnsuRjRO6BxXemmJDm3HbxrbRzXglbN2S4sOkopdU4IsDxTI8jO19W_A4K8ZPJijNLis4EZsHeY559a4DFOd50_OqgHGuERTqYZyuhtF39yxJPAjUESwxk2J5k_4zM3O-vtd1Ghyo4IbqKKSy6J9mTniYJPenn5-HIirE"
pk <- openssl::read_pubkey(pubkeystring)

# Should return: 1234567890
jwt_decode_sig(sig, pk)

[sdanbury]

That does work yes.

I copied the whole JWT string from the x-amzn-oidc-data header that is returned by the ALB after a successful login (more info found here).

I used this gist to write a shiny app behind the ALB which in turn gets me the x-amzn-oidc-data header. I then copied that into the R code that I sent to you, so I assume it is correct.

So you think it is a problem with the third part of the JWT (the signature)? If it helps, I put the public key, and the JWT into the https://jwt.io/ debugger and it said "valid signature".

[jeroen]

It's hard to say from here. You can debug(jwt_decode_sig) and then see where things are going wrong. Does this at least work for your jwt? jose:::jwt_split(jwt) ?

[jeroen]

Also what is your version of OpenSSL? openssl::openssl_config()

[sdanbury]

I did as you suggested and tried to debug(jwt_decode_sig).

The error occurs here: https://github.com/jeroen/jose/blob/c1385e453a47042144aa6cff4f213a74a0269851/R/jwt.R#L117

and then here:

https://github.com/jeroen/openssl/blob/ff46f36c8de85854c09ca2996d030ea5697aaf1a/R/signing.R#L52

OpenSSL version:

$ R

R version 3.4.4 (2018-03-15) -- "Someone to Lean On"
...
> openssl::openssl_config()
$version
[1] "OpenSSL 1.0.2g  1 Mar 2016"

$ec
[1] TRUE

[jeroen]

What is the length of out$sig?

[sdanbury]

...
> debug(jwt_decode_sig)        
> jwt_decode_sig(sig, pk)
debugging in: jwt_decode_sig(sig, pk)
debug: {
    out <- jwt_split(jwt)
    if (out$type != "RSA" && out$type != "ECDSA") 
        stop("Invalid algorithm: ", out$type)
    key <- read_pubkey(pubkey)
    if ((!inherits(key, "rsa") && !inherits(key, "ecdsa")) || 
        !inherits(key, "pubkey")) 
        stop("Key must be rsa/ecdsa public key")
    dgst <- sha2(out$data, size = out$keysize)
    if (!signature_verify(dgst, out$sig, hash = NULL, pubkey = key)) 
        stop(out$type, " signature verification failed!", call. = FALSE)
    structure(out$payload, class = c("jwt_claim", "list"))
}
Browse[2]> 
debug: out <- jwt_split(jwt)
Browse[2]> 
debug: if (out$type != "RSA" && out$type != "ECDSA") stop("Invalid algorithm: ", 
    out$type)
Browse[2]> out$sig
 [1] 82 55 33 5f d9 21 26 2e b8 f8 60 52 03 52 63 94 5b 6d 4b 57 51 55 2f 42 8b
[26] 57 54 b6 12 7e b9 85 af 36 0c 35 b0 65 15 64 68 a0 48 7e f6 f9 44 b0 0e bb
[51] e0 57 ee 9d 36 68 13 2b 1a 3e 9e cc 43 f4
Browse[2]> length(out$sig)
[1] 64
Browse[2]> nchar(out$sig)
 [1] 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
[39] 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
Browse[2]>

[jeroen]

Hmm I am confused 🤔The spec says that the signature has to be 64 bytes but openssl seems to expect signatures of 70 or 71 bytes:

library(openssl)
test <- charToRaw('hello')
signature_create(test, key = ec_keygen('P-256'))

Perhaps there is an issue with the DER encoding of the signature in openssl. But it seems to work just fine for other applications.

Possibly related: auth0/java-jwt#187, mpdavis/python-jose#47

[sdanbury]

Very interesting. So are you saying that there could be an issue in the library that is used to create the JWT signature that I am getting back from the AWS ALB because this library (which follows the spec) is unable to verify it?

[jeroen]

No, there is a bug here. Apparently the jwt spec uses an unconventional way to encode the signature to make sure it is always 64 bytes. We need to convert it from what openssl gives us, exactly the same problem as the java and python people have run into above.

[jeroen]

I have pushed a fix based on what the python/java implementations linked above are doing. Could you test this please?

remotes::install_github("jeroen/jose")

I should add better test cases because roundtripping apparently isn't enough.

[sdanbury]

Thanks for the really quick turnaround on this!

I tried it, by doing the following:

$ mkdir libs
$ R_LIBS=libs Rscript -e 'install.packages("remotes")'
...
$ R_LIBS=libs Rscript -e 'remotes::install_github("jeroen/jose")'
...
$ R_LIBS=libs Rscript jwt.R
Loading required package: openssl

 *** caught segfault ***
address 0x7, cause 'memory not mapped'

Traceback:
 1: .Call(R_write_ecdsa, r, s)
 2: openssl::ecdsa_write(r, s)
 3: jwt_decode_sig(sig, pk)
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault (core dumped)

Where jwt.R is:

library(jose)

pubkeystring <- "-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEkWo7tYMmQslyqwidviEsTQYfU6So
+Osx64z2wGRKzMpliIBpMAGqS1Eof5F+6ZJfWudUn7eeMioNPJ4g49Xz9A==
-----END PUBLIC KEY-----"

sig <- "blahblahblah"

pk = openssl::read_pubkey(pubkeystring)

jwt_decode_sig(sig, pk)

So yeah, it segfaults. Would you like me to get you an actual JWT to work with? Or have you recreated it with Auth0 and AWS ALB on your side?

[jeroen]

I think something went wrong in updating the openssl package. Can you try again please? Make sure neither jose or openssl is loaded in your R session when you install the update:

remotes::install_github("jeroen/openssl")
remotes::install_github("jeroen/jose")

[sdanbury]

I am still getting the error with a clean environment. Starting from scratch, into a fresh libs directory, I install remotes, then run the two lines you just posted, then run the code, and it gives:

> 
> pk = openssl::read_pubkey(pubkeystring)
> jwt_decode_sig(sig, pk)
Error in jwt_decode_sig(sig, pk) : 
  could not find function "jwt_decode_sig"
> library(jose)
Loading required package: openssl
> jwt_decode_sig(sig, pk)

 *** caught segfault ***
address 0x7, cause 'memory not mapped'

Traceback:
 1: .Call(R_write_ecdsa, r, s)
 2: openssl::ecdsa_write(r, s)
 3: jwt_decode_sig(sig, pk)

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: 1
R is aborting now ...
Segmentation fault (core dumped)

[jeroen]

Hmm that's very odd. Which OS are you on?

[sdanbury]

Linux Ubuntu 16.04

$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.5 LTS"

[jeroen]

Do you only get this crash for your particular signature, or also when running example(ecdsa_write)

[sdanbury]

I am guessing a lot of these problems are to do with the fact that AWS ALB returns a base64 encoded token, not a base64 URL encoded token. I have been experiencing issues with javascript libraries as well, this issue pretty much sums it up: auth0/node-jsonwebtoken#514.

Possible that the same is happening here? Maybe the == padding on each of the three sections of the JWT is causing issues.

Anyway, I will try your code this afternoon with example(ecdsa_write) and see what happens. Thanks again.

[jeroen]

Hmm invalid input should never cause a crash.

I do have an amazon account, can you explain me where I would create a test jwt token?

[sdanbury]

You would need to create the following:

• AWS Application Load Balanacer (ALB). It is the ALB that does the OIDC dance with an identity provider like Auth0 (I am using Auth0) and then passes the JWT token down to the services that run behind the ALB.
• An EC2 instance that will sit behind the ALB instance. On this instance you need some sort of web app that will print out the headers, so you can see the JWT header that is sent by the ALB.
• You need to create a target group and register the EC2 instance in that target group, and then create an ALB rule to route all traffic / to your target group (your EC2 instance).

This post does a good job of explaining how it all fits together.

Basically people can use the ALB built-in OIDC auth to authenticate apps so the apps behind the load balancer don't have to (or can't authenticate themselves).

[jeroen]

OK that is more complicated than I thought. Is there no simpler application of jwt's in amazon?

If you can generate a dummy jwt that would be super helpful. You can revoke it immediately and email it to me because we only need to test the parser.

[jeroen]

I finally found the problem. Should be fixed in the openssl package 1.2.1.

[sdanbury]

Great! Thanks so much for continuing the effort. What was the issue in the end?

Do you have a patreon or similar that I can contribute to?

[jeroen]

The issue was some uninitiated null pointer in openssl, that could make it crash. I uploaded jose 1.0 to cran. Does everything work now for you as expected?

I don't have patreon, your feedback is a contribution in itself :) You can always donate to WWF if you want to make me happy.

[sdanbury]

@jeroen sorry for the late response! From initial testing, it looks like it is now fixed. When I find some time to do some more thorough testing, I will raise a new issue if anything pops up.

Thanks again for all of the effort. I made a donation to WWF on your behalf.