Add OIDC Twitter provider configuration

[sberyozkin]

Description

Now that PKCE is supported it should be possible to add a twitter configuration, Steph, please verify and add it once it works for you, thanks

https://developer.twitter.com/en/docs/authentication/oauth-2-0/authorization-code

Implementation ideas

No response

[quarkus-bot]

/cc @pedroigor

[FroMage]

OK, I tried it. Doesn't work. Twitter refuses to tell me why not, which isn't helpful.

My config:

# Twitter
quarkus.oidc.twitter.authentication.pkce-required=true
quarkus.oidc.twitter.authentication.pkce-secret=eUk1p7UB3nFiXZGUXi0uph1Y9p34YhBU
quarkus.oidc.twitter.auth-server-url=https://api.twitter.com/2/oauth2
quarkus.oidc.twitter.client-id=SECRET
quarkus.oidc.twitter.credentials.secret=SECRET
quarkus.oidc.twitter.application-type=web-app
quarkus.oidc.twitter.discovery-enabled=false
quarkus.oidc.twitter.authorization-path=https://twitter.com/i/oauth2/authorize
quarkus.oidc.twitter.token-path=token
#quarkus.oidc.twitter.jwks-path=https://api.twitter.com/2/oauth2/token
quarkus.oidc.twitter.authentication.scopes=offline.access
quarkus.oidc.twitter.authentication.id-token-required=false
quarkus.oidc.twitter.authentication.user-info-required=true
quarkus.oidc.twitter.user-info-path=https://api.twitter.com/2/users/me
# GET /2/users/me
#        ret.setUserInfoPath("https://api.github.com/user");
#        ret.getAuthentication().setUserInfoRequired(true);
#        ret.getAuthentication().setIdTokenRequired(false);

The URI I land on:

https://twitter.com/i/oauth2/authorize?response_type=code&client_id=SECRET&scope=openid+offline.access+offline.access&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2F_renarde%2Fsecurity%2Flogin-twitter&state=8994f6d5-8431-4d2d-bae0-ee977325f62e&code_challenge=Bl_F3XIUrvZbV8GknYypCqjpof-E9TIowdtaDN3biSU&code_challenge_method=S256

Twitter only tells me:

Something went wrong
You weren’t able to give access to the App. Go back and try logging in again.

Which is not helpful

[sberyozkin]

@FroMage Can you please try error-path and see what is reported:

https://quarkus.io/version/main/guides/security-openid-connect-web-authentication#customize-authentication-error-response,

make it a public resource

[sberyozkin]

If that returns an internal error then we need to find a way to report the issue to Twitter Oauth2 developers. FYI, the integration PKCE tests in Quarkus run directly against Keycloak which also supports PKCE, so we do not assume it works by running wiremock tests

[sberyozkin]

@FroMage Also, the error message might contain some hints that the Renarde app might not be configured correctly in Twitter, please check the redirect_uris match, etc

[FroMage]

I got further this time. Turns I forgot my redirect URI from the config. Now it doesn't work because the scope is set to openid+offline.access+offline.access when my config says:

quarkus.oidc.twitter.authentication.scopes=offline.access

The duplicate offline.access doesn't matter (though it's weird) but the openid scope is rejected by twitter.

[FroMage]

I can fix this manually by repeating the redirect with the proper scopes, but then I get the infamous 401 when I'm back on Quarkus after Twitter authorized me, and I still have to use a debugger because Quarkus doesn't log anything useful. This is driving me crazy BTW, we absolutely must fix this lack of logging.

[FroMage]

I can't even replay the failure in a debugger because OIDC clears the auth cookie:

set-cookie: q_auth_twitter=; Max-Age=0; Expires=Wed, 30 Mar 2022 14:39:17 GMT; Path=/

Making this just this much harder to debug, because I don't even know where to step to figure out why the hell OIDC rejects this callback.

[FroMage]

Found the place to step in, getting this swallowed exception io.quarkus.oidc.OIDCException: {"error":"invalid_request","error_description":"Value passed for the authorization code was invalid."}

[FroMage]

OK, so this second one seems to stem from OidcCommonUtils.isClientSecretBasicAuthRequired which returns true so we're in the second case in https://developer.twitter.com/en/docs/authentication/oauth-2-0/user-access-token Step 3:

curl --location --request POST 'https://api.twitter.com/2/oauth2/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--header 'Authorization: Basic V1ROclFTMTRiVWhwTWw4M2FVNWFkVGQyTldNNk1UcGphUTotUm9LeDN4NThKQThTbTlKSXQyZm1BanEzcTVHWC1icVozdmpKeFNlR3NkbUd0WEViUA=='\
--data-urlencode 'code=VGNibzFWSWREZm01bjN1N3dicWlNUG1oa2xRRVNNdmVHelJGY2hPWGxNd2dxOjE2MjIxNjA4MjU4MjU6MToxOmFjOjE' \
--data-urlencode 'grant_type=authorization_code' \
--data-urlencode 'redirect_uri=https://www.example.com' \
--data-urlencode 'code_verifier=challenge'

Here's my travel so far:

Got to Twitter via:

https://twitter.com/i/oauth2/authorize?
 response_type=code
 &client_id=CLIENTID
 &scope=offline.access+offline.access
 &redirect_uri=http%3A%2F%2Flocalhost%3A8080%2F_renarde%2Fsecurity%2Foidc-success
 &state=a58a8734-090c-4eb3-9904-0c76684d2a5c
 &code_challenge=Qn4o2CWmZkhjFO8YimcLkrn38K4hVTV06J2a0b8BbNM
 &code_challenge_method=S256

Got back via:

http://localhost:8080/_renarde/security/oidc-success?state=a58a8734-090c-4eb3-9904-0c76684d2a5c&code=CODE

Then, OIDC made a POST to https://api.twitter.com/2/oauth2/token:

Authorization: Basic SOMETHING

grant_type=authorization_code
code=CODE
redirect_uri=http://localhost:8080/_renarde/security/oidc-success
code_verifier=itaYF7KIVtMgGnrDaPxt2a54liJLW68S4-A53ldoEi4

Got back a 400:

400 {"error":"invalid_request","error_description":"Value passed for the authorization code was invalid."}

Now, I'm not sure what they mean by authorization code, if it's the OIDC code token which seems the same as they gave me. Or is it the authorization header? Or is it the code_verifier parameter? The authorization header looks correct to me, it comes from the right credentials, at least. How can I check the code_verifier part?

[FroMage]

I found how to test it, apparently the code verifier must be hashed with sha-256 to get the code challenge.

But if I use https://gchq.github.io/CyberChef/#recipe=From_Base64('A-Za-z0-9-_',true)SHA2('256',64,160)From_Hex('Auto')To_Base64('A-Za-z0-9-_')&input=aXRhWUY3S0lWdE1nR25yRGFQeHQyYTU0bGlKTFc2OFM0LUE1M2xkb0VpNA I don't see sha256(itaYF7KIVtMgGnrDaPxt2a54liJLW68S4-A53ldoEi4) producing Qn4o2CWmZkhjFO8YimcLkrn38K4hVTV06J2a0b8BbNM but TrBrUe_XUyFKbaI-uZIlZTSXbhE9CIJfXuQQTbOHL7s.

I'm a bit at a loss what goes wrong then.

[sberyozkin]

Thanks @FroMage, I'll have a look asap (the issue about improving the debugging experience is open so will be addressed a bit later too :-) )

[sberyozkin]

@FroMage

Encoder encoder = Base64.getUrlEncoder().withoutPadding();
        byte[] codeChallengeBytes = OidcUtils.getSha256Digest("itaYF7KIVtMgGnrDaPxt2a54liJLW68S4-A53ldoEi4".getBytes(StandardCharsets.ISO_8859_1));
        String codeChallenge = encoder.encodeToString(codeChallengeBytes);
        System.out.println(codeChallenge);

gives Qn4o2CWmZkhjFO8YimcLkrn38K4hVTV06J2a0b8BbNM.

This is the code used in CodeAuthenticationMechanism:

Encoder encoder = Base64.getUrlEncoder().withoutPadding();

            // code verifier
            byte[] codeVerifierBytes = new byte[32];
            secureRandom.nextBytes(codeVerifierBytes);
            String codeVerifier = encoder.encodeToString(codeVerifierBytes);
            ....

            // code challenge
            try {
                byte[] codeChallengeBytes = OidcUtils.getSha256Digest(codeVerifier.getBytes(StandardCharsets.ISO_8859_1));
                String codeChallenge = encoder.encodeToString(codeChallengeBytes);
                ...
            } catch (Exception ex) {
                throw new AuthenticationFailedException(ex);
            }

[sberyozkin]

@FroMage

Now, I'm not sure what they mean by authorization code, if it's the OIDC code token which seems the same as they gave me.

They should be referring to code here, but the error can be caused because they can not match the code verifier with the code challenge, or possibly, if you were debugging, breakpoint after a redirect then some time is spent on debugging, then the POST is done, then it is possible a code has expired.

[sberyozkin]

@FroMage See https://twittercommunity.com/t/value-passed-for-the-authorization-code-was-invalid/164422, looks like the code has 30 secs limit, can you try again please without any breakpoints ? Can you open please a twitter community forum thread if it does not work ? (perhaps they can confirm that it is not SHA256 but plain SHA digest is expected, or may be they use URL encoder with padding enabled)

[FroMage]

I know it doesn't work without breakpoints, because I get a 401 with no info. Now, it's possible that with breakpoints it fails for a different reason, such as the timeout. You didn't explain why the CyberChef gives a different result than the OIDC code we have. Hopefully it's me who misconfigured it, but I find it suspect that I can't get the same result there as we do.

[sberyozkin]

@FroMage Hi, looks like in your recipe you have the input Base64 URL decoded first, while Quarkus takes the bytes directly from the Base64 URL encoded representation.

Can you please update that code where you see the error message being swallowed to log it and see what is actually reported ?

[FroMage]

Well I can't know which error message is being swallowed if I don't debug it.

[FroMage]

Hi, looks like in your recipe you have the input Base64 URL decoded first, while Quarkus takes the bytes directly from the Base64 URL encoded representation.

OK, you're right, that was my mistake, and indeed it looks correct.

[sberyozkin]

@FroMage I see in the code that there are only 2 places where no debug messages are done,

here and here, the second one happens already after the code flow has completed, during the verification one, but it does look like the first one explains what is happening,

Twitter is like GitHub in that it is not an OpenId Connect provider, so we just need to add one more property, as in the GitHub case, that id-token is not required ?

[sberyozkin]

@FroMage Sorry, I see you are setting it in your config...

[FroMage]

So, if I remove the openid scope that required me to manually intervene and edit the redirect, and enable debug I get this:

2022-04-04 15:29:20,477 DEBUG [io.qua.oid.run.CodeAuthenticationMechanism] (vert.x-eventloop-thread-8) Authentication request redirect_uri parameter: http://localhost:8080/_renarde/security/oidc-success
2022-04-04 15:29:20,480 DEBUG [io.qua.oid.run.CodeAuthenticationMechanism] (vert.x-eventloop-thread-8) q_auth_twitter cookie 'max-age' parameter is set to 1800
2022-04-04 15:29:25,395 DEBUG [io.qua.oid.run.CodeAuthenticationMechanism] (vert.x-eventloop-thread-8) Token request redirect_uri parameter: http://localhost:8080/_renarde/security/oidc-success
2022-04-04 15:29:26,281 DEBUG [io.qua.oid.run.OidcProviderClient] (vert.x-eventloop-thread-8) Request has failed: status: 403, error message: {
  "title": "Forbidden",
  "type": "about:blank",
  "status": 403,
  "detail": "Forbidden"
}

See how not helpful this is?

[FroMage]

This is already a different error that I was getting, it seems

[sberyozkin]

@FroMage Looks like some progress, so this 403 is coming from Twitter if it is reported by OidcProviderClient

[sberyozkin]

I agree the debugging experience is poor, no doubt. Will be addressed. Can you check, where is it coming from, perhaps when trying to get UserInfo ? if yes, then it is nearly 100% a scope setup issue

[sberyozkin]

Set the only breakpoint in OidcProviderClient, getUserInfo

[FroMage]

Can it be while getting user info? I thought this was from the token exchange

[sberyozkin]

@FroMage Yes, you are right, it does look like the access token Twitter gives back does not have enough permissions in it - which is a scope related problem

[FroMage]

It is the damn UserInfo, you're right, now I've added debug logging I can see it.

[sberyozkin]

@FroMage See https://twittercommunity.com/t/403-forbidden-attempting-to-use-user-access-token/167619/17

[FroMage]

Yeah, docs say I need more scopes:

quarkus.oidc.twitter.authentication.scopes=offline.access,tweet.read,users.read

And finally I'm getting further. Now it looks like the exception is in my camp. Thanks. I'll report back when I get it to work with the changes required.

[sberyozkin]

@FroMage Great stuff, so re the oidc scope - it must be provided for most of OIDC providers, so if adding it causes a problem then we can avoid automatically adding it if it is configured that the id token is not required since it would imply it is not an OIDC provider... and if someone ever needs it even in this case then they'd just add it in the configuration

[FroMage]

It looks like twitter API V2 doesn't give me access to the user's email. I need the V1.1 API for that, which requires elevated access. Why can't things be simple…

[FroMage]

But the V1.1 API requires OAuth 1.0a, do we support that?

[sberyozkin]

@FroMage Sadly no, OAuth 1.0a is not supported. But V2 does give something else, right ? (Access to the email is on their roadmap according to https://stackoverflow.com/questions/70704711/get-email-from-twitter-oauth2).

Lets add a twitter provider configuration to quarkus-oidc in any case, in some integrations a user name would be sufficient

[sberyozkin]

@FroMage Once they add a scope for the email then we will update this configuration

[FroMage]

See #24774