Unclear error message when OAuth credentials are invalid

[bonniecpk]

Scenario:

• Installing googleads library via NuGet as a new console project
• Web flow OAuth client_id and client_secret are used
• Refresh token is invalid
• all other necessary config details are given to the library

The returned error is:

Exception thrown: 'Google.Ads.GoogleAds.Lib.GoogleAdsException' in Google.Api.Gax.Grpc.dll
Failure:
Message: Status(StatusCode=Unavailable, Detail="Getting metadata from plugin failed with error: Exception occurred in metadata credentials plugin.")
Failure: 
Request ID: 

[jtattermusch]

This likely means that the auth library (whichever you are using) has thrown an exception when trying to generate the secret tokens to attach to the gRPC call.
https://github.com/grpc/grpc/blob/86f93801e5d452cff40d6958d21a8eb211da0654/src/csharp/Grpc.Core/Internal/NativeMetadataCredentialsPlugin.cs#L95

Can you turn on extra logging to see what the problem in the auth library was?

[jtattermusch]

What's puzzling is that the inner exception detail should be included in the error message. This is something we added
more than 2 years ago: https://github.com/grpc/grpc/pull/16543/files#diff-f4b2e97473d81e64e28a360c95c40da1b923d88aa44585809e497430f7d14e24R90
(ok and now I realized that I'm responding to a 2-years old issue).

Ok, so I think the resolution is that gRPC had made a fix to make the error message clearer: grpc/grpc#16543
(available starting from gRPC 1.16.0)

I think this issue can be closed now.

[boby-404]

Grpc Error

[AnashOommen]

@jtattermusch this is not fixed as of GRPC 2.34. Note that an RpcException is thrown, but the inner exception is not populated for me to capture this and rethrow.

The inner exception should be set to Google.Apis.Auth.OAuth2.Responses.TokenResponseException in this case, after ignoring the intermediate Grpc.Core.Internal.CoreErrorDetailException for this exception to be consumed effectively.

Status(StatusCode="Unavailable", Detail="Getting metadata from plugin failed with error: Exception occurred in metadata credentials plugin. Google.Apis.Auth.OAuth2.Responses.TokenResponseException: Error:"invalid_grant", Description:"Bad Request", Uri:""
   at Google.Apis.Auth.OAuth2.Responses.TokenResponse.FromHttpResponseAsync(HttpResponseMessage response, IClock clock, ILogger logger)
   at Google.Apis.Auth.OAuth2.Requests.TokenRequestExtenstions.ExecuteAsync(TokenRequest request, HttpClient httpClient, String tokenServerUrl, CancellationToken taskCancellationToken, IClock clock, ILogger logger)
   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.FetchTokenAsync(String userId, TokenRequest request, CancellationToken taskCancellationToken)
   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.FetchTokenAsync(String userId, TokenRequest request, CancellationToken taskCancellationToken)
   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.RefreshTokenAsync(String userId, String refreshToken, CancellationToken taskCancellationToken)
   at Google.Apis.Auth.OAuth2.UserCredential.RefreshTokenAsync(CancellationToken taskCancellationToken)
   at Google.Apis.Auth.OAuth2.TokenRefreshManager.RefreshTokenAsync()
   at Google.Apis.Auth.OAuth2.TokenRefreshManager.GetAccessTokenForRequestAsync(CancellationToken cancellationToken)
   at Google.Apis.Auth.OAuth2.UserCredential.GetAccessTokenWithHeadersForRequestAsync(String authUri, CancellationToken cancellationToken)
   at Grpc.Auth.GoogleAuthInterceptors.<>c__DisplayClass3_0.<<FromCredential>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Grpc.Core.Internal.NativeMetadataCredentialsPlugin.GetMetadataAsync(AuthInterceptorContext context, IntPtr callbackPtr, IntPtr userDataPtr)", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1631715013.223000000","description":"Getting metadata from plugin failed with error: Exception occurred in metadata credentials plugin. Google.Apis.Auth.OAuth2.Responses.TokenResponseException: Error:"invalid_grant", Description:"Bad Request", Uri:""\r\n   at Google.Apis.Auth.OAuth2.Responses.TokenResponse.FromHttpResponseAsync(HttpResponseMessage response, IClock clock, ILogger logger)\r\n   at Google.Apis.Auth.OAuth2.Requests.TokenRequestExtenstions.ExecuteAsync(TokenRequest request, HttpClient httpClient, String tokenServerUrl, CancellationToken taskCancellationToken, IClock clock, ILogger logger)\r\n   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.FetchTokenAsync(String userId, TokenRequest request, CancellationToken taskCancellationToken)\r\n   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.FetchTokenAsync(String userId, TokenRequest request, CancellationToken taskCancellationToken)\r\n   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.RefreshTokenAsync(String userId, String refreshToken, CancellationToken taskCancellationToken)\r\n   at Google.Apis.Auth.OAuth2.UserCredential.RefreshTokenAsync(CancellationToken taskCancellationToken)\r\n   at Google.Apis.Auth.OAuth2.TokenRefreshManager.RefreshTokenAsync()\r\n   at Google.Apis.Auth.OAuth2.TokenRefreshManager.GetAccessTokenForRequestAsync(CancellationToken cancellationToken)\r\n   at Google.Apis.Auth.OAuth2.UserCredential.GetAccessTokenWithHeadersForRequestAsync(String authUri, CancellationToken cancellationToken)\r\n   at Grpc.Auth.GoogleAuthInterceptors.<>c__DisplayClass3_0.<<FromCredential>b__0>d.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at Grpc.Core.Internal.NativeMetadataCredentialsPlugin.GetMetadataAsync(AuthInterceptorContext context, IntPtr callbackPtr, IntPtr userDataPtr)","file":"..\..\..\src\core\lib\security\credentials\plugin\plugin_credentials.cc","file_line":93,"grpc_status":14}")

[jtattermusch]

@jtattermusch this is not fixed as of GRPC 2.34. Note that an RpcException is thrown, but the inner exception is not populated for me to capture this and rethrow.

The inner exception should be set to Google.Apis.Auth.OAuth2.Responses.TokenResponseException in this case, after ignoring the intermediate Grpc.Core.Internal.CoreErrorDetailException for this exception to be consumed effectively.

Status(StatusCode="Unavailable", Detail="Getting metadata from plugin failed with error: Exception occurred in metadata credentials plugin. Google.Apis.Auth.OAuth2.Responses.TokenResponseException: Error:"invalid_grant", Description:"Bad Request", Uri:""
   at Google.Apis.Auth.OAuth2.Responses.TokenResponse.FromHttpResponseAsync(HttpResponseMessage response, IClock clock, ILogger logger)
   at Google.Apis.Auth.OAuth2.Requests.TokenRequestExtenstions.ExecuteAsync(TokenRequest request, HttpClient httpClient, String tokenServerUrl, CancellationToken taskCancellationToken, IClock clock, ILogger logger)
   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.FetchTokenAsync(String userId, TokenRequest request, CancellationToken taskCancellationToken)
   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.FetchTokenAsync(String userId, TokenRequest request, CancellationToken taskCancellationToken)
   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.RefreshTokenAsync(String userId, String refreshToken, CancellationToken taskCancellationToken)
   at Google.Apis.Auth.OAuth2.UserCredential.RefreshTokenAsync(CancellationToken taskCancellationToken)
   at Google.Apis.Auth.OAuth2.TokenRefreshManager.RefreshTokenAsync()
   at Google.Apis.Auth.OAuth2.TokenRefreshManager.GetAccessTokenForRequestAsync(CancellationToken cancellationToken)
   at Google.Apis.Auth.OAuth2.UserCredential.GetAccessTokenWithHeadersForRequestAsync(String authUri, CancellationToken cancellationToken)
   at Grpc.Auth.GoogleAuthInterceptors.<>c__DisplayClass3_0.<<FromCredential>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Grpc.Core.Internal.NativeMetadataCredentialsPlugin.GetMetadataAsync(AuthInterceptorContext context, IntPtr callbackPtr, IntPtr userDataPtr)", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1631715013.223000000","description":"Getting metadata from plugin failed with error: Exception occurred in metadata credentials plugin. Google.Apis.Auth.OAuth2.Responses.TokenResponseException: Error:"invalid_grant", Description:"Bad Request", Uri:""\r\n   at Google.Apis.Auth.OAuth2.Responses.TokenResponse.FromHttpResponseAsync(HttpResponseMessage response, IClock clock, ILogger logger)\r\n   at Google.Apis.Auth.OAuth2.Requests.TokenRequestExtenstions.ExecuteAsync(TokenRequest request, HttpClient httpClient, String tokenServerUrl, CancellationToken taskCancellationToken, IClock clock, ILogger logger)\r\n   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.FetchTokenAsync(String userId, TokenRequest request, CancellationToken taskCancellationToken)\r\n   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.FetchTokenAsync(String userId, TokenRequest request, CancellationToken taskCancellationToken)\r\n   at Google.Apis.Auth.OAuth2.Flows.AuthorizationCodeFlow.RefreshTokenAsync(String userId, String refreshToken, CancellationToken taskCancellationToken)\r\n   at Google.Apis.Auth.OAuth2.UserCredential.RefreshTokenAsync(CancellationToken taskCancellationToken)\r\n   at Google.Apis.Auth.OAuth2.TokenRefreshManager.RefreshTokenAsync()\r\n   at Google.Apis.Auth.OAuth2.TokenRefreshManager.GetAccessTokenForRequestAsync(CancellationToken cancellationToken)\r\n   at Google.Apis.Auth.OAuth2.UserCredential.GetAccessTokenWithHeadersForRequestAsync(String authUri, CancellationToken cancellationToken)\r\n   at Grpc.Auth.GoogleAuthInterceptors.<>c__DisplayClass3_0.<<FromCredential>b__0>d.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at Grpc.Core.Internal.NativeMetadataCredentialsPlugin.GetMetadataAsync(AuthInterceptorContext context, IntPtr callbackPtr, IntPtr userDataPtr)","file":"..\..\..\src\core\lib\security\credentials\plugin\plugin_credentials.cc","file_line":93,"grpc_status":14}")

Sadly, setting the InnerException to Google.Apis.Auth.OAuth2.Responses.TokenResponseException is not possible, due to the way the credentials are implemented.
Basically, the credentials in Grpc.Core are a native object (implemented in C-core) that can call back into the managed code when the auth token (= RPC metadata) need to be added to an RPC.
If there is an error invoking the managed auth plugin (which is the case in your example), the managed code has access to the exception object while in the managed callback, but since it it invoked from native code (the credential implementation in C-core), it can only store the exception message as a string and return this message back to the native code.
This is why the RpcException that gets thrown eventually can't really contain the TokenResponseException object itself, but only its message represented as a string. I understand that's less convenient, but since the original exception message is there, it should be possible for users to debug this situation since all the info is there.

This is a limitation that comes from the fact that Grpc.Core intermixes native and managed code and it's very difficult to overcome.

FTR this is the site where the message is generated: https://github.com/grpc/grpc/blob/93733de2532c390433691478765979db297a32d7/src/csharp/Grpc.Core/Internal/NativeMetadataCredentialsPlugin.cs#L94

[AnashOommen]

We added some parsing code in #478. The next step is to actually use this code from the reporting handlers.