Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add context dealine when calling token APIs #126

Merged
merged 1 commit into from
Oct 25, 2024
Merged

Conversation

eamonnotoole
Copy link
Contributor

In this PR we extensively rework the DoRetries function in pkg/token-util:

  • we pass-in a context from the calling function
  • from this context we create a child context with a deadline of 3 seconds, which is less than the retry interval of 5 seconds
  • we pass-in a function signature that:
    • takes a context as input
    • returns a request, respone and error
  • we execute this passed-in function with the new child context
  • the passed-in function is expected to do the following:
    • create a new request with the context passed-in to the function, this is the child request with the deadline of 3 seconds
    • use a httpclient.Do function to execute the request
    • return the request, response and any error from the httpclient.Do
  • if the request returned is non-nil and the context has exceeded its deadline, then we retry and decrement the number of retries
  • if the error returned is non-nil then we return the error
  • if the response status can't be retried we return the response
  • if we reach the bottom of the loop, which means that we have a response code that can be retried, we decrement the retry counter and retry

The objective of the above is to cancel requests that have stalled for 3 seconds and then retry. This is to help alleviate errors we are seeing with IAM token generation requests that stall.

We've had to rework the calls to DoRetries from issuertoken and identitytoken.

We've also had to rework unit-tests. We've included a new unit-test to check that a DeadlineExceeded error from the request context will be retried.

In this PR we extensively rework the DoRetries function in
pkg/token-util:
- we pass-in a context from the calling function
- from this context we create a child context with a deadline of 3
  seconds, which is less than the retry interval of 5 seconds
- we pass-in a function signature that:
  - takes a context as input
  - returns a request, respone and error
- we execute this passed-in function with the new child context
- the passed-in function is expected to do the following:
  - create a new request with the context passed-in to the function,
    this is the child request with the deadline of 3 seconds
  - use a httpclient.Do function to execute the request
  - return the request, response and any error from the httpclient.Do
- if the request returned is non-nil and the context has exceeded its
  deadline, then we retry and decrement the number of retries
- if the error returned is non-nil then we return the error
- if the response status can't be retried we return the response
- if we reach the bottom of the loop, which means that we have a
  response code that can be retried, we decrement the retry counter and
  retry

The objective of the above is to cancel requests that have stalled for 3
seconds and then retry.  This is to help alleviate errors we are seeing
with IAM token generation requests that stall.

We've had to rework the calls to DoRetries from issuertoken and
identitytoken.

We've also had to rework unit-tests.  We've included a new unit-test to
check that a DeadlineExceeded error from the request context will be
retried.

Signed-off-by: Eamonn O'Toole <[email protected]>
@eamonnotoole eamonnotoole merged commit d3c8aa0 into main Oct 25, 2024
4 checks passed
@eamonnotoole eamonnotoole deleted the add-context-deadline branch October 25, 2024 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants