Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error TimeoutOverflowWarning: xxxxx does not fit into a 32-bit signed integer when using ManagedIdentityCredential for ServiceBusClient #24003

Closed
6 tasks
cloudxpc opened this issue Nov 27, 2022 · 14 comments
Assignees
Labels
Azure.Identity Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that

Comments

@cloudxpc
Copy link

  • Package Name: @azure/service-bus
  • Package Version: 7.7.3
  • Operating system: Azure Function Windows
  • nodejs
    • version: ~16
  • browser
    • name/version: -
  • typescript
    • version: -
  • Is the bug related to documentation in

Describe the bug
Using ManagedIdentityCredential to create ServiceBusClient, when receiving messages, keep getting:

(node:21960) TimeoutOverflowWarning: 1669616210777 does not fit into a 32-bit signed integer.
Timeout duration was set to 1.

Possible cause:
The expiry time of Managed Identity token is 24h by default. It is used in calculation at source code:
this._tokenTimeout = tokenObject.expiresOnTimestamp - Date.now() - 2 * 60 * 1000;
Then I'm not pretty sure where but it is probably used to 'setTimeout' for refreshing token and issue comes.

To Reproduce
Steps to reproduce the behavior:

  1. create node js code
const credential = new ManagedIdentityCredential('xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx');
const sbClient = new ServiceBusClient('xxxxx.servicebus.windows.net', credential);
const receiver = sbClient.createReceiver(TOPIC_NAME, SUBSCRIPTION_NAME);
const messages = await receiver.receiveMessages(5);
  1. deploy to azure function so that managed identity credential can work

Expected behavior
Should be working fine without errors/warnings.

Screenshots
image

Additional context
please consider to handle the case that token expiry time is longer than 24h

@ghost ghost added needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Nov 27, 2022
@azure-sdk
Copy link
Collaborator

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Azure.Identity:0.55933166,Service Bus:0.42610386,SQL:0.001241071'

@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Nov 28, 2022
@jeremymeng jeremymeng added Client This issue points to a problem in the data-plane of the library. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. labels Nov 28, 2022
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Nov 28, 2022
@jeremymeng
Copy link
Member

@cloudxpc Thank you for reporting the issue! This is likely a known issue in our identity library. /cc @KarishmaGhiya

@azure-sdk
Copy link
Collaborator

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Azure.Identity:0.55933166,Service Bus:0.42610386,SQL:0.001241071'

@KarishmaGhiya
Copy link
Member

Hello, can you confirm the version of @azure/identity being used. I believe this was a bug in 3.1.0 version and was fixed in 3.1.1 @cloudxpc

@cloudxpc
Copy link
Author

Hello, i was using 3.1.0 before. but I changed to @azure/identity 3.1.1 and tested, the issue still comes.

@cmpcdaly
Copy link

cmpcdaly commented Dec 1, 2022

Also experiencing this issue using @azure/identity 3.1.1

@nwf-msr
Copy link
Contributor

nwf-msr commented Dec 3, 2022

@KarishmaGhiya: Presumably commit 4ced48e6f0 was intended to fix this in 3.1.1, but the timeline seems confused. That commit notes the fix to the changelog as if it were part of 3.1.1, but then it bumps the package version fields to "3.1.2". Perhaps two commits (the actual fix and associated changelog entry and version bump to 3.1.1, and then a later one to advance the version field again to 3.1.2) got squashed together by mistake? In any case, that commit is not part of the @azure/identity_3.1.1 tag, having missed it by a few hours: it was committed at 11:33 PM on Nov 18, while the tag's latest commit was at 8:32 PM.

In any case, please cut an @azure/identity 3.1.2 release ASAP as this bug is causing excess CPU load, and I cannot roll back to earlier versions due to needing a version that includes #23232 due to also experiencing #22620 .

@nwf-msr
Copy link
Contributor

nwf-msr commented Dec 3, 2022

Hm. That seems not sufficient in any case. Instrumenting a running application, I have chased values upstream from the timeout registration at

this._tokenRenewalTimer = setTimeout(async () => {
try {
await this._negotiateClaim({
setTokenRenewal: true,
abortSignal: undefined,
timeoutInMs: Constants.defaultOperationTimeoutInMs,
});
} catch (err: any) {
this._logger.logError(
err,
"%s %s '%s' with address %s, an error occurred while renewing the token",
this.logPrefix,
this._type,
this.name,
this.address
);
}
}, this._tokenTimeout);
to the calculation of the timeout value at
this._tokenTimeout = tokenObject.expiresOnTimestamp - Date.now() - 2 * 60 * 1000;
. Confusingly, the value for tokenObject.expiresOnTimestamp is 3340290306000 (and Date.now() was, in the same instance, 1670101953743). That is, expiresOnTimestamp seems to be twice the value of a UNIX timestamp set twelve hours in the future: (3340290306000 / 2 - 1670101953743) / 1000 / 3600 = 11.99979361111111.

@nwf-msr
Copy link
Contributor

nwf-msr commented Dec 3, 2022

Found it after chasing a lot of false leads. #24102 is one possible fix; please feel free to adapt it as appropriate.

@KarishmaGhiya
Copy link
Member

Thanks for the PR @nwf-msr. I shipped the release for @azure/identity 3.1.2 https://www.npmjs.com/package/@azure/identity/v/3.1.2 along with your PR commits. I did the bug fix from a different branch - hotfix/identity_3.1.2

We usually do bug-fixes from a different branch of the name type - hotfix/identity_version which is branched off of the previous release tag and pushed commits required for the fix in that branch. After the release we merge that branch back into the main branch. Perhaps that's why just looking at the commits in main branch will be confusing

@KarishmaGhiya
Copy link
Member

@nwf-msr @cloudxpc @cmpcdaly Please try version @azure/identity 3.1.2 https://www.npmjs.com/package/@azure/identity/v/3.1.2 for the fix and let me know if you still face any problems.

@KarishmaGhiya KarishmaGhiya added the needs-author-feedback Workflow: More information is needed from author to address the issue. label Dec 6, 2022
@nwf-msr
Copy link
Contributor

nwf-msr commented Dec 6, 2022

Thanks! Testing with 3.1.2 and the warnings seem to be gone and the CPU seems to no longer be spinning. :)

@cloudxpc
Copy link
Author

cloudxpc commented Dec 6, 2022

Hi, thanks for the fix! tried with @azure/identity 3.1.2 and issue seems gone.

@ghost ghost added needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team and removed needs-author-feedback Workflow: More information is needed from author to address the issue. labels Dec 6, 2022
@KarishmaGhiya
Copy link
Member

Thanks for the feedback. I will close the issue now :)

@github-actions github-actions bot locked and limited conversation to collaborators Apr 11, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Azure.Identity Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that
Projects
None yet
Development

No branches or pull requests

6 participants