Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: reduce crp throttling in attach disk scenario #621

Merged
merged 1 commit into from
May 8, 2021

Conversation

andyzhangx
Copy link
Member

What type of PR is this?
/kind bug

What this PR does / why we need it:
fix: reduce crp throttling in attach disk scenario
original CacheReadTypeForceRefresh is not necessary since once attach/detach completed, cache is already cleaned, this operation could clean cache for every disk attach, finally cause crp throttling, e.g.

I0508 05:26:08.763336       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I0508 05:26:08.763360       1 utils.go:96] GRPC request: {"node_id":"aks-agentpool-15002531-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"skuname":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1620324123891-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/xxx/resourceGroups/aks55h93-nodegroup/providers/Microsoft.Compute/disks/pvc-f0dfbac9-6737-4462-98ff-1d8d793a0021"}
E0508 05:26:08.782792       1 azure_vmss.go:700] VirtualMachineScaleSetVMsClient.List failed: &{true 0 2021-05-08 05:29:59.777620028 +0000 UTC m=+1787.759114644 azure cloud provider throttled for operation VMSSVMList with reason "client throttled"}
E0508 05:26:08.782826       1 azure_controller_common.go:445] error of getting data disks for node "aks-agentpool-15002531-vmss00000n": Retriable: true, RetryAfter: 230s, HTTPStatusCode: 0, RawError: azure cloud provider throttled for operation VMSSVMList with reason "client throttled"
E0508 05:26:08.782842       1 controllerserver.go:463] Attach volume "/subscriptions/xxx/resourceGroups/aks55h93-nodegroup/providers/Microsoft.Compute/disks/pvc-d48cabdc-9985-411a-a31d-769d19fd7595" to instance "aks-agentpool-15002531-vmss00000n" failed with Retriable: true, RetryAfter: 230s, HTTPStatusCode: 0, RawError: azure cloud provider throttled for operation VMSSVMList with reason "client throttled"

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Release note:

fix: reduce crp throttling in attach disk scenario

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 8, 2021
@k8s-ci-robot k8s-ci-robot requested review from feiskyer and nilo19 May 8, 2021 07:46
@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels May 8, 2021
@andyzhangx
Copy link
Member Author

/retest

Copy link
Member

@ZeroMagic ZeroMagic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 8, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx, ZeroMagic

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@andyzhangx
Copy link
Member Author

/retest

1 similar comment
@andyzhangx
Copy link
Member Author

/retest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants