Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor gce.RegenerateMigInstancesCache() to use Instance.List #6955

Merged
merged 1 commit into from
Jul 9, 2024

Conversation

damikag
Copy link
Contributor

@damikag damikag commented Jun 21, 2024

refactor gce.RegenerateMigInstancesCache() to use Instance.List API

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR refactor the RegenerateMigInstancesCache() to use Instance.List API when fetching MIG instances.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

With this change GCE cloud provider will use Instance.List api to list mig instances.
 IGM.ListManagedInstances api will be used as a fall back mechanism and for listing instances for migs that have instances in creating or deleting states.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 21, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @damikag. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 21, 2024
cluster-autoscaler/cloudprovider/gce/mig_info_provider.go Outdated Show resolved Hide resolved
cluster-autoscaler/cloudprovider/gce/mig_info_provider.go Outdated Show resolved Hide resolved
cluster-autoscaler/cloudprovider/gce/mig_info_provider.go Outdated Show resolved Hide resolved
for _, err := range errors {
func (c *cachingMigInfoProvider) isMigInCreatingOrDeletingInstanceState(mig Mig) bool {
migRef := mig.GceRef()
state, found := c.cache.GetMigInstancesState(migRef)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't thought about this before, but we need a better name for this. Maybe MigInstancesCount or MigInstancesStateCount? Because now, when I read below state[cloudprovider.InstanceCreating] it reads as state[state] and doesn't parse well in my head. It can be a separate PR though.

cluster-autoscaler/cloudprovider/gce/mig_info_provider.go Outdated Show resolved Hide resolved
cluster-autoscaler/cloudprovider/gce/mig_info_provider.go Outdated Show resolved Hide resolved
func groupInstancesToMigs(instances []GceInstance) map[GceRef][]GceInstance {
migToInstances := map[GceRef][]GceInstance{}
for _, instance := range instances {
if _, found := migToInstances[instance.Igm]; found {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

igm can be an empty GceRef, or a non-existent GceRef how do we want to handle that?

func createIgmRef(gceInstance *gce.Instance, loggingQuota *klogx.Quota) GceRef {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We update cache only for the migs that are known and simply ignore instances from empty GceRefs or non-existent GceRefs. Did you notice any issue with that?

@damikag damikag force-pushed the refactor-mig-fetch branch from 95c4ee6 to 22f6e7e Compare July 2, 2024 12:16
@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 2, 2024
@damikag damikag force-pushed the refactor-mig-fetch branch 2 times, most recently from 201869c to 8ea6aec Compare July 2, 2024 15:20
Copy link
Member

@bskiba bskiba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks good, minor comments. Thank you for the extensive unit tests!

}
return updateErr
}

migs := c.migLister.GetMigs()
errors := make([]error, len(migs))
workqueue.ParallelizeUntil(context.Background(), c.concurrentGceRefreshes, len(migs), func(piece int) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be skipped if bulkGceMigInstancesListingEnabled=true?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We are doing the same thing in different ways.

cluster-autoscaler/cloudprovider/gce/mig_info_provider.go Outdated Show resolved Hide resolved
cluster-autoscaler/cloudprovider/gce/mig_info_provider.go Outdated Show resolved Hide resolved
cluster-autoscaler/config/autoscaling_options.go Outdated Show resolved Hide resolved
cluster-autoscaler/config/autoscaling_options.go Outdated Show resolved Hide resolved
cluster-autoscaler/main.go Outdated Show resolved Hide resolved
@damikag damikag force-pushed the refactor-mig-fetch branch from 8ea6aec to 8971a29 Compare July 4, 2024 14:48
@bskiba
Copy link
Member

bskiba commented Jul 8, 2024

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 8, 2024
@x13n
Copy link
Member

x13n commented Jul 9, 2024

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bskiba, damikag, x13n

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 9, 2024
@k8s-ci-robot k8s-ci-robot merged commit 01e8918 into kubernetes:master Jul 9, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/cluster-autoscaler area/provider/gce cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants