Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add kube client qps configuration w/ env vars and cli overrides #723

Merged
merged 1 commit into from
Oct 1, 2021

Conversation

bwagner5
Copy link
Contributor

@bwagner5 bwagner5 commented Oct 1, 2021

1. Issue, if available:
N/A

2. Description of changes:

  • Allow ENV vars to be used to configure CLI args (easier w/ Helm).
    • Precedense: CLI Arg, ENV Var, hard coded default
  • Set default QPS for kubernetes client to 200 base, 300 burst QPS. 10x the default values to allow karpenter to bind pods to nodes without getting client-side throttled often.

3. Does this change impact docs?

  • Yes, PR includes docs updates
  • Yes, issue opened: link to issue
  • No

Docs Issue: #722

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@netlify
Copy link

netlify bot commented Oct 1, 2021

✔️ Deploy Preview for karpenter-docs-prod canceled.

🔨 Explore the source changes: b25dba1

🔍 Inspect the deploy log: https://app.netlify.com/sites/karpenter-docs-prod/deploys/61574794f94cb7000860de26

@bwagner5 bwagner5 requested a review from ellistarn October 1, 2021 16:58
flag.IntVar(&options.MetricsPort, "metrics-port", env.WithDefaultInt("METRICS_PORT", 8080), "The port the metric endpoint binds to for operating metrics about the controller itself")
flag.IntVar(&options.HealthProbePort, "health-probe-port", env.WithDefaultInt("HEALTH_PROBE_PORT", 8081), "The port the health probe endpoint binds to for reporting controller health")
flag.IntVar(&options.KubeClientQPS, "kube-client-qps", env.WithDefaultInt("KUBE_CLIENT_QPS", 200), "The smoothed rate of qps to kube-apiserver")
flag.IntVar(&options.KubeClientBurst, "kube-client-burst", env.WithDefaultInt("KUBE_CLIENT_BURST", 300), "The maximum allowed burst of queries to the kube-apiserver")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slick

flag.Parse()

config := controllerruntime.GetConfigOrDie()
config.RateLimiter = flowcontrol.NewTokenBucketRateLimiter(float32(options.KubeClientQPS), options.KubeClientBurst)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this different from setting config.KubeClientQPS? Do they potentially do different things? The same thing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's the same and overrides those values:

	// Rate limiter for limiting connections to the master from this client. If present overwrites QPS/Burst
	RateLimiter flowcontrol.RateLimiter

https://pkg.go.dev/k8s.io/client-go/rest#Config

pkg/utils/env/env.go Outdated Show resolved Hide resolved
@bwagner5 bwagner5 merged commit 7541834 into aws:main Oct 1, 2021
@bwagner5 bwagner5 deleted the qps-config branch October 1, 2021 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants