Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduling benchmark #1594

Merged
merged 4 commits into from
Mar 30, 2022
Merged

Scheduling benchmark #1594

merged 4 commits into from
Mar 30, 2022

Conversation

tzneal
Copy link
Contributor

@tzneal tzneal commented Mar 30, 2022

1. Issue, if available:
N/A

2. Description of changes:

Extends scheduling benchmark and adds a minimum pods/sec scheduling threshold to CI builds.

3. How was this change tested?

Unit tests & on EKS scaling up/down the inflate deployment.

4. Does this change impact docs?

  • Yes, PR includes docs updates
  • Yes, issue opened: link to issue
  • No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@netlify
Copy link

netlify bot commented Mar 30, 2022

Deploy Preview for karpenter-docs-prod canceled.

Name Link
🔨 Latest commit 820be0d
🔍 Latest deploy log https://app.netlify.com/sites/karpenter-docs-prod/deploys/6243b47896ef4e0008308cfc

@tzneal tzneal force-pushed the scheduling-benchmark branch from 184115e to 3744b39 Compare March 30, 2022 00:32
@tzneal tzneal marked this pull request as ready for review March 30, 2022 00:45
@tzneal tzneal requested a review from a team as a code owner March 30, 2022 00:45
tzneal added 2 commits March 29, 2022 19:55
name               old time/op   new time/op   delta
Scheduling1-12      1.08ms ± 1%   0.85ms ± 2%  -21.52%  (p=0.008 n=5+5)
Scheduling50-12     33.3ms ± 3%   21.5ms ± 4%  -35.43%  (p=0.008 n=5+5)
Scheduling100-12    62.4ms ± 2%   39.9ms ± 3%  -36.05%  (p=0.008 n=5+5)
Scheduling500-12     231ms ± 1%    151ms ± 2%  -34.80%  (p=0.008 n=5+5)
Scheduling1000-12    504ms ± 1%    367ms ± 8%  -27.16%  (p=0.008 n=5+5)
Scheduling2000-12    1.18s ± 4%    0.88s ± 2%  -25.07%  (p=0.008 n=5+5)
Scheduling5000-12    4.50s ± 2%    3.94s ± 6%  -12.48%  (p=0.008 n=5+5)

name               old pods/sec  new pods/sec  delta
Scheduling1-12         925 ± 1%     1178 ± 2%  +27.43%  (p=0.008 n=5+5)
Scheduling50-12      1.50k ± 3%    2.33k ± 4%  +54.94%  (p=0.008 n=5+5)
Scheduling100-12     1.60k ± 2%    2.51k ± 3%  +56.42%  (p=0.008 n=5+5)
Scheduling500-12     2.16k ± 1%    3.32k ± 2%  +53.41%  (p=0.008 n=5+5)
Scheduling1000-12    1.99k ± 1%    2.73k ± 7%  +37.57%  (p=0.008 n=5+5)
Scheduling2000-12    1.70k ± 4%    2.26k ± 2%  +33.43%  (p=0.008 n=5+5)
Scheduling5000-12    1.11k ± 2%    1.27k ± 6%  +14.42%  (p=0.008 n=5+5)
@tzneal tzneal force-pushed the scheduling-benchmark branch from 3744b39 to ee5d730 Compare March 30, 2022 00:55
Makefile Outdated
@@ -6,6 +6,13 @@ LDFLAGS ?= -ldflags=-X=github.com/aws/karpenter/pkg/utils/project.Version=$(shel
GOFLAGS ?= -tags=$(CLOUD_PROVIDER) $(LDFLAGS)
WITH_GOFLAGS = GOFLAGS="$(GOFLAGS)"

# detect nice binary
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the interest of keeping this Makefile manageable, I think it's fair to assume that you need nice installed if you want to run the benchmark.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine removing it entirely, doubt CI has it and if you want it locally you can just run nice -19 make benchmark

Copy link
Contributor

@ellistarn ellistarn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's leave out of CI for now. GHA is flakey. Cool stuff though!

@tzneal tzneal merged commit 3739122 into aws:main Mar 30, 2022
@tzneal tzneal deleted the scheduling-benchmark branch March 30, 2022 02:00
@suket22 suket22 mentioned this pull request May 23, 2022
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants