Benchmarking existing Buildkite jobs against v2 of BK cluster topology. #5610
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 tasks
O1ahmad
force-pushed
the
benchmark_coda_jobs
branch
from
737d11f to
2966958
Compare
O1ahmad
force-pushed
the
benchmark_coda_jobs
branch
4 times, most recently
from
fe16c7e to
782df48
Compare
O1ahmad
changed the title
O1ahmad
force-pushed
the
benchmark_coda_jobs
branch
from
4d3ce9c to
92e3f1c
Compare
O1ahmad
changed the title
O1ahmad
force-pushed
the
benchmark_coda_jobs
branch
from
a5d6229 to
70254d7
Compare
O1ahmad
force-pushed
the
benchmark_coda_jobs
branch
from
f8f059d to
9b4a625
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pre-emptible C2-CPU instances are about $0.20 (20 cents)/hour, which is considerably lower than the non-preemptible n1-standard-* machine types per hour previously provisioned and only ~8 cents/hr more than the pre-emptible variants. Due to this reasoning (almost negligible price difference and hopefully significant performance all around) on top of a similar basis for other types (albeit with likely less performance gains), this change replaces all Buildkite agent GKE cluster node-pools with
c2-standard-16machine types.Also, two new Size classes were defined to allow for a bit more granularity when assigning jobs while more expressive agent targeting logic is in the works (basically, more of smaller types were provisioned with the idea that we should strive to fit most jobs on the smaller types for general availability at scale) and enough of the larger types to mitigate the risk of bottlenecks.
Testing: Buildkite CI (examples)
Checklist:
Closes #5028