Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ICE cache bin-packing validation fix #1127

Merged
merged 3 commits into from
Jan 11, 2022
Merged

Conversation

bwagner5
Copy link
Contributor

1. Issue, if available:
#1126

2. Description of changes:
Fixes an issue where if an instance type offering is found to be unavailable and therefore placed in the ICE cache, the binpacking logic could still use the instance type in the packing.

The bug is occurring in the bin packing's packable validation logic. Packables (Instance Type and Offering) are validated by zone and capacity type separately even though an offering is the tuple of (zone, capacity type). This change validates an offering and removes the independent zone and capacity-type validation.

The issue was originally found while doing load tests where I ran into an Insufficient Capacity Error (ICE). After the ICE, Karpenter behaved very strangely, by throwing "Security Group does not belong to the same network as Subnet". This error is because of the overrides bug described in the issue linked.

3. How was this change tested?

  • Reproduced in a cluster by injecting ICEs directly into the cache (was able to reproduce and verified the fix)
  • Added a unit-test that fails without the patch and passes with the patch.

4. Does this change impact docs?

  • Yes, PR includes docs updates
  • Yes, issue opened: link to issue
  • No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@netlify
Copy link

netlify bot commented Jan 11, 2022

✔️ Deploy Preview for karpenter-docs-prod canceled.

🔨 Explore the source changes: ed5011d

🔍 Inspect the deploy log: https://app.netlify.com/sites/karpenter-docs-prod/deploys/61ddd661f8115800072fde07

ellistarn
ellistarn previously approved these changes Jan 11, 2022
@bwagner5 bwagner5 merged commit f8de85c into aws:main Jan 11, 2022
@bwagner5 bwagner5 deleted the ice-cache-fix branch January 11, 2022 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants