Cut v0.2.4 release #415
Merged
Cut v0.2.4 release #415
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JacobGabrielson
approved these changes
May 19, 2021
gfcroft
pushed a commit
to gfcroft/karpenter-provider-aws
that referenced
this pull request
Nov 25, 2023
There was a race with deprovisioning with steps 2 & 3 where we could: 1) See the cluster as unconsolidated 2) Run the deprovisioners unsuccessfully 3) Something happens that makes consolidation possible (e.g. pod deletion) 4) We mark the cluster as unconsolidatable 5) On the next round, do nothing since cluster is unconsolidatable The end result is that we mark the cluster as unconsolidatable which prevents us from trying again to save CPU time. Thankfully this was built with a 5 minute timeout to handle recovering from ICE situations automatically which limited the severity of the problem. This change modifies the code to use a monotonically increasing timestamp. If the cluster state changes (or nothing changes for five minutes), it increases. The race above is now prevented: 1) See the cluster state as X 2) Run the deprovisioners unsuccessfully 3) Something happens that makes consolidation possible (e.g. pod deletion), this increases cluster state to X+1 4) We mark the cluster as unconsolidatable as of X 5) On the next round, see the cluster state as X+1 and retry since we last tried at state X
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.