Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: do not require pod readiness when switching desired service selector on abort #3338

Merged
merged 5 commits into from
Feb 5, 2024
+5 −3
Merged

fix: do not require pod readiness when switching desired service selector on abort #3338

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c5f4254
do not switch service selectors back when using alb due to race betwe…
zachaller Jan 30, 2024
aca715a
update tests for alb
zachaller Jan 30, 2024
1ed3dbe
lets not check for readiness instead
zachaller Jan 31, 2024
cb3e7c4
clean up notes
zachaller Feb 5, 2024
38ab9c6
fix /
zachaller Feb 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions rollout/trafficrouting.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,13 +179,15 @@ func (c *rolloutContext) reconcileTrafficRouting() error {
desiredWeight = c.calculateDesiredWeightOnAbortOrStableRollback()
if (c.rollout.Spec.Strategy.Canary.DynamicStableScale && desiredWeight == 0) || !c.rollout.Spec.Strategy.Canary.DynamicStableScale {
// If we are using dynamic stable scale we need to also make sure that desiredWeight=0 aka we are completely
// done with aborting before resetting the canary service selectors back to stable
err = c.ensureSVCTargets(c.rollout.Spec.Strategy.Canary.CanaryService, c.stableRS, true)
// done with aborting before resetting the canary service selectors back to stable. For non-dynamic scale we do not check for availability because we are
// fully aborted and stable pods will be there, if we check for availability it causes issues with ALB readiness gates if all stable pods
// have the desired readiness gate on them during an abort we get stuck in a loop because all the stable go unready and rollouts won't be able
// to switch the desired services because there is no ready pods which causes pods to get stuck progressing forever waiting for readiness.
err = c.ensureSVCTargets(c.rollout.Spec.Strategy.Canary.CanaryService, c.stableRS, false)
if err != nil {
return err
}
}

err := reconciler.RemoveManagedRoutes()
if err != nil {
return err
Expand Down
Loading