Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the rollout will stuck in "progressing", if the new spec is applied with ("replicas" increasement, and pod spec is changed) when the rollout is in "paused". #3256

Closed
2 tasks done
andyliuliming opened this issue Dec 15, 2023 · 4 comments · Fixed by #3272
Labels
bug Something isn't working

Comments

@andyliuliming
Copy link
Contributor

andyliuliming commented Dec 15, 2023

Checklist:

  • I've included steps to reproduce the bug.
  • I've included the version of argo rollouts.

Describe the bug

To Reproduce

  1. deploy one "Rollout" with:
    replicas: 3
    strategy:
    canary:
    dynamicStableScale: true
    minPodsPerReplicaSet: 1
    maxSurge: 1
    maxUnavailable: 1
    steps:
    • setCanaryScale: {weight: 60}
    • setWeight: 60
    • pause: {duration: 1000m}

and then deploy change the replias of the Rollout.

then it will be stuck there. the promote also not work.

I think it's because the FindNewReplicaSet will return nil in this case.

Expected behavior

Screenshots

Version

latest in master branch.

# Paste the logs from the rollout controller

# Logs for the entire controller:
kubectl logs -n argo-rollouts deployment/argo-rollouts

# Logs for a specific rollout:
kubectl logs -n argo-rollouts deployment/argo-rollouts | grep rollout=<ROLLOUTNAME

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

@andyliuliming andyliuliming added the bug Something isn't working label Dec 15, 2023
@zachaller
Copy link
Collaborator

zachaller commented Dec 15, 2023

Are you able to reproduce the 100% of the time, I am suspicious that something is going on but am unable to reproduce. If you can get this to trigger easily can you maybe explain a bit more your exact steps, not sure what your doing exactly when you say deploy change the replias of the Rollout

@zachaller
Copy link
Collaborator

I have been able to replicate something similar if I update both the replica count and the pod template at the same time, was this what you where doing?

@andyliuliming andyliuliming changed the title the rollout will stuck in "progressing", if the "replicas" is increased when the rollout is in "paused". the rollout will stuck in "progressing", if the new spec is applied with ("replicas" increasement, and pod spec is changed) when the rollout is in "paused". Dec 16, 2023
@andyliuliming
Copy link
Contributor Author

I have been able to replicate something similar if I update both the replica count and the pod template at the same time, was this what you where doing?

yes, forgot to mention that the pod spec is changed together with the replias.

@andyliuliming
Copy link
Contributor Author

I think this might help fix that:
#3257

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants