MetalK8s Upgrade may fail because salt-minion failed to restart #3247
Assignees
Labels
kind:bug
Something isn't working
priority:high
High priority issues, should be worked on ASAP (after urgent issues), not postponed
topic:lifecycle
Issues related to upgrade or downgrade of MetalK8s
topic:operations
Operations-related issues
Milestone
Comments
thomasdanan
added
kind:bug
TeddyAndrieux
added a commit
that referenced
this issue
Apr 8, 2021
As suggested by Salt to restart Salt-minion process we should just `service.restart` in Background and then wait for the Salt-minion to be ready, in our case as part of the orchestrate Sees: https://docs.saltproject.io/en/latest/faq.html#restart-using-states Fixes: #3247
TeddyAndrieux
added a commit
that referenced
this issue
Apr 8, 2021
As suggested by Salt to restart Salt-minion process we should just `service.restart` in Background and then wait for the Salt-minion to be ready, in our case as part of the orchestrate Sees: https://docs.saltproject.io/en/latest/faq.html#restart-using-states Fixes: #3247
TeddyAndrieux
added a commit
that referenced
this issue
Apr 8, 2021
As suggested by Salt to restart Salt-minion process we should just `service.restart` in Background and then wait for the Salt-minion to be ready, in our case as part of the orchestrate Sees: https://docs.saltproject.io/en/latest/faq.html#restart-using-states Fixes: #3247
TeddyAndrieux
added a commit
that referenced
this issue
Apr 8, 2021
As suggested by Salt to restart Salt-minion process we should just `service.restart` in Background and then wait for the Salt-minion to be ready, in our case as part of the orchestrate Sees: https://docs.saltproject.io/en/latest/faq.html#restart-using-states Fixes: #3247
TeddyAndrieux
added a commit
that referenced
this issue
Apr 8, 2021
As suggested by Salt to restart Salt-minion process we should just `service.restart` in Background and then wait for the Salt-minion to be ready, in our case as part of the orchestrate Sees: https://docs.saltproject.io/en/latest/faq.html#restart-using-states Fixes: #3247
TeddyAndrieux
added a commit
that referenced
this issue
Apr 9, 2021
As suggested by Salt to restart Salt-minion process we should just `service.restart` in Background and then wait for the Salt-minion to be ready, in our case as part of the orchestrate Sees: https://docs.saltproject.io/en/latest/faq.html#restart-using-states Fixes: #3247
TeddyAndrieux
added a commit
that referenced
this issue
Apr 9, 2021
As suggested by Salt to restart Salt-minion process we should just `service.restart` in Background and then wait for the Salt-minion to be ready, in our case as part of the orchestrate Sees: https://docs.saltproject.io/en/latest/faq.html#restart-using-states Fixes: #3247
TeddyAndrieux
added a commit
that referenced
this issue
Apr 9, 2021
As suggested by Salt to restart Salt-minion process we should just `service.restart` in Background and then wait for the Salt-minion to be ready, in our case as part of the orchestrate Sees: https://docs.saltproject.io/en/latest/faq.html#restart-using-states Fixes: #3247
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Component:
'upgrade', 'salt'
What happened:
While upgrading a 6 nodes cluster (3CP + 3WP), it systematically failed when restarting
salt-minion. From the minion logs, we can see that some salt-minion service is being shut down and not restarted because some other salt minion is already running. In then salt-minion is never restarted and upgrade process failed. This is happened for each node ofthe cluster.2021-03-29 07:44:47,269 [salt.utils.parsers:1104][WARNING ][1814] Minion received a SIGTERM. Exiting. 2021-03-29 07:44:47,270 [salt.cli.daemons :91 ][INFO ][1814] Shutting down the Salt Minion 2021-03-29 07:46:17,844 [salt.cli.daemons :296 ][INFO ][24545] Setting up the Salt Minion "bootstrap" 2021-03-29 07:46:18,421 [salt.cli.daemons :296 ][INFO ][24569] Setting up the Salt Minion "bootstrap" 2021-03-29 07:46:18,423 [salt.cli.daemons :91 ][INFO ][24569] An instance is already running. Exiting the Salt Minion 2021-03-29 07:46:18,423 [salt.cli.daemons :91 ][INFO ][24569] Shutting down the Salt MinionThis is maybe du to the platform being slow to let the minion completing its job before being shutdown.
What was expected:
The upgrade process should wait for salt minion being properly shutdown before restarting it ...
Steps to reproduce
Resolution proposal (optional):
The text was updated successfully, but these errors were encountered: