Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade: Wait properly for containers to be "running" after bootstrap kubelet upgrade #3105

Merged
merged 3 commits into from
Feb 10, 2021

Conversation

TeddyAndrieux
Copy link
Collaborator

Component:

'lifecycle'

Context:

Improve robustness of the upgrade script

Summary:

After kubelet upgrade all static pod may restart, so we need to wait for
some "useful" pods to be running before going further in the upgrade
process.
NOTE: That before this PR we already sleep 20 seconds, so this commit
only add a proper check after this sleep


@TeddyAndrieux TeddyAndrieux added topic:flakiness Some test are flaky and cause CI to do transient failing topic:lifecycle Issues related to upgrade or downgrade of MetalK8s complexity:easy Something that requires less than a day to fix labels Feb 10, 2021
@TeddyAndrieux TeddyAndrieux requested a review from a team February 10, 2021 10:49
@bert-e
Copy link
Contributor

bert-e commented Feb 10, 2021

Hello teddyandrieux,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Status report is not available.

@bert-e
Copy link
Contributor

bert-e commented Feb 10, 2021

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • one peer

Peer approvals must include at least 1 approval from the following list:

@TeddyAndrieux TeddyAndrieux force-pushed the improvement/wait-usefull-container-in-upgrade branch from 84be703 to 149fb27 Compare February 10, 2021 10:50
Copy link
Contributor

@alexandre-allard alexandre-allard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only few typos, otherwise LGTM.

CHANGELOG.md Outdated Show resolved Hide resolved
salt/tests/unit/modules/test_cri.py Outdated Show resolved Hide resolved
salt/tests/unit/modules/test_cri.py Outdated Show resolved Hide resolved
salt/tests/unit/modules/test_cri.py Outdated Show resolved Hide resolved
scripts/upgrade.sh.in Outdated Show resolved Hide resolved
After kubelet upgrade all static pod may restart, so we need to wait for
some "useful" pods to be running before going further in the upgrade
process.
NOTE: That before this PR we already sleep 20 seconds, so this commit
only add a proper check after this sleep
Time to time, especially on really slow platform, we got failure because
salt state execution timeout. Increase salt-master default timeout to 20
@TeddyAndrieux
Copy link
Collaborator Author

Only few typos, otherwise LGTM.

Fixed 😃

@TeddyAndrieux
Copy link
Collaborator Author

/approve

@bert-e
Copy link
Contributor

bert-e commented Feb 10, 2021

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

  • ✔️ development/2.8

The following branches will NOT be impacted:

  • development/1.0
  • development/1.1
  • development/1.2
  • development/1.3
  • development/2.0
  • development/2.1
  • development/2.2
  • development/2.3
  • development/2.4
  • development/2.5
  • development/2.6
  • development/2.7

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

  • Any commit you add on the source branch will trigger a new cycle after the
    current queue is merged.
  • Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Feb 10, 2021

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/2.8

The following branches have NOT changed:

  • development/1.0
  • development/1.1
  • development/1.2
  • development/1.3
  • development/2.0
  • development/2.1
  • development/2.2
  • development/2.3
  • development/2.4
  • development/2.5
  • development/2.6
  • development/2.7

Please check the status of the associated issue None.

Goodbye teddyandrieux.

@bert-e bert-e merged commit ea3d265 into development/2.8 Feb 10, 2021
@bert-e bert-e deleted the improvement/wait-usefull-container-in-upgrade branch February 10, 2021 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
complexity:easy Something that requires less than a day to fix topic:flakiness Some test are flaky and cause CI to do transient failing topic:lifecycle Issues related to upgrade or downgrade of MetalK8s
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants