Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"pause" image might be removed when a node goes in "DiskPressure" #3601

Closed
TeddyAndrieux opened this issue Nov 15, 2021 · 2 comments · Fixed by #3624
Closed

"pause" image might be removed when a node goes in "DiskPressure" #3601

TeddyAndrieux opened this issue Nov 15, 2021 · 2 comments · Fixed by #3624
Assignees
@TeddyAndrieux
Labels
complexity:easy Something that requires less than a day to fix kind:bug Something isn't working priority:high High priority issues, should be worked on ASAP (after urgent issues), not postponed

Comments

@TeddyAndrieux
Copy link
Collaborator

TeddyAndrieux commented Nov 15, 2021 •
edited
Loading

Component:

'kubernetes'

What happened:

If for whatever reason a node goes in "DiskPressure" kubelet will remove the "pause" image, which means if the node does not have internet access, new containers will not be able to get scheduled on this node

What was expected:

Pause image should NOT get removed

Resolution proposal (optional):

See: kubernetes/kubernetes#81756
We have to set --pod-infra-container-image for kubelet
@TeddyAndrieux TeddyAndrieux added kind:bug Something isn't working complexity:easy Something that requires less than a day to fix priority:high High priority issues, should be worked on ASAP (after urgent issues), not postponed labels Nov 15, 2021
@NicolasT
Copy link
Contributor

Whoa, good catch! We inject it at deployment time so things work but that's where things end. Should indeed ensure it gets pulled from the internal repo.

@TeddyAndrieux
Copy link
Collaborator Author

NOTE: We may face the exact same issue for the image used by the registry container image: docker.io/library/nginx:1.19.6-alpine, we should consider having some logic (likely in salt) so that we make sure that those 2 images are always available

@TeddyAndrieux TeddyAndrieux self-assigned this Dec 1, 2021
@TeddyAndrieux TeddyAndrieux mentioned this issue Dec 2, 2021
Merged
TeddyAndrieux added a commit that referenced this issue Dec 2, 2021
@TeddyAndrieux
salt,build: Mark the pause image as pod infra container image
40a05b8 
We mark the `pause` image used by containerD as `pod infra container
image` for kubelet, so that kubelet does not remove this image.

We also put `pause` image in the embedded registry, so that we are sure
that node does not fetch this image from the internet.

NOTE: Since we fetch the pause image from the embedded registry we no
longer need to import it manually with salt on every node but only for
the registry node

Fixes: #3601
TeddyAndrieux added a commit that referenced this issue Dec 2, 2021
@TeddyAndrieux
salt,build: Mark the pause image as pod infra container image
d84abec 
We mark the `pause` image used by containerD as `pod infra container
image` for kubelet, so that kubelet does not remove this image.

We also put `pause` image in the embedded registry, so that we are sure
that node does not fetch this image from the internet.

NOTE: Since we fetch the pause image from the embedded registry we no
longer need to import it manually with salt on every node but only for
the registry node

Fixes: #3601
@bert-e bert-e closed this as completed in bef4690 Dec 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@TeddyAndrieux TeddyAndrieux

Labels
complexity:easy Something that requires less than a day to fix kind:bug Something isn't working priority:high High priority issues, should be worked on ASAP (after urgent issues), not postponed
Projects
None yet
Milestone
No milestone
2 participants
@NicolasT @TeddyAndrieux