Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dind not working using ubuntu 22.04.1 LTS image #14139

Closed
Tracked by #14140
ricardojdsilva87 opened this issue Aug 18, 2022 · 9 comments
Closed
Tracked by #14140

dind not working using ubuntu 22.04.1 LTS image #14139

ricardojdsilva87 opened this issue Aug 18, 2022 · 9 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed.

Comments

@ricardojdsilva87
Copy link

/kind bug

1. What kops version are you running? The command kops version, will display
this information.

1.24.1

2. What Kubernetes version are you running? kubectl version will print the
version if a cluster is running or provide the Kubernetes version specified as
a kops flag.

1.22.12

3. What cloud provider are you using?
AWS

4. What commands did you run? What is the simplest way to reproduce this issue?
Added the following image to the Instance group:
099720109477/ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-20220810

5. What happened after the commands executed?
We have configured some pods with dind containers that work with the image:
099720109477/ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220810
but after changing to the Ubuntu 22.04 image the logs show the error
cgroup mountpoint does not exist

6. What did you expect to happen?
The same behaviour on either Ubuntu AMIs

7. Please provide your cluster manifest. Execute
kops get --name my.example.com -o yaml to display your cluster manifest.
You may want to remove your cluster name and other sensitive information.

apiVersion: kops.k8s.io/v1alpha2
kind: Cluster
metadata:
  creationTimestamp: "2018-09-14T12:20:46Z"
  generation: 79
  name: cluster-name
spec:
  api:
    loadBalancer:
      class: Classic
      type: Internal
  authorization:
    rbac: {}
  certManager:
    enabled: true
    managed: false
  channel: stable
  cloudConfig:
    awsEBSCSIDriver:
      enabled: true
  cloudControllerManager: {}
  cloudLabels:
    label1: label
    env: prod
  cloudProvider: aws
  configBase: s3://bucket name
  containerRuntime: containerd
  containerd:
    registryMirrors:
      docker.io:
      - https://proxy.url.com
      - https://registry-1.docker.io
    version: 1.6.6
  dnsZone: cluster domain
  etcdClusters:
  - etcdMembers:
    - encryptedVolume: true
      instanceGroup: master-eu-central-1a
      name: a
    - encryptedVolume: true
      instanceGroup: master-eu-central-1b
      name: b
    - encryptedVolume: true
      instanceGroup: master-eu-central-1c
      name: c
    manager:
      env:
      - name: ETCD_MANAGER_HOURLY_BACKUPS_RETENTION
        value: 15d
      - name: ETCD_MANAGER_DAILY_BACKUPS_RETENTION
        value: 90d
    name: main
  - etcdMembers:
    - encryptedVolume: true
      instanceGroup: master-eu-central-1a
      name: a
    - encryptedVolume: true
      instanceGroup: master-eu-central-1b
      name: b
    - encryptedVolume: true
      instanceGroup: master-eu-central-1c
      name: c
    manager:
      env:
      - name: ETCD_MANAGER_HOURLY_BACKUPS_RETENTION
        value: 15d
      - name: ETCD_MANAGER_DAILY_BACKUPS_RETENTION
        value: 90d
    name: events
  externalPolicies:
    master:
    - arn:aws:iam::account:policy/efs_csi_volume_policy
    - arn:aws:iam::account:policy:policy/ebs_csi_volume_policy
    - arn:aws:iam::account:policy:policy/additional_master_node_policy
    node:
    - arn:aws:iam::account:policy:policy/efs_csi_volume_policy
    - arn:aws:iam::account:policy:policy/additional_worker_node_policy
    - arn:aws:iam::account:policy:policy/ebs_csi_volume_policy
  iam:
    legacy: false
  kubeAPIServer:
    allowPrivileged: true
  kubeDNS:
    nodeLocalDNS:
      enabled: true
    provider: CoreDNS
  kubelet:
    anonymousAuth: false
    authenticationTokenWebhook: true
    authorizationMode: Webhook
  kubernetesApiAccess:
  - 0.0.0.0/0
  kubernetesVersion: v1.22.12
  masterInternalName: api.xxxxx
  masterPublicName: api.xxxxx
  metricsServer:
    enabled: true
  networkCIDR: 10.xxx
  networkID: vpc-xxx
  networking:
    flannel:
      backend: vxlan
  nonMasqueradeCIDR: 100.xxx
  sshAccess:
  - 10.xxx
  subnets:
  - cidr: 10.xxx
    name: eu-central-1a
    type: Private
    zone: eu-central-1a
  - cidr: 10.xxx
    name: eu-central-1b
    type: Private
    zone: eu-central-1b
  - cidr: 10.xxx
    name: eu-central-1c
    type: Private
    zone: eu-central-1c
  - cidr: 10.xxx
    name: public-eu-central-1a
    type: Utility
    zone: eu-central-1a
  - cidr: 10.xxx
    name: public-eu-central-1b
    type: Utility
    zone: eu-central-1b
  - cidr: 10.xxx
    name: public-eu-central-1c
    type: Utility
    zone: eu-central-1c
  topology:
    dns:
      type: Public
    masters: private
    nodes: private

9. Anything else do we need to know?
Wanted to ask if there is some setting that could be enabled on the cluster spec regarding the cgroup setting. I've found this one, but I think it's not related to this issue.
Also found the following PRs that are related to cgroup enablement (in this case the systemd should be enabled by default since we are using k8s 1.22.12)
#12917
#10846

This is the pod template we are using for dind:

  - name: dind
    env:
      - name: DOCKER_TLS_CERTDIR
        value: ''
    image: docker:stable-dind
    command: [ 'dockerd-entrypoint.sh', '--registry-mirror', 'https://url']
    securityContext:
      privileged: true
    volumeMounts:
      - name: dind-storage
        mountPath: /var/lib/docker

What I suspect is that the cgroup is somehow missing from the Ubuntu 22.04 base AMI, but since kops supports the image just wanted to ask if I've missed some configuration.
Like mentioned before keeping the same k8s version and the latest kops version 1.24.1 and reverting back to Ubuntu 20.04 AMI, the builds started working again.
Thanks for the help!

@k8s-ci-robot k8s-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Aug 18, 2022
@olemarkus
Copy link
Member

We know of several issues with ubuntu 2204. So while you can use it as a base image, I wouldn't call it supported. We''ll track this issue though.

@ricardojdsilva87
Copy link
Author

Thanks for the confirmation @olemarkus.
I've reverted back to Ubuntu 20.04 and everything is working fine. The ticket can serve as tracking as you said.

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 17, 2022
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 17, 2022
@johngmyers
Copy link
Member

/remove-lifecycle rotten

@k8s-ci-robot k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jan 3, 2023
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 3, 2023
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle rotten
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels May 4, 2023
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

@k8s-ci-robot k8s-ci-robot closed this as not planned Won't fix, can't repro, duplicate, stale Jun 3, 2023
@k8s-ci-robot
Copy link
Contributor

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed.
Projects
None yet
Development

No branches or pull requests

5 participants