csi-driver/cinder is not working on fedora core os hosts

[svenuthe]

When using fedora core os as target system, no additional users (like kube-cert) are added. Hence some of the csi_driver/cinder tasks fails tasks fail:

Can we omit the group assignment group: "{{ kube_cert_group }}" from extra_playbooks/roles/kubernetes-apps/csi_driver/cinder/tasks/main.yml when deploying to fedora core os?

Environment:

• Cloud provider or hardware configuration: external openstack (which was setuped by contrib terraform module)

• OS (printf "$(uname -srm)\n$(cat /etc/os-release)\n"):
  OS of the target host systems:

NAME="Fedora Linux"
VERSION="35.20220116.3.0 (CoreOS)"
ID=fedora
VERSION_ID=35
VERSION_CODENAME=""
PLATFORM_ID="platform:f35"
PRETTY_NAME="Fedora CoreOS 35.20220116.3.0"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:35"
HOME_URL="https://getfedora.org/coreos/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora-coreos/"
SUPPORT_URL="https://github.com/coreos/fedora-coreos-tracker/"
BUG_REPORT_URL="https://github.com/coreos/fedora-coreos-tracker/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=35
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=35
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="CoreOS"
VARIANT_ID=coreos
OSTREE_VERSION='35.20220116.3.0'
DEFAULT_HOSTNAME=localhost

• Version of Ansible (ansible --version):

ansible 2.9.27
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['~/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.10/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.10.2 (main, Jan 17 2022, 00:00:00) [GCC 11.2.1 20211203 (Red Hat 11.2.1-7)]

• Version of Python (python --version):
  Python 3.10.2

Kubespray version (commit) (git rev-parse --short HEAD):
5695c892

Network plugin used:
calico

Full inventory with variables (ansible -i inventory/sample/inventory.ini all -m debug -a "var=hostvars[inventory_hostname]"):

Command used to invoke ansible:
ansible-playbook --become -e '{"ansible_ssh_common_args": "-o StrictHostKeyChecking=no -o ProxyJump=\"[email protected]""}' -i inventory/$CLUSTER/hosts cluster.yml

Output of ansible run:

[...]
TASK [kubernetes-apps/ansible : Kubernetes Apps | Start Resources] *******************************************************************************************
ok: [carl-k8s-master-nf-1] => (item=coredns-clusterrole.yml)
ok: [carl-k8s-master-nf-1] => (item=coredns-clusterrolebinding.yml)
ok: [carl-k8s-master-nf-1] => (item=coredns-config.yml)
ok: [carl-k8s-master-nf-1] => (item=coredns-deployment.yml)
ok: [carl-k8s-master-nf-1] => (item=coredns-sa.yml)
ok: [carl-k8s-master-nf-1] => (item=coredns-svc.yml)
ok: [carl-k8s-master-nf-1] => (item=dns-autoscaler.yml)
ok: [carl-k8s-master-nf-1] => (item=dns-autoscaler-clusterrole.yml)
ok: [carl-k8s-master-nf-1] => (item=dns-autoscaler-clusterrolebinding.yml)
ok: [carl-k8s-master-nf-1] => (item=dns-autoscaler-sa.yml)
ok: [carl-k8s-master-nf-1] => (item=nodelocaldns-config.yml)
ok: [carl-k8s-master-nf-1] => (item=nodelocaldns-sa.yml)
ok: [carl-k8s-master-nf-1] => (item=nodelocaldns-daemonset.yml)
Wednesday 09 February 2022  16:02:33 +0100 (0:00:07.363)       0:08:28.708 **** 
Wednesday 09 February 2022  16:02:33 +0100 (0:00:00.041)       0:08:28.749 **** 
Wednesday 09 February 2022  16:02:33 +0100 (0:00:00.043)       0:08:28.793 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.043)       0:08:28.836 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.040)       0:08:28.877 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.035)       0:08:28.912 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.037)       0:08:28.950 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.037)       0:08:28.988 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.043)       0:08:29.031 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.033)       0:08:29.065 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.029)       0:08:29.094 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.068)       0:08:29.163 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.071)       0:08:29.234 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.025)       0:08:29.259 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.027)       0:08:29.287 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.025)       0:08:29.312 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.028)       0:08:29.341 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.027)       0:08:29.368 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.030)       0:08:29.399 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.032)       0:08:29.432 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.041)       0:08:29.473 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.037)       0:08:29.511 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.033)       0:08:29.544 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.033)       0:08:29.578 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.028)       0:08:29.607 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.027)       0:08:29.635 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.024)       0:08:29.660 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.035)       0:08:29.695 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.033)       0:08:29.729 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.027)       0:08:29.757 **** 
Wednesday 09 February 2022  16:02:34 +0100 (0:00:00.027)       0:08:29.784 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.034)       0:08:29.818 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.033)       0:08:29.852 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.031)       0:08:29.883 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.035)       0:08:29.919 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.034)       0:08:29.954 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.036)       0:08:29.990 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.024)       0:08:30.015 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.036)       0:08:30.051 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.030)       0:08:30.082 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.034)       0:08:30.117 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.105)       0:08:30.222 **** 
Wednesday 09 February 2022  16:02:35 +0100 (0:00:00.048)       0:08:30.271 **** 

TASK [kubernetes-apps/csi_driver/csi_crd : CSI CRD | Generate Manifests] *************************************************************************************
ok: [carl-k8s-master-nf-1] => (item={'name': 'volumesnapshotclasses', 'file': 'volumesnapshotclasses.yml'})
ok: [carl-k8s-master-nf-1] => (item={'name': 'volumesnapshotcontents', 'file': 'volumesnapshotcontents.yml'})
ok: [carl-k8s-master-nf-1] => (item={'name': 'volumesnapshots', 'file': 'volumesnapshots.yml'})
Wednesday 09 February 2022  16:02:38 +0100 (0:00:02.847)       0:08:33.118 **** 

TASK [kubernetes-apps/csi_driver/csi_crd : CSI CRD | Apply Manifests] ****************************************************************************************
ok: [carl-k8s-master-nf-1] => (item=volumesnapshotclasses.yml)
ok: [carl-k8s-master-nf-1] => (item=volumesnapshotcontents.yml)
ok: [carl-k8s-master-nf-1] => (item=volumesnapshots.yml)
Wednesday 09 February 2022  16:02:40 +0100 (0:00:01.711)       0:08:34.829 **** 
included: /home/$USER/kubespray/roles/kubernetes-apps/csi_driver/cinder/tasks/cinder-credential-check.yml for carl-k8s-master-nf-1
Wednesday 09 February 2022  16:02:40 +0100 (0:00:00.049)       0:08:34.878 **** 
Wednesday 09 February 2022  16:02:40 +0100 (0:00:00.034)       0:08:34.913 **** 
Wednesday 09 February 2022  16:02:40 +0100 (0:00:00.039)       0:08:34.952 **** 
Wednesday 09 February 2022  16:02:40 +0100 (0:00:00.044)       0:08:34.997 **** 
Wednesday 09 February 2022  16:02:40 +0100 (0:00:00.046)       0:08:35.044 **** 
Wednesday 09 February 2022  16:02:40 +0100 (0:00:00.045)       0:08:35.089 **** 
Wednesday 09 February 2022  16:02:40 +0100 (0:00:00.040)       0:08:35.130 **** 
Wednesday 09 February 2022  16:02:40 +0100 (0:00:00.053)       0:08:35.184 **** 
Wednesday 09 February 2022  16:02:40 +0100 (0:00:00.045)       0:08:35.229 **** 
Wednesday 09 February 2022  16:02:40 +0100 (0:00:00.097)       0:08:35.327 **** 

TASK [kubernetes-apps/csi_driver/cinder : Cinder CSI Driver | Write Cinder cloud-config] *********************************************************************
fatal: [carl-k8s-master-nf-1]: FAILED! => {"changed": false, "checksum": "1a9f9bf70737580829ddc93b94b35a32b684a189", "gid": 0, "group": "root", "mode": "0644", "msg": "chgrp failed: failed to look up group kube-cert", "owner": "root", "path": "/etc/kubernetes/cinder_cloud_config", "secontext": "system_u:object_r:kubernetes_file_t:s0", "size": 357, "state": "file", "uid": 0}

Anything else do we need to know:
I set cinder_csi_enabled: true in group_vars/all/openstack.yml

[cristicalin]

We took a different approach for this file for the external cloud controller: #7603, can't we do the same (move the cloud-config into a kubernetes secret) here and not worry about users on the host?

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.