Merge branch 'master' into feature/virtualbox

* master: (21 commits)
  Fix quotation of nerdctl_extra_flags (kubernetes-sigs#8668)
  fix: uninstall contailer engine if service is running (kubernetes-sigs#8662)
  fix disk controller type in Vagrantfile (kubernetes-sigs#8656)
  Add ETCD_EXPERIMENTAL_INITIAL_CORRUPT_CHECK flag to etcd config (kubernetes-sigs#8664)
  Update vagrant.md (kubernetes-sigs#8663)
  Add missing 2.10 ansible test (kubernetes-sigs#8665)
  [ansible] make ansible 5.x the new default version (kubernetes-sigs#8660)
  Run 0100-dhclient-hooks if dhcpclient is enabled (kubernetes-sigs#8658)
  terrform/openstack: Fix templating of ansible_ssh_common_args in no_floating.yml if used as TF module (kubernetes-sigs#8646)
  [ansible] add support for ansible 5 (ansible-core 2.12) (kubernetes-sigs#8512)
  [etcd] add 0 hash for arm v3.5.2 to prevent deployment failures
  Update cert-manager to v1.7.2 (kubernetes-sigs#8648)
  fix vagrant parameter (kubernetes-sigs#8650)
  Remove centos7 molecule while opensuse mirror is flaky
  Remove k8s module for ns creation
  Update kubernetes image to 2.18.1
  Vagrantfile: add var to set ansible verbosity level (kubernetes-sigs#8639)
  Have ingress_controller and external_provisioner in upgrade-cluster.yml (kubernetes-sigs#8640)
  [sysctl] set fs.may_detach_mounts=1 even when CRIs don't set it themselves (kubernetes-sigs#8635)
  Fixed cluster roles for openstack cloud controller (kubernetes-sigs#8638)
  ...

.gitlab-ci.yml

@@ -8,7 +8,7 @@ stages:
   - deploy-special
 
 variables:
-  KUBESPRAY_VERSION: v2.18.0
+  KUBESPRAY_VERSION: v2.18.1
   FAILFASTCI_NAMESPACE: 'kargo-ci'
   GITLAB_REPOSITORY: 'kargo-ci/kubernetes-sigs-kubespray'
   ANSIBLE_FORCE_COLOR: "true"

.gitlab-ci/lint.yml

@@ -52,7 +52,7 @@ tox-inventory-builder:
     - ./tests/scripts/rebase.sh
     - apt-get update && apt-get install -y python3-pip
     - update-alternatives --install /usr/bin/python python /usr/bin/python3 10
-    - python -m pip uninstall -y ansible
+    - python -m pip uninstall -y ansible ansible-base ansible-core
     - python -m pip install -r tests/requirements.txt
   script:
     - pip3 install tox

.gitlab-ci/packet.yml

@@ -31,18 +31,26 @@ packet_ubuntu20-calico-aio:
   variables:
     RESET_CHECK: "true"
 
-# Exericse ansible variants
+# Exericse ansible variants during the nightly jobs
 packet_ubuntu20-calico-aio-ansible-2_9:
   stage: deploy-part1
-  extends: .packet_pr
+  extends: .packet_periodic
   when: on_success
   variables:
     ANSIBLE_MAJOR_VERSION: "2.9"
     RESET_CHECK: "true"
 
+packet_ubuntu20-calico-aio-ansible-2_10:
+  stage: deploy-part1
+  extends: .packet_periodic
+  when: on_success
+  variables:
+    ANSIBLE_MAJOR_VERSION: "2.10"
+    RESET_CHECK: "true"
+
 packet_ubuntu20-calico-aio-ansible-2_11:
   stage: deploy-part1
-  extends: .packet_pr
+  extends: .packet_periodic
   when: on_success
   variables:
     ANSIBLE_MAJOR_VERSION: "2.11"

.gitlab-ci/vagrant.yml

@@ -11,7 +11,7 @@ molecule_tests:
     - tests/scripts/rebase.sh
     - apt-get update && apt-get install -y python3-pip
     - update-alternatives --install /usr/bin/python python /usr/bin/python3 10
-    - python -m pip uninstall -y ansible
+    - python -m pip uninstall -y ansible ansible-base ansible-core
     - python -m pip install -r tests/requirements.txt
     - ./tests/scripts/vagrant_clean.sh
   script:
@@ -38,7 +38,7 @@ molecule_tests:
   before_script:
     - apt-get update && apt-get install -y python3-pip
     - update-alternatives --install /usr/bin/python python /usr/bin/python3 10
-    - python -m pip uninstall -y ansible
+    - python -m pip uninstall -y ansible ansible-base ansible-core
     - python -m pip install -r tests/requirements.txt
     - ./tests/scripts/vagrant_clean.sh
   script:

README.md

@@ -57,10 +57,10 @@ A simple way to ensure you get all the correct version of Ansible is to use the
 You will then need to use [bind mounts](https://docs.docker.com/storage/bind-mounts/) to get the inventory and ssh key into the container, like this:
 
 ```ShellSession
-docker pull quay.io/kubespray/kubespray:v2.18.0
+docker pull quay.io/kubespray/kubespray:v2.18.1
 docker run --rm -it --mount type=bind,source="$(pwd)"/inventory/sample,dst=/inventory \
   --mount type=bind,source="${HOME}"/.ssh/id_rsa,dst=/root/.ssh/id_rsa \
-  quay.io/kubespray/kubespray:v2.18.0 bash
+  quay.io/kubespray/kubespray:v2.18.1 bash
 # Inside the container you may now run the kubespray playbooks:
 ansible-playbook -i /inventory/inventory.ini --private-key /root/.ssh/id_rsa cluster.yml
 ```
@@ -161,7 +161,7 @@ Note: Upstart/SysV init based OS types are not supported.
 ## Requirements
 
 - **Minimum required version of Kubernetes is v1.21**
-- **Ansible v2.9.x, Jinja 2.11+ and python-netaddr is installed on the machine that will run Ansible commands, Ansible 2.10.x is experimentally supported for now**
+- **Ansible v2.9.x, Jinja 2.11+ and python-netaddr is installed on the machine that will run Ansible commands**
 - The target servers must have **access to the Internet** in order to pull docker images. Otherwise, additional configuration is required (See [Offline Environment](docs/offline-environment.md))
 - The target servers are configured to allow **IPv4 forwarding**.
 - If using IPv6 for pods and services, the target servers are configured to allow **IPv6 forwarding**.

Vagrantfile

@@ -55,7 +55,7 @@ $subnet_ipv6 ||= "fd3c:b398:0698:0756"
 $os ||= "ubuntu1804"
 $network_plugin ||= "flannel"
 # Setting multi_networking to true will install Multus: https://github.com/intel/multus-cni
-$multi_networking ||= false
+$multi_networking ||= "False"
 $download_run_once ||= "True"
 $download_force_cache ||= "False"
 # The first three nodes are etcd servers
@@ -70,9 +70,11 @@ $kube_node_instances_with_disks_size ||= "20G"
 $kube_node_instances_with_disks_number ||= 2
 $override_disk_size ||= false
 $disk_size ||= "20GB"
-$local_path_provisioner_enabled ||= false
+$local_path_provisioner_enabled ||= "False"
 $local_path_provisioner_claim_root ||= "/opt/local-path-provisioner/"
 $libvirt_nested ||= false
+# boolean or string (e.g. "-vvv")
+$ansible_verbosity ||= false
 
 $playbook ||= "cluster.yml"
 
@@ -169,7 +171,7 @@ Vagrant.configure("2") do |config|
           # always make /dev/sd{a/b/c} so that CI can ensure that
           # virtualbox and libvirt will have the same devices to use for OSDs
           (1..$kube_node_instances_with_disks_number).each do |d|
-            lv.storage :file, :device => "hd#{driverletters[d]}", :path => "disk-#{i}-#{d}-#{DISK_UUID}.disk", :size => $kube_node_instances_with_disks_size, :bus => "ide"
+            lv.storage :file, :device => "hd#{driverletters[d]}", :path => "disk-#{i}-#{d}-#{DISK_UUID}.disk", :size => $kube_node_instances_with_disks_size, :bus => "scsi"
           end
         end
       end
@@ -244,6 +246,7 @@ Vagrant.configure("2") do |config|
       if i == $num_instances
         node.vm.provision "ansible" do |ansible|
           ansible.playbook = $playbook
+          ansible.verbose = $ansible_verbosity
           $ansible_inventory_path = File.join( $inventory, "hosts.ini")
           if File.exist?($ansible_inventory_path)
             ansible.inventory_path = $ansible_inventory_path

ansible_version.yml

@@ -5,7 +5,7 @@
   vars:
     minimal_ansible_version: 2.9.0
     minimal_ansible_version_2_10: 2.10.11
-    maximal_ansible_version: 2.12.0
+    maximal_ansible_version: 2.13.0
     ansible_connection: local
   tags: always
   tasks:

contrib/terraform/openstack/modules/compute/main.tf

@@ -228,7 +228,7 @@ resource "openstack_compute_instance_v2" "bastion" {
   }
 
   provisioner "local-exec" {
-    command = "sed s/USER/${var.ssh_user}/ ${path.root}/ansible_bastion_template.txt | sed s/BASTION_ADDRESS/${var.bastion_fips[0]}/ > ${var.group_vars_path}/no_floating.yml"
+    command = "sed -e s/USER/${var.ssh_user}/ -e s/BASTION_ADDRESS/${var.bastion_fips[0]}/ ${path.module}/ansible_bastion_template.txt > ${var.group_vars_path}/no_floating.yml"
   }
 }
 
@@ -284,7 +284,7 @@ resource "openstack_compute_instance_v2" "k8s_master" {
   }
 
   provisioner "local-exec" {
-    command = "sed s/USER/${var.ssh_user}/ ${path.root}/ansible_bastion_template.txt | sed s/BASTION_ADDRESS/${element(concat(var.bastion_fips, var.k8s_master_fips), 0)}/ > ${var.group_vars_path}/no_floating.yml"
+    command = "sed -e s/USER/${var.ssh_user}/ -e s/BASTION_ADDRESS/${element(concat(var.bastion_fips, var.k8s_master_fips), 0)}/ ${path.module}/ansible_bastion_template.txt > ${var.group_vars_path}/no_floating.yml"
   }
 }
 
@@ -340,7 +340,7 @@ resource "openstack_compute_instance_v2" "k8s_master_no_etcd" {
   }
 
   provisioner "local-exec" {
-    command = "sed s/USER/${var.ssh_user}/ ${path.root}/ansible_bastion_template.txt | sed s/BASTION_ADDRESS/${element(concat(var.bastion_fips, var.k8s_master_fips), 0)}/ > ${var.group_vars_path}/no_floating.yml"
+    command = "sed -e s/USER/${var.ssh_user}/ -e s/BASTION_ADDRESS/${element(concat(var.bastion_fips, var.k8s_master_fips), 0)}/ ${path.module}/ansible_bastion_template.txt > ${var.group_vars_path}/no_floating.yml"
   }
 }
 
@@ -547,7 +547,7 @@ resource "openstack_compute_instance_v2" "k8s_node" {
   }
 
   provisioner "local-exec" {
-    command = "sed s/USER/${var.ssh_user}/ ${path.root}/ansible_bastion_template.txt | sed s/BASTION_ADDRESS/${element(concat(var.bastion_fips, var.k8s_node_fips), 0)}/ > ${var.group_vars_path}/no_floating.yml"
+    command = "sed -e s/USER/${var.ssh_user}/ -e s/BASTION_ADDRESS/${element(concat(var.bastion_fips, var.k8s_node_fips), 0)}/ ${path.module}/ansible_bastion_template.txt > ${var.group_vars_path}/no_floating.yml"
   }
 }
 
@@ -653,7 +653,7 @@ resource "openstack_compute_instance_v2" "k8s_nodes" {
   }
 
   provisioner "local-exec" {
-    command = "%{if each.value.floating_ip}sed s/USER/${var.ssh_user}/ ${path.root}/ansible_bastion_template.txt | sed s/BASTION_ADDRESS/${element(concat(var.bastion_fips, [for key, value in var.k8s_nodes_fips : value.address]), 0)}/ > ${var.group_vars_path}/no_floating.yml%{else}true%{endif}"
+    command = "%{if each.value.floating_ip}sed -e s/USER/${var.ssh_user}/ -e s/BASTION_ADDRESS/${element(concat(var.bastion_fips, [for key, value in var.k8s_nodes_fips : value.address]), 0)}/ ${path.module}/ansible_bastion_template.txt > ${var.group_vars_path}/no_floating.yml%{else}true%{endif}"
   }
 }
 

docs/ansible.md

@@ -267,7 +267,7 @@ that explains in detail the need and the evolution plan.
 You first need to uninstall your old ansible (pre 2.10) version and install the new one.
 
 ```ShellSession
-pip uninstall ansible
+pip uninstall ansible ansible-base ansible-core
 cd kubespray/
 pip install -U .
 ```

docs/vagrant.md

@@ -1,34 +1,64 @@
 # Vagrant
 
-Assuming you have Vagrant 2.0+ installed with virtualbox, libvirt/qemu or vmware, but is untested) you should be able to launch a 3 node Kubernetes cluster by simply running `vagrant up`. This will spin up 3 VMs and install kubernetes on them.  Once they are completed you can connect to any of them by running `vagrant ssh k8s-[1..3]`.
-
-To give an estimate of the expected duration of a provisioning run: On a dual core i5-6300u laptop with an SSD, provisioning takes around 13 to 15 minutes, once the container images and other files are cached. Note that libvirt/qemu is recommended over virtualbox as it is quite a bit faster, especially during boot-up time.
-
-For proper performance a minimum of 12GB RAM is recommended. It is possible to run a 3 node cluster on a laptop with 8GB of RAM using the default Vagrantfile, provided you have 8GB zram swap configured and not much more than a browser and a mail client running. If you decide to run on such a machine, then also make sure that any tmpfs devices, that are mounted, are mostly empty and disable any swapfiles mounted on HDD/SSD or you will be in for some serious swap-madness. Things can get a bit sluggish during provisioning, but when that's done, the system will actually be able to perform quite well.
+Assuming you have Vagrant 2.0+ installed with virtualbox, libvirt/qemu or
+vmware, but is untested) you should be able to launch a 3 node Kubernetes
+cluster by simply running `vagrant up`.
+
+This will spin up 3 VMs and install kubernetes on them.
+Once they are completed you can connect to any of them by running `vagrant ssh k8s-[1..3]`.
+
+To give an estimate of the expected duration of a provisioning run:
+On a dual core i5-6300u laptop with an SSD, provisioning takes around 13
+to 15 minutes, once the container images and other files are cached.
+Note that libvirt/qemu is recommended over virtualbox as it is quite a bit
+faster, especially during boot-up time.
+
+For proper performance a minimum of 12GB RAM is recommended.
+It is possible to run a 3 node cluster on a laptop with 8GB of RAM using
+the default Vagrantfile, provided you have 8GB zram swap configured and
+not much more than a browser and a mail client running.
+If you decide to run on such a machine, then also make sure that any tmpfs
+devices, that are mounted, are mostly empty and disable any swapfiles
+mounted on HDD/SSD or you will be in for some serious swap-madness.
+Things can get a bit sluggish during provisioning, but when that's done,
+the system will actually be able to perform quite well.
 
 ## Customize Vagrant
 
-You can override the default settings in the `Vagrantfile` either by directly modifying the `Vagrantfile` or through an override file. In the same directory as the `Vagrantfile`, create a folder called `vagrant` and create `config.rb` file in it. An example of how to configure this file is given below.
+You can override the default settings in the `Vagrantfile` either by
+directly modifying the `Vagrantfile` or through an override file.
+In the same directory as the `Vagrantfile`, create a folder called
+`vagrant` and create `config.rb` file in it.
+An example of how to configure this file is given below.
 
 ## Use alternative OS for Vagrant
 
-By default, Vagrant uses Ubuntu 18.04 box to provision a local cluster. You may use an alternative supported operating system for your local cluster.
+By default, Vagrant uses Ubuntu 18.04 box to provision a local cluster.
+You may use an alternative supported operating system for your local cluster.
 
 Customize `$os` variable in `Vagrantfile` or as override, e.g.,:
 
 ```ShellSession
 echo '$os = "flatcar-stable"' >> vagrant/config.rb
 ```
 
-The supported operating systems for vagrant are defined in the `SUPPORTED_OS` constant in the `Vagrantfile`.
+The supported operating systems for vagrant are defined in the `SUPPORTED_OS`
+constant in the `Vagrantfile`.
 
 ## File and image caching
 
-Kubespray can take quite a while to start on a laptop. To improve provisioning speed, the variable 'download_run_once' is set. This will make kubespray download all files and containers just once and then redistributes them to the other nodes and as a bonus, also cache all downloads locally and re-use them on the next provisioning run. For more information on download settings see [download documentation](/docs/downloads.md).
+Kubespray can take quite a while to start on a laptop. To improve provisioning
+speed, the variable 'download_run_once' is set. This will make kubespray
+download all files and containers just once and then redistributes them to
+the other nodes and as a bonus, also cache all downloads locally and re-use
+them on the next provisioning run. For more information on download settings
+see [download documentation](/docs/downloads.md).
 
 ## Example use of Vagrant
 
-The following is an example of setting up and running kubespray using `vagrant`. For repeated runs, you could save the script to a file in the root of the kubespray and run it by executing 'source <name_of_the_file>.
+The following is an example of setting up and running kubespray using `vagrant`.
+For repeated runs, you could save the script to a file in the root of the
+kubespray and run it by executing 'source <name_of_the_file>.
 
 ```ShellSession
 # use virtualenv to install all python requirements
@@ -74,7 +104,8 @@ sudo ln -s $PWD/$INV/artifacts/kubectl /usr/local/bin/kubectl
 export PATH=$PATH:$PWD/$INV/artifacts
 ```
 
-If a vagrant run failed and you've made some changes to fix the issue causing the fail, here is how you would re-run ansible:
+If a vagrant run failed and you've made some changes to fix the issue causing
+the fail, here is how you would re-run ansible:
 
 ```ShellSession
 ansible-playbook -vvv -i .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory cluster.yml

requirements-2.12.txt

@@ -0,0 +1,10 @@
+ansible==5.5.0
+ansible-core==2.12.3
+cryptography==2.8
+jinja2==2.11.3
+netaddr==0.7.19
+pbr==5.4.4
+jmespath==0.9.5
+ruamel.yaml==0.16.10
+ruamel.yaml.clib==0.2.6
+MarkupSafe==1.1.1

requirements.txt

@@ -1 +1 @@
-requirements-2.10.txt
+requirements-2.12.txt

roles/container-engine/cri-o/molecule/default/molecule.yml

@@ -13,12 +13,6 @@ platforms:
     memory: 1024
     groups:
       - kube_control_plane
-  - name: centos7
-    box: centos/7
-    cpus: 2
-    memory: 1024
-    groups:
-      - kube_control_plane
   - name: almalinux8
     box: almalinux/8
     cpus: 2