Merge branch 'kubernetes-sigs:master' into master

.gitattributes

@@ -0,0 +1 @@
+docs/_sidebar.md linguist-generated=true

.github/dependabot.yml

@@ -3,5 +3,5 @@ updates:
   - package-ecosystem: "pip"
     directory: "/"
     schedule:
-      interval: "daily"
+      interval: "weekly"
     labels: [ "dependencies" ]

.gitlab-ci/packet.yml

@@ -61,7 +61,17 @@ packet_ubuntu22-calico-all-in-one:
   extends: .packet_pr
   when: on_success
 
-packet_ubuntu22-calico-etcd-datastore:
+packet_ubuntu24-all-in-one-docker:
+  stage: deploy-part2
+  extends: .packet_pr
+  when: on_success
+
+packet_ubuntu24-calico-all-in-one:
+  stage: deploy-part2
+  extends: .packet_pr
+  when: on_success
+
+packet_ubuntu24-calico-etcd-datastore:
   stage: deploy-part2
   extends: .packet_pr
   when: on_success
@@ -174,6 +184,11 @@ packet_almalinux8-docker:
   extends: .packet_pr
   when: on_success
 
+packet_amazon-linux-2-all-in-one:
+  stage: deploy-part2
+  extends: .packet_pr
+  when: on_success
+
 packet_fedora38-docker-weave:
   stage: deploy-part2
   extends: .packet_pr
@@ -183,7 +198,7 @@ packet_fedora38-docker-weave:
 packet_opensuse-docker-cilium:
   stage: deploy-part2
   extends: .packet_pr
-  when: manual
+  when: on_success
 
 # ### MANUAL JOBS
 
@@ -240,11 +255,6 @@ packet_fedora37-calico-swap-selinux:
   extends: .packet_pr
   when: manual
 
-packet_amazon-linux-2-all-in-one:
-  stage: deploy-part2
-  extends: .packet_pr
-  when: manual
-
 packet_almalinux8-calico-nodelocaldns-secondary:
   stage: deploy-part2
   extends: .packet_pr

.pre-commit-config.yaml

@@ -1,6 +1,5 @@
 ---
 repos:
-
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v3.4.0
     hooks:
@@ -25,14 +24,14 @@ repos:
     rev: v0.11.0
     hooks:
       - id: markdownlint
-        args: [ -r, "~MD013,~MD029" ]
+        args: [-r, "~MD013,~MD029"]
         exclude: "^.git"
 
   - repo: https://github.com/jumanjihouse/pre-commit-hooks
     rev: 3.0.0
     hooks:
       - id: shellcheck
-        args: [ --severity, "error" ]
+        args: [--severity, "error"]
         exclude: "^.git"
         files: "\\.sh$"
 
@@ -64,6 +63,12 @@ repos:
         language: script
         pass_filenames: false
 
+      - id: generate-docs-sidebar
+        name: generate-docs-sidebar
+        entry: scripts/gen_docs_sidebar.sh
+        language: script
+        pass_filenames: false
+
       - id: ci-matrix
         name: ci-matrix
         entry: tests/scripts/md-table/test.sh

README.md

@@ -5,7 +5,7 @@
 If you have questions, check the documentation at [kubespray.io](https://kubespray.io) and join us on the [kubernetes slack](https://kubernetes.slack.com), channel **\#kubespray**.
 You can get your invite [here](http://slack.k8s.io/)
 
-- Can be deployed on **[AWS](docs/aws.md), GCE, [Azure](docs/azure.md), [OpenStack](docs/openstack.md), [vSphere](docs/vsphere.md), [Equinix Metal](docs/equinix-metal.md) (bare metal), Oracle Cloud Infrastructure (Experimental), or Baremetal**
+- Can be deployed on **[AWS](docs/cloud_providers/aws.md), GCE, [Azure](docs/cloud_providers/azure.md), [OpenStack](docs/cloud_providers/openstack.md), [vSphere](docs/cloud_providers/vsphere.md), [Equinix Metal](docs/cloud_providers/equinix-metal.md) (bare metal), Oracle Cloud Infrastructure (Experimental), or Baremetal**
 - **Highly available** cluster
 - **Composable** (Choice of the network plugin for instance)
 - Supports most popular **Linux distributions**
@@ -19,7 +19,7 @@ Below are several ways to use Kubespray to deploy a Kubernetes cluster.
 
 #### Usage
 
-Install Ansible according to [Ansible installation guide](/docs/ansible.md#installing-ansible)
+Install Ansible according to [Ansible installation guide](/docs/ansible/ansible.md#installing-ansible)
 then run the following steps:
 
 ```ShellSession
@@ -86,7 +86,7 @@ ansible-playbook -i /inventory/inventory.ini --private-key /root/.ssh/id_rsa clu
 
 #### Collection
 
-See [here](docs/ansible_collection.md) if you wish to use this repository as an Ansible collection
+See [here](docs/ansible/ansible_collection.md) if you wish to use this repository as an Ansible collection
 
 ### Vagrant
 
@@ -99,7 +99,7 @@ python -V && pip -V
 
 If this returns the version of the software, you're good to go. If not, download and install Python from here <https://www.python.org/downloads/source/>
 
-Install Ansible according to [Ansible installation guide](/docs/ansible.md#installing-ansible)
+Install Ansible according to [Ansible installation guide](/docs/ansible/ansible.md#installing-ansible)
 then run the following step:
 
 ```ShellSession
@@ -109,62 +109,62 @@ vagrant up
 ## Documents
 
 - [Requirements](#requirements)
-- [Kubespray vs ...](docs/comparisons.md)
-- [Getting started](docs/getting-started.md)
-- [Setting up your first cluster](docs/setting-up-your-first-cluster.md)
-- [Ansible inventory and tags](docs/ansible.md)
-- [Integration with existing ansible repo](docs/integration.md)
-- [Deployment data variables](docs/vars.md)
-- [DNS stack](docs/dns-stack.md)
-- [HA mode](docs/ha-mode.md)
+- [Kubespray vs ...](docs/getting_started/comparisons.md)
+- [Getting started](docs/getting_started/getting-started.md)
+- [Setting up your first cluster](docs/getting_started/setting-up-your-first-cluster.md)
+- [Ansible inventory and tags](docs/ansible/ansible.md)
+- [Integration with existing ansible repo](docs/operations/integration.md)
+- [Deployment data variables](docs/ansible/vars.md)
+- [DNS stack](docs/advanced/dns-stack.md)
+- [HA mode](docs/operations/ha-mode.md)
 - [Network plugins](#network-plugins)
-- [Vagrant install](docs/vagrant.md)
-- [Flatcar Container Linux bootstrap](docs/flatcar.md)
-- [Fedora CoreOS bootstrap](docs/fcos.md)
-- [openSUSE setup](docs/opensuse.md)
-- [Downloaded artifacts](docs/downloads.md)
-- [Cloud providers](docs/cloud.md)
-- [OpenStack](docs/openstack.md)
-- [AWS](docs/aws.md)
-- [Azure](docs/azure.md)
-- [vSphere](docs/vsphere.md)
-- [Equinix Metal](docs/equinix-metal.md)
-- [Large deployments](docs/large-deployments.md)
-- [Adding/replacing a node](docs/nodes.md)
-- [Upgrades basics](docs/upgrades.md)
-- [Air-Gap installation](docs/offline-environment.md)
-- [NTP](docs/ntp.md)
-- [Hardening](docs/hardening.md)
-- [Mirror](docs/mirror.md)
-- [Roadmap](docs/roadmap.md)
+- [Vagrant install](docs/developers/vagrant.md)
+- [Flatcar Container Linux bootstrap](docs/operating_systems/flatcar.md)
+- [Fedora CoreOS bootstrap](docs/operating_systems/fcos.md)
+- [openSUSE setup](docs/operating_systems/opensuse.md)
+- [Downloaded artifacts](docs/advanced/downloads.md)
+- [Cloud providers](docs/cloud_providers/cloud.md)
+- [OpenStack](docs/cloud_providers/openstack.md)
+- [AWS](docs/cloud_providers/aws.md)
+- [Azure](docs/cloud_providers/azure.md)
+- [vSphere](docs/cloud_providers/vsphere.md)
+- [Equinix Metal](docs/cloud_providers/equinix-metal.md)
+- [Large deployments](docs/operations/large-deployments.md)
+- [Adding/replacing a node](docs/operations/nodes.md)
+- [Upgrades basics](docs/operations/upgrades.md)
+- [Air-Gap installation](docs/operations/offline-environment.md)
+- [NTP](docs/advanced/ntp.md)
+- [Hardening](docs/operations/hardening.md)
+- [Mirror](docs/operations/mirror.md)
+- [Roadmap](docs/roadmap/roadmap.md)
 
 ## Supported Linux Distributions
 
 - **Flatcar Container Linux by Kinvolk**
 - **Debian** Bookworm, Bullseye, Buster
 - **Ubuntu** 20.04, 22.04
-- **CentOS/RHEL** 7, [8, 9](docs/centos.md#centos-8)
+- **CentOS/RHEL** 7, [8, 9](docs/operating_systems/centos.md#centos-8)
 - **Fedora** 37, 38
-- **Fedora CoreOS** (see [fcos Note](docs/fcos.md))
+- **Fedora CoreOS** (see [fcos Note](docs/operating_systems/fcos.md))
 - **openSUSE** Leap 15.x/Tumbleweed
-- **Oracle Linux** 7, [8, 9](docs/centos.md#centos-8)
-- **Alma Linux** [8, 9](docs/centos.md#centos-8)
-- **Rocky Linux** [8, 9](docs/centos.md#centos-8)
-- **Kylin Linux Advanced Server V10** (experimental: see [kylin linux notes](docs/kylinlinux.md))
-- **Amazon Linux 2** (experimental: see [amazon linux notes](docs/amazonlinux.md))
-- **UOS Linux** (experimental: see [uos linux notes](docs/uoslinux.md))
-- **openEuler** (experimental: see [openEuler notes](docs/openeuler.md))
+- **Oracle Linux** 7, [8, 9](docs/operating_systems/centos.md#centos-8)
+- **Alma Linux** [8, 9](docs/operating_systems/centos.md#centos-8)
+- **Rocky Linux** [8, 9](docs/operating_systems/centos.md#centos-8)
+- **Kylin Linux Advanced Server V10** (experimental: see [kylin linux notes](docs/operating_systems/kylinlinux.md))
+- **Amazon Linux 2** (experimental: see [amazon linux notes](docs/operating_systems/amazonlinux.md))
+- **UOS Linux** (experimental: see [uos linux notes](docs/operating_systems/uoslinux.md))
+- **openEuler** (experimental: see [openEuler notes](docs/operating_systems/openeuler.md))
 
 Note: Upstart/SysV init based OS types are not supported.
 
 ## Supported Components
 
 - Core
-  - [kubernetes](https://github.com/kubernetes/kubernetes) v1.29.3
+  - [kubernetes](https://github.com/kubernetes/kubernetes) v1.29.5
   - [etcd](https://github.com/etcd-io/etcd) v3.5.12
   - [docker](https://www.docker.com/) v24.0 (see [Note](#container-runtime-notes))
   - [containerd](https://containerd.io/) v1.7.16
-  - [cri-o](http://cri-o.io/) v1.29.1 (experimental: see [CRI-O Note](docs/cri-o.md). Only on fedora, ubuntu and centos based OS)
+  - [cri-o](http://cri-o.io/) v1.29.1 (experimental: see [CRI-O Note](docs/CRI/cri-o.md). Only on fedora, ubuntu and centos based OS)
 - Network Plugin
   - [cni-plugins](https://github.com/containernetworking/plugins) v1.2.0
   - [calico](https://github.com/projectcalico/calico) v3.27.3
@@ -178,9 +178,9 @@ Note: Upstart/SysV init based OS types are not supported.
 - Application
   - [cert-manager](https://github.com/jetstack/cert-manager) v1.13.2
   - [coredns](https://github.com/coredns/coredns) v1.11.1
-  - [ingress-nginx](https://github.com/kubernetes/ingress-nginx) v1.9.6
+  - [ingress-nginx](https://github.com/kubernetes/ingress-nginx) v1.10.1
   - [krew](https://github.com/kubernetes-sigs/krew) v0.4.4
-  - [argocd](https://argoproj.github.io/) v2.8.4
+  - [argocd](https://argoproj.github.io/) v2.11.0
   - [helm](https://helm.sh/) v3.14.2
   - [metallb](https://metallb.universe.tf/)  v0.13.9
   - [registry](https://github.com/distribution/distribution) v2.8.1
@@ -204,7 +204,7 @@ Note: Upstart/SysV init based OS types are not supported.
 
 - **Minimum required version of Kubernetes is v1.27**
 - **Ansible v2.14+, Jinja 2.11+ and python-netaddr is installed on the machine that will run Ansible commands**
-- The target servers must have **access to the Internet** in order to pull docker images. Otherwise, additional configuration is required (See [Offline Environment](docs/offline-environment.md))
+- The target servers must have **access to the Internet** in order to pull docker images. Otherwise, additional configuration is required (See [Offline Environment](docs/operations/offline-environment.md))
 - The target servers are configured to allow **IPv4 forwarding**.
 - If using IPv6 for pods and services, the target servers are configured to allow **IPv6 forwarding**.
 - The **firewalls are not managed**, you'll need to implement your own rules the way you used to.
@@ -225,7 +225,7 @@ These limits are safeguarded by Kubespray. Actual requirements for your workload
 
 You can choose among ten network plugins. (default: `calico`, except Vagrant uses `flannel`)
 
-- [flannel](docs/flannel.md): gre/vxlan (layer 2) networking.
+- [flannel](docs/CNI/flannel.md): gre/vxlan (layer 2) networking.
 
 - [Calico](https://docs.tigera.io/calico/latest/about/) is a networking and network policy provider. Calico supports a flexible set of networking options
     designed to give you the most efficient networking across a range of situations, including non-overlay
@@ -234,32 +234,32 @@ You can choose among ten network plugins. (default: `calico`, except Vagrant use
 
 - [cilium](http://docs.cilium.io/en/latest/): layer 3/4 networking (as well as layer 7 to protect and secure application protocols), supports dynamic insertion of BPF bytecode into the Linux kernel to implement security services, networking and visibility logic.
 
-- [weave](docs/weave.md): Weave is a lightweight container overlay network that doesn't require an external K/V database cluster.
+- [weave](docs/CNI/weave.md): Weave is a lightweight container overlay network that doesn't require an external K/V database cluster.
     (Please refer to `weave` [troubleshooting documentation](https://www.weave.works/docs/net/latest/troubleshooting/)).
 
-- [kube-ovn](docs/kube-ovn.md): Kube-OVN integrates the OVN-based Network Virtualization with Kubernetes. It offers an advanced Container Network Fabric for Enterprises.
+- [kube-ovn](docs/CNI/kube-ovn.md): Kube-OVN integrates the OVN-based Network Virtualization with Kubernetes. It offers an advanced Container Network Fabric for Enterprises.
 
-- [kube-router](docs/kube-router.md): Kube-router is a L3 CNI for Kubernetes networking aiming to provide operational
+- [kube-router](docs/CNI/kube-router.md): Kube-router is a L3 CNI for Kubernetes networking aiming to provide operational
     simplicity and high performance: it uses IPVS to provide Kube Services Proxy (if setup to replace kube-proxy),
     iptables for network policies, and BGP for ods L3 networking (with optionally BGP peering with out-of-cluster BGP peers).
     It can also optionally advertise routes to Kubernetes cluster Pods CIDRs, ClusterIPs, ExternalIPs and LoadBalancerIPs.
 
-- [macvlan](docs/macvlan.md): Macvlan is a Linux network driver. Pods have their own unique Mac and Ip address, connected directly the physical (layer 2) network.
+- [macvlan](docs/CNI/macvlan.md): Macvlan is a Linux network driver. Pods have their own unique Mac and Ip address, connected directly the physical (layer 2) network.
 
-- [multus](docs/multus.md): Multus is a meta CNI plugin that provides multiple network interface support to pods. For each interface Multus delegates CNI calls to secondary CNI plugins such as Calico, macvlan, etc.
+- [multus](docs/CNI/multus.md): Multus is a meta CNI plugin that provides multiple network interface support to pods. For each interface Multus delegates CNI calls to secondary CNI plugins such as Calico, macvlan, etc.
 
 - [custom_cni](roles/network-plugin/custom_cni/) : You can specify some manifests that will be applied to the clusters to bring you own CNI and use non-supported ones by Kubespray.
   See `tests/files/custom_cni/README.md` and `tests/files/custom_cni/values.yaml`for an example with a CNI provided by a Helm Chart.
 
 The network plugin to use is defined by the variable `kube_network_plugin`. There is also an
 option to leverage built-in cloud provider networking instead.
-See also [Network checker](docs/netcheck.md).
+See also [Network checker](docs/advanced/netcheck.md).
 
 ## Ingress Plugins
 
 - [nginx](https://kubernetes.github.io/ingress-nginx): the NGINX Ingress Controller.
 
-- [metallb](docs/metallb.md): the MetalLB bare-metal service LoadBalancer provider.
+- [metallb](docs/ingress/metallb.md): the MetalLB bare-metal service LoadBalancer provider.
 
 ## Community docs and resources
 
@@ -280,4 +280,4 @@ See also [Network checker](docs/netcheck.md).
 
 CI/end-to-end tests sponsored by: [CNCF](https://cncf.io), [Equinix Metal](https://metal.equinix.com/), [OVHcloud](https://www.ovhcloud.com/), [ELASTX](https://elastx.se/).
 
-See the [test matrix](docs/test_cases.md) for details.
+See the [test matrix](docs/developers/test_cases.md) for details.

Vagrantfile

@@ -21,13 +21,15 @@ SUPPORTED_OS = {
   "flatcar-edge"        => {box: "flatcar-edge",               user: "core", box_url: FLATCAR_URL_TEMPLATE % ["edge"]},
   "ubuntu2004"          => {box: "generic/ubuntu2004",         user: "vagrant"},
   "ubuntu2204"          => {box: "generic/ubuntu2204",         user: "vagrant"},
+  "ubuntu2404"          => {box: "bento/ubuntu-24.04",         user: "vagrant"},
   "centos"              => {box: "centos/7",                   user: "vagrant"},
   "centos-bento"        => {box: "bento/centos-7.6",           user: "vagrant"},
   "centos8"             => {box: "centos/8",                   user: "vagrant"},
   "centos8-bento"       => {box: "bento/centos-8",             user: "vagrant"},
   "almalinux8"          => {box: "almalinux/8",                user: "vagrant"},
   "almalinux8-bento"    => {box: "bento/almalinux-8",          user: "vagrant"},
-  "rockylinux8"         => {box: "generic/rocky8",             user: "vagrant"},
+  "rockylinux8"         => {box: "rockylinux/8",               user: "vagrant"},
+  "rockylinux9"         => {box: "rockylinux/9",               user: "vagrant"},
   "fedora37"            => {box: "fedora/37-cloud-base",       user: "vagrant"},
   "fedora38"            => {box: "fedora/38-cloud-base",       user: "vagrant"},
   "opensuse"            => {box: "opensuse/Leap-15.4.x86_64",  user: "vagrant"},
@@ -36,6 +38,8 @@ SUPPORTED_OS = {
   "oraclelinux8"        => {box: "generic/oracle8",            user: "vagrant"},
   "rhel7"               => {box: "generic/rhel7",              user: "vagrant"},
   "rhel8"               => {box: "generic/rhel8",              user: "vagrant"},
+  "debian11"            => {box: "debian/bullseye64",          user: "vagrant"},
+  "debian12"            => {box: "debian/bookworm64",          user: "vagrant"},
 }
 
 if File.exist?(CONFIG)
@@ -185,6 +189,14 @@ Vagrant.configure("2") do |config|
             lv.storage :file, :device => "hd#{driverletters[d]}", :path => "disk-#{i}-#{d}-#{DISK_UUID}.disk", :size => $kube_node_instances_with_disks_size, :bus => "scsi"
           end
         end
+        node.vm.provider :virtualbox do |vb|
+          # always make /dev/sd{a/b/c} so that CI can ensure that
+          # virtualbox and libvirt will have the same devices to use for OSDs
+          (1..$kube_node_instances_with_disks_number).each do |d|
+            vb.customize ['createhd', '--filename', "disk-#{i}-#{driverletters[d]}-#{DISK_UUID}.disk", '--size', $kube_node_instances_with_disks_size] # 10GB disk
+            vb.customize ['storageattach', :id, '--storagectl', 'SATA Controller', '--port', d, '--device', 0, '--type', 'hdd', '--medium', "disk-#{i}-#{driverletters[d]}-#{DISK_UUID}.disk", '--nonrotational', 'on', '--mtype', 'normal']
+          end
+        end
       end
 
       if $expose_docker_tcp
@@ -235,6 +247,13 @@ Vagrant.configure("2") do |config|
         SHELL
       end
 
+      # Rockylinux boxes needs UEFI
+      if ["rockylinux8", "rockylinux9"].include? $os
+        config.vm.provider "libvirt" do |domain|
+          domain.loader = "/usr/share/OVMF/x64/OVMF_CODE.fd"
+        end
+      end
+
       # Disable firewalld on oraclelinux/redhat vms
       if ["oraclelinux","oraclelinux8","rhel7","rhel8","rockylinux8"].include? $os
         node.vm.provision "shell", inline: "systemctl stop firewalld; systemctl disable firewalld"

contrib/azurerm/README.md

@@ -49,7 +49,7 @@ If you need to delete all resources from a resource group, simply call:
 
 ## Installing Ansible and the dependencies
 
-Install Ansible according to [Ansible installation guide](/docs/ansible.md#installing-ansible)
+Install Ansible according to [Ansible installation guide](/docs/ansible/ansible.md#installing-ansible)
 
 ## Generating an inventory for kubespray
 

contrib/terraform/equinix/README.md

@@ -35,7 +35,7 @@ now six total etcd replicas.
 ## Requirements
 
 - [Install Terraform](https://www.terraform.io/intro/getting-started/install.html)
-- [Install Ansible dependencies](/docs/ansible.md#installing-ansible)
+- [Install Ansible dependencies](/docs/ansible/ansible.md#installing-ansible)
 - Account with Equinix Metal
 - An SSH key pair
 

contrib/terraform/openstack/README.md

@@ -619,7 +619,7 @@ Edit `inventory/$CLUSTER/group_vars/k8s_cluster/k8s_cluster.yml`:
 
 - Set variable **kube_network_plugin** to your desired networking plugin.
   - **flannel** works out-of-the-box
-  - **calico** requires [configuring OpenStack Neutron ports](/docs/openstack.md) to allow service and pod subnets
+  - **calico** requires [configuring OpenStack Neutron ports](/docs/cloud_providers/openstack.md) to allow service and pod subnets
 
 ```yml
 # Choose network plugin (calico, weave or flannel)