Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade-cluster.yml not upgrade kubeadm/kubectl on route reflectors (calico-rr) #6386

Closed
gaima8 opened this issue Jul 10, 2020 · 0 comments · Fixed by #6447
Closed

upgrade-cluster.yml not upgrade kubeadm/kubectl on route reflectors (calico-rr) #6386

gaima8 opened this issue Jul 10, 2020 · 0 comments · Fixed by #6447
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@gaima8
Copy link
Contributor

gaima8 commented Jul 10, 2020

Environment:

  • Cloud provider or hardware configuration:

Self hosted virtual machines

  • OS (printf "$(uname -srm)\n$(cat /etc/os-release)\n"):

Linux 4.19.0-8-amd64 x86_64
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

  • Version of Ansible (ansible --version):

ansible 2.9.6

  • Version of Python (python --version):

Python 3.6.10

Kubespray version (commit) (git rev-parse --short HEAD):

3d6b9d6

Tags v2.13.1 and v2.13.2

Network plugin used:

calico

Full inventory with variables (ansible -i inventory/sample/inventory.ini all -m debug -a "var=hostvars[inventory_hostname]"):

[all]

[kube-master]
bddpink8sm1
bddpink8sm2

[etcd]
bddpink8se1
bddpink8se2
bddpink8se3

[kube-node]
bddpink8sw1
bddpink8sw2
bddpink8sw3

[calico-rr]
bddpink8srr1
bddpink8srr2

[k8s-cluster:children]
kube-master
kube-node
calico-rr

[rack0]
bddpink8srr1
bddpink8srr2
bddpink8sm1
bddpink8sm2
bddpink8sw1
bddpink8sw2
bddpink8sw3

[rack0:vars]
cluster_id="10.16.35.18"

Command used to invoke ansible:

ansible-playbook --become -i ../bddpink8s/inventory.ini upgrade-cluster.yml

Output of ansible run:

Anything else do we need to know:

This cluster was deployed from tag v2.12.4, then upgraded to tag v2.13.1, and then v2.13.2 (as more of a sanity check than anything).
During both upgrades kubeadm and kubectl on the route reflectors were not upgraded.
A very naive inspection of the kubespray output shows that the task kubernetes/node : install | Copy kubelet binary from download dir did not get executed on the route reflector nodes.

Before upgrade

$ kubectl get nodes -o wide                                
NAME                                STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                       KERNEL-VERSION   CONTAINER-RUNTIME
bddpink8sm1    Ready    master   91d   v1.16.8   10.16.35.11   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8sm2    Ready    master   91d   v1.16.8   10.16.35.12   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8srr1   Ready    <none>   91d   v1.16.8   10.16.35.18   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8srr2   Ready    <none>   91d   v1.16.8   10.16.35.19   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8sw1    Ready    <none>   91d   v1.16.8   10.16.35.21   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8sw2    Ready    <none>   91d   v1.16.8   10.16.35.22   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8sw3    Ready    <none>   91d   v1.16.8   10.16.35.23   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7

After upgrade

$ kubectl get nodes -o wide
NAME                                STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                       KERNEL-VERSION   CONTAINER-RUNTIME
bddpink8sm1    Ready    master   91d   v1.17.6   10.16.35.11   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8sm2    Ready    master   91d   v1.17.6   10.16.35.12   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8srr1   Ready    <none>   91d   v1.16.8   10.16.35.18   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8srr2   Ready    <none>   91d   v1.16.8   10.16.35.19   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8sw1    Ready    <none>   91d   v1.17.6   10.16.35.21   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8sw2    Ready    <none>   91d   v1.17.6   10.16.35.22   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
bddpink8sw3    Ready    <none>   91d   v1.17.6   10.16.35.23   <none>        Debian GNU/Linux 10 (buster)   4.19.0-8-amd64   docker://18.9.7
@gaima8 gaima8 added the kind/bug Categorizes issue or PR as related to a bug. label Jul 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant