Skip to content

Commit

Permalink
Fix multi-node cluster not working after restarting docker
Browse files Browse the repository at this point in the history
In a multi-node cluster with single controlplane node, if the
controlplane node's IP changes, kube-controller-manager and
kube-scheduler would fail to connect kube-apiserver.

enable_network_magic in the entrypoint of the node image has taken care
of fixing up stale IPs in configuration files. This patch does the same
for server address in kubeconfig file. But we update it to loopback
address as the server certificate isn't valid for the new IP.

Signed-off-by: Quan Tian <[email protected]>
  • Loading branch information
tnqn committed Mar 11, 2022
1 parent 575a480 commit 99b7c18
Showing 1 changed file with 14 additions and 0 deletions.
14 changes: 14 additions & 0 deletions images/base/files/usr/local/bin/entrypoint
Original file line number Diff line number Diff line change
Expand Up @@ -372,6 +372,13 @@ enable_network_magic(){
sed -i "s#${old_ipv4}#${curr_ipv4}#" /etc/kubernetes/manifests/*.yaml || true
# this is no longer required with autodiscovery
sed -i "s#${old_ipv4}#${curr_ipv4}#" /var/lib/kubelet/kubeadm-flags.env || true
# controller-manager and scheduler connect to local API endpoint. If the Node's IP changes, server address in
# kubeconfig should be updated. However, the server certificate isn't valid for the new IP. We update it to
# loopback address as it's an alternative address of the certificate.
if [[ "${old_ipv4}" != "${curr_ipv4}" ]]; then
sed -i "s#${old_ipv4}#127.0.0.1#" /etc/kubernetes/controller-manager.conf || true
sed -i "s#${old_ipv4}#127.0.0.1#" /etc/kubernetes/scheduler.conf || true
fi
fi
if [[ -n $curr_ipv4 ]]; then
echo -n "${curr_ipv4}" >/kind/old-ipv4
Expand All @@ -391,6 +398,13 @@ enable_network_magic(){
sed -i "s#${old_ipv6}#${curr_ipv6}#" /etc/kubernetes/manifests/*.yaml || true
# this is no longer required with autodiscovery
sed -i "s#${old_ipv6}#${curr_ipv6}#" /var/lib/kubelet/kubeadm-flags.env || true
# controller-manager and scheduler connect to local API endpoint. If the Node's IP changes, server address in
# kubeconfig should be updated. However, the server certificate isn't valid for the new IP. We update it to
# loopback address as it's an alternative address of the certificate.
if [[ "${old_ipv6}" != "${curr_ipv6}" ]]; then
sed -i "s#${old_ipv6}#::1#" /etc/kubernetes/controller-manager.conf || true
sed -i "s#${old_ipv6}#::1#" /etc/kubernetes/scheduler.conf || true
fi
fi
if [[ -n $curr_ipv6 ]]; then
echo -n "${curr_ipv6}" >/kind/old-ipv6
Expand Down

0 comments on commit 99b7c18

Please sign in to comment.