msg="failed to get CA certs

[choeyhao]

Environmental Info:
K3s Version: v1.21.3+k3s1

Node(s) CPU architecture, OS, and Version:
Master:
k3s version:v1.21.3+k3s1
linux version:Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-151-generic x86_64)

Agent:
linux version :Linux version 3.10.0-327.el7.x86_64 ([email protected])

Cluster Configuration:
"1 servers, 1 agents".

Describe the bug:
The Agent node can't add to the master ,when I check the Agent ,find that:

k3s-agent.service - Lightweight Kubernetes
Loaded: loaded (/etc/systemd/system/k3s-agent.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-08-17 16:18:18 CST; 17h ago
Docs: https://k3s.io
Main PID: 64812 (k3s-agent)
Memory: 42.0M
CGroup: /system.slice/k3s-agent.service
└─64812 /usr/local/bin/k3s agent

Aug 18 09:46:13 sutpcnode1 k3s[64812]: time="2021-08-18T09:46:13.501046169+08:00" level=error msg="failed to get CA certs: Get "https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:51283->...reset by peer"
Aug 18 09:46:19 sutpcnode1 k3s[64812]: time="2021-08-18T09:46:19.513018926+08:00" level=error msg="failed to get CA certs: Get "https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:51289->...reset by peer"
Aug 18 09:46:25 sutpcnode1 k3s[64812]: time="2021-08-18T09:46:25.528046663+08:00" level=error msg="failed to get CA certs: Get "https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:51295->...reset by peer"

Steps To Reproduce:
1 change the hostname and add to both hosts
2 restall the K3S many times
3 update the /etc/rancher/node/password of agent and the /var/lib/rancher/k3s/server/cred/node-passwd of master
4 iptables

Expected behavior:
Add the agent to master success.

Actual behavior:

Additional context / logs:

Backporting

• Needs backporting to older releases

[ieugen]

I've also encountered this issue seems to be related to #1395 .

I do set --server on the agent.
ExecStart=/usr/local/bin/k3s agent --server https://REDACTED:6443 --token-file /etc/rancher/k3s/cluster-token --config /etc/rancher/k3s/config.yaml

k3s is installed via https://github.com/PyratLabs/ansible-role-k3s .

[nachtschatt3n]

I have the same issue, with Raspbian OS newest patchlevel as of (24.11.21). I also used the ansible k3s role. But It failed and I checked the logs. "level=error msg="failed to get CA certs: Get "https://127.0.0.1:6444/cacerts\""

When I try to connect to curl 127.0.0.1:6444 it does not connect and when I run the K3s installer with the static ip address as master it's also not working.

It seems to be an issue with the new raspbian version, I installed this like it 4 months ago and it worked without issue.

[helletheone]

i have the same problem with ubuntu news version.
" level=error msg="Failed to configure agent: https://127.0.0.1:6444/v1-k3s/serving-kubelet.crt: 503 Service Unavailable""

[brandond]

You might see if the fix here works for you: #4188 (comment)

[helletheone]

not really @brandond ;/

[sjmach]

@helletheone I faced a similar issue today. Can you see whether iptables rules are fine. Can you try with iptables -S on agent node? The above command will clear all rules, so you might need to add rules you want again.

[helletheone]

@sjmach now it works!! :) thank you both guys: @brandond and @sjmach

[lukemilby]

@helletheone was it the iptables rules?

[helletheone]

@lukemilby yes

[KalaRaja]

@sjmach now it works!! :) thank you both guys: @brandond and @sjmach

what did you do to fix it ?

[sjmach]

@KalaRaja Use this command -> iptables -S

[stale]

This repository uses a bot to automatically label issues which have not had any activity (commit/comment/label) for 180 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the bot can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the bot will automatically close the issue in 14 days. Thank you for your contributions.