Running playbook - Error loading unit file 'k8s-certs-renew.timer'

[mhabicht]

UPDATE: I have narrowed this down to when this setting is set to true, the error occurs

inventory/mycluster/group_vars/k8s-cluster/k8s-cluster.yml

Automatically renew K8S control plane certificates on first Monday of each month

auto_renew_certificates: true

Yet setting this one to true does not cause a problem

roles/kubernetes/master/defaults/main/main.yml

Automatically renew K8S control plane certificates on first Monday of each month

auto_renew_certificates: true

---

ORIGINAL:
When running playbook I get this error, not sure what is causing this error. I saw another person had this problem, but was closed do to OS version, no resolve.
FAILED! => {"changed": false, "msg": "Error loading unit file 'k8s-certs-renew.timer': org.freedesktop.DBus.Error.InvalidArgs \"Invalid argument\""}

Full output
https://gist.github.com/mhabicht/8498befb5c7d3184ec5fc5dae66b5f10

roles/kubernetes/master/tasks/main.yml

• name: Renew K8S control plane certificates monthly 2/2
  systemd:
  name: k8s-certs-renew.timer
  enabled: yes
  state: started
  daemon-reload: "{{ k8s_certs_units is changed }}"
  when: auto_renew_certificates

Bare Metal-Dell servers

Linux 4.4.0-210-generic x86_64
NAME="Ubuntu"
VERSION="16.04.7 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.7 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

• Version of Ansible (ansible --version):
  ansible 2.9.18
  config file = /root/kubespray-2.15.1/ansible.cfg
  configured module search path = ['/root/kubespray-2.15.1/library']
  ansible python module location = /usr/local/lib/python3.5/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.5.2 (default, Jan 26 2021, 13:30:48) [GCC 5.4.0 20160609]

• Version of Python (python --version): 2.7.12

**Kubespray version : 2.15.1

Network plugin used: Calico

Full inventory with variables (ansible -i inventory/sample/inventory.ini all -m debug -a "var=hostvars[inventory_hostname]"):
https://gist.github.com/mhabicht/b848b4d4e6b5019195cc0b6d57dcab7f

Command used to invoke ansible:
ansible-playbook -i inventory/mycluster/hosts.yaml cluster.yml -u root --ask-pass -b --become-user=root -vv

Output of ansible run:
https://gist.github.com/mhabicht/82db919441d9552858c2c5cb2b40e2c4

[haminhcong]

I have same issue with @mhabicht in CentOS 7

[champtar]

@haminhcong are you by any chance using an old CentOS 7 version ?

[champtar]

@mhabicht Ubuntu 16.04 is just too old and the version of systemd there doesn't support the 'timespec' we use.

[mhabicht]

@champtar Understood. What is the minimum Ubuntu version, I did not see that listed in the requirements for 15.1? If it is 18, let me ask if 20 is supported yet?

[champtar]

Kubespray CI has both 18.04 and 20.04 if I remember correctly, so yes 20.04 is supported

[mhabicht]

@champtar Does this setting affect the self cert renewal at 365 days?

[champtar]

Not sure to understand your question

[haminhcong]

@haminhcong are you by any chance using an old CentOS 7 version ?

@champtar No, i don't think my OS is too old. It is 2020 update version

uname -a
Linux master-10-207-59-2 3.10.0-1062.12.1.el7.x86_64 #1 SMP Tue Feb 4 23:02:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

[mhabicht]

@champtar The core certs expire after 365 days. Originally it was thought that in Kube 15.3 had the fix to renew these certs, but there was a bug. This bug was fixed in 19.9.
Is the k8s-certs-renew.timer required for the certs to auto redeploy prior to 365 days? or at day 366 will I get the cert exp error when I run kubectl get pods?

[champtar]

The timer just runs the script to do the cert renew, you can run the script manually

[champtar]

Both: maybe you don't have #7472 in the version you are using

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.