Skip to content
This repository has been archived by the owner on Sep 30, 2020. It is now read-only.

TLS bootstrapping with the cluster-level CA #406

Closed
danielfm opened this issue Mar 11, 2017 · 6 comments · Fixed by #449
Closed

TLS bootstrapping with the cluster-level CA #406

danielfm opened this issue Mar 11, 2017 · 6 comments · Fixed by #449

Comments

@danielfm
Copy link
Contributor

Kubernetes 1.4 introduces an experimental API for requesting certificates from a cluster-level Certificate Authority (CA) for TLS client certificate bootstrapping for kubelets.

What do you guys think about this? Do you consider adding this to kube-aws at some point?

@mumoshu
Copy link
Contributor

mumoshu commented Mar 12, 2017

Hi @danielfm!
Yes. Perhaps once kubeadm automated the approval process. AFAIK, it requires us to manually run kubectl approve.

@mumoshu mumoshu changed the title TLS bootstrapping TLS bootstrapping with the cluster-level CA Mar 12, 2017
@danielfm
Copy link
Contributor Author

According to the documentation, you can set the --insecure-experimental-approve-all-kubelet-csrs-for-group flag to the group (specified via --token-auth-file) used to send the certificate signing requests.

@mumoshu
Copy link
Contributor

mumoshu commented Mar 13, 2017

Thanks for the info!
Just to be clear, the flag is not intended for production-use:

Use of this flag circumvents makes the “approval” process described below and is not recommended for production use.

It'll be automated via an external controller in the future according to:

This is intended to eventually be an automated process handled by an external approval controller, but for the alpha version of the API it can be done manually by a cluster administrator using kubectl

@danielfm
Copy link
Contributor Author

Just took a quick look at kube-adm, it seems to be using this experimental flag for the TLS bootstrapping.

@danielfm danielfm mentioned this issue Mar 23, 2017
11 tasks
mumoshu pushed a commit that referenced this issue Apr 2, 2017
* Add experimental Kubelet TLS bootstrapping

This PR introduces an experimental feature for provisioning TLS certificates for worker nodes via the alpha [certificate signing requests](https://kubernetes.io/docs/admin/kubelet-tls-bootstrapping/) workflow.

If RBAC is enabled in `cluster.yaml` via the `tlsBootstrap.enabled` key, I also made sure to set up the appropriate cluster role bindings so that the token used for sending the certificate signing requests can only make requests related to certificate provisioning, as instructed by the official documentation.

Checklist:

- [x] Add a `kube-aws render token-file` that creates a token auth file with a cluster TLS bootstrap (if the feature is turned on in `cluster.yaml`, or an empty file otherwise)
- [x] Check whether the auth token file has a proper bootstrap token when the TLS bootstrap feature is turned on; if not, return a self-explaining message so the user knows what to do in order to fix the problem
- [x] Read the bootstrap token from the auth token file and generate an encrypted version of it
- [x] Send the encrypted bootstrap token in worker userdata
- [x] Replace the decrypted token in the kubeconfig used for the TLS bootstrapping workflow
- [x] Fix / add more tests wherever possible
- [x] Manual tests
  - [x] Create clusters with TLS bootstrapping enabled and disabled
  - [x] Make sure it plays well with other experimental features (i.e. self-hosted Calico, AWS node labels, node draining)
  - [x] Try to add worker nodes (and stop/start the kubelet on an existing node) to see if they can join the cluster without the need to manually approve every new certificate signing request
  - [x] Test token rollout methods (**Edit:** `kube-aws update` works as expected, since the generated certificates remain valid even though the token used to request it has changed)

Closes #406.
kylehodgetts pushed a commit to HotelsDotCom/kube-aws that referenced this issue Mar 27, 2018
* Add experimental Kubelet TLS bootstrapping

This PR introduces an experimental feature for provisioning TLS certificates for worker nodes via the alpha [certificate signing requests](https://kubernetes.io/docs/admin/kubelet-tls-bootstrapping/) workflow.

If RBAC is enabled in `cluster.yaml` via the `tlsBootstrap.enabled` key, I also made sure to set up the appropriate cluster role bindings so that the token used for sending the certificate signing requests can only make requests related to certificate provisioning, as instructed by the official documentation.

Checklist:

- [x] Add a `kube-aws render token-file` that creates a token auth file with a cluster TLS bootstrap (if the feature is turned on in `cluster.yaml`, or an empty file otherwise)
- [x] Check whether the auth token file has a proper bootstrap token when the TLS bootstrap feature is turned on; if not, return a self-explaining message so the user knows what to do in order to fix the problem
- [x] Read the bootstrap token from the auth token file and generate an encrypted version of it
- [x] Send the encrypted bootstrap token in worker userdata
- [x] Replace the decrypted token in the kubeconfig used for the TLS bootstrapping workflow
- [x] Fix / add more tests wherever possible
- [x] Manual tests
  - [x] Create clusters with TLS bootstrapping enabled and disabled
  - [x] Make sure it plays well with other experimental features (i.e. self-hosted Calico, AWS node labels, node draining)
  - [x] Try to add worker nodes (and stop/start the kubelet on an existing node) to see if they can join the cluster without the need to manually approve every new certificate signing request
  - [x] Test token rollout methods (**Edit:** `kube-aws update` works as expected, since the generated certificates remain valid even though the token used to request it has changed)

Closes kubernetes-retired#406.
@xdrus
Copy link

xdrus commented Jun 25, 2018

@mumoshu

It'll be automated via an external controller in the future according to...

Do you know any specific plans on it or alpha/beta projects? I'm using heptio-authenticator for worker nodes and it works like a charm, but I need to auto-sign kubelet server certificates to make kubectl logs/exec/port-forward commands work in a cluster. As far as I can see right now kubelet generate clients and server certificates in a bit different ways (NewKubeletClientCertificateManager and NewKubeletServerCertificateManager here) and the main difference that client's certificate signing request doesn't include IPs and DNSnames of a host. And this line in auto approver controller prevents them from being auto signed.

I would love to do this automatically, but can't find the way, as '--insecure-experimental-approve-all-kubelet-csrs-for-group' doesn't work for me too.

@jpsriram
Copy link

Looks like, the experimental flag '--insecure-experimental-approve-all-kubelet-csrs-for-group' for kube-controller-manager has beed removed in version 1.13 [1]

[1] https://github.com/kubernetes/sig-release/blob/master/releases/release-1.13/release-notes-draft.md

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants