Kubernetes secrets signed by kops CA

[olemarkus]

1. Describe IN DETAIL the feature/behavior/change you would like to see.

There are a couple of use cases for having kubernetes TLS secets signed by the kops CA. Metrics Server can use these secrets as client/serving certs. Addons using webhooks would also benefits from this.

2. Feel free to provide a design supporting your feature request.

This feature feels like a natural extension on of kops-controller and there are two ways of implementing this that I can think of.

One would be to use a light-weight controller-like component that would get take a list of secret names passed through options and create them. Would probably need to implement leader-elections and such manually as using controller-runtime can get a bit awkward. There is nothing to watch and reconcile on.

The perhaps better alternative would be to introduce a CRD named KubeletSecret or similar, then use a proper controller-runtime controller to watch and reconcile into k8s secrets. The KubeletSecret resources could be provisioned as part of the addon. One could still pass in a list of allowed KubeletSecrets for extra safety to prevent users from creating arbitrary secrets. This would probably be a much simpler implementation, but it does introduce a kOps CRD.

[johngmyers]

I maintain proofpoint/kapprover and proofpoint/certificate-init-container for the TLS server cert use case. We since wrote (but haven't open-sourced) a mutating admission webhook to inject the init container. It provides a per-pod certificate/key in a volume, not one shared using a Secret.

[olemarkus]

If we were do go down this path, the addon could just specify the init-container, so the mutating webhook is probably not that useful. In our case, kops-controller would be the signer, so the code there would be somewhat similar to the CRD approach.

Validating the CSR is probably more complex than just writing a secret to a well-known location and trust that only e.g metrics-server has the RBAC privileges to read it though.

To me, this feels a bit too complex.

[johngmyers]

The CRD approach looks equivalent to a cert-manager Issuer type that issues from the cluster CA. I would not like us to invent a CRD that duplicates cert-manager's Certificate.

The mutating webhook abstracts away the implementation detail of the init container. The workload doesn't need to know (or update) the init container's version. The init container implementation could be replaced with a FlexVolume implementation, for example.

Validating the CSR is not that complex.

[olemarkus]

The CRD approach is analog to cert-manager, yes. We could very well be using cert-manager if we want to add that as an addon. We could then create an Issuer of type external that kops-operator is responsible for.

I was slightly reluctant to supporting cert-manager given its happiness fo breaking backwards-compatibility. But now that they have reached 1.0, I expect it to be more stable.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[olemarkus]

The underlying problem has been solved by using cert-manager instead.
/close

[k8s-ci-robot]

@olemarkus: Closing this issue.

In response to this:

The underlying problem has been solved by using cert-manager instead.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.