Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add multiple signers to CSRs #1400

Merged
merged 1 commit into from
Jan 15, 2020
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 129 additions & 7 deletions keps/sig-auth/20190607-certificates-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ approvers:
- "@liggitt"
- "@smarterclayton"
creation-date: 2019-06-07
last-updated: 2019-09-04
last-updated: 2019-12-09
status: implementable
---

Expand All @@ -26,6 +26,8 @@ status: implementable
- [Proposal](#proposal)
- [Design Details](#design-details)
- [Sequence of an Issuance](#sequence-of-an-issuance)
- [Signers](#signers)
- [Limiting approval and signer powers for certain signers.](#limiting-approval-and-signer-powers-for-certain-signers)
- [CertificateSigningRequest API Definition](#certificatesigningrequest-api-definition)
- [Manual CSR Approval With Kubectl](#manual-csr-approval-with-kubectl)
- [Automatic CSR Approval Implementations](#automatic-csr-approval-implementations)
Expand Down Expand Up @@ -90,26 +92,26 @@ signer can interact.

A client requesting a certificate post a CertificateSigningRequest to the
Certificates API. The client may only provide the encoded [Certificate
Request](https://tools.ietf.org/html/rfc2986) and usages of the certificate in
the spec (and the standard object metadata) on the initial creation of the
Request](https://tools.ietf.org/html/rfc2986), usages of the certificate in
the spec, the standard object metadata, and the requested signer on the initial creation of the
CertificateSigningRequest. The kube-apiserver also asserts authentication
attributes of the requestor in the CertificateSigningRequest spec before
committing it to storage so that they can be used later during CSR approval. The
information contained in the spec is immutable after the request is created.

The approver updates approval status of the CertificateSigningRequest via the
An approver updates approval status of the CertificateSigningRequest via the
CertificateSigningRequestStatus. The approval condition can only be updated via
the `/approval` subresource allowing approval permission to be authorized
independently of other operations on the CertificateSigningRequest.

Contingent on approval, the signer posts a signed certificate to the status. The
Contingent on approval, a signer posts a signed certificate to the status. The
certificate field of the status can only be updated via the `/status`
subresource allowing signing permission to be authorized independently of other
operations on the CertificateSigningRequest.

The API is designed to support the standard asynchronous controller model of
Kubernetes where the approver and signer act as independent controllers of the
Certificates API. Since issuance is asynchronous, the approver can perform
Certificates API. Since issuance is asynchronous, an approver can perform
out-of-band verification of the CSR before making an authorization decision.

The approver is designed to be explicitly independent of the signer. This
Expand All @@ -132,6 +134,105 @@ A typical successful issuance proceeds as follows.
the `.Status.Certificate` field.
1. The requestor observes the update, and stores the certificate locally.

### Signers
liggitt marked this conversation as resolved.
Show resolved Hide resolved

CSRs have a `signerName` field which is used to specify which signer the CSR creator wants to sign the certificate.
deads2k marked this conversation as resolved.
Show resolved Hide resolved
To support migration from v1beta1 to v1, this required field will be defaulted in v1beta1 (optional in openapi), but
not defaulted and required in v1 :
1. If it's a kubelet client certificate, it is assigned "kubernetes.io/kubelet-client".
deads2k marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think this should be kube-apiserver-client-kubelet now?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2. If it's a kubelet serving certificate, it is assigned "kubernetes.io/kubelet-serving".
see https://github.com/kubernetes/kubernetes/blob/release-1.10/pkg/controller/certificates/approver/sarapprove.go#L211-L223 for details.
3. Otherwise, it is assigned "kubernetes.io/legacy-unknown".

There will be field selector support to make approvers and signers easier to write.

All signers should provide information about how they work so that clients can predict what will happen to their CSRs.
liggitt marked this conversation as resolved.
Show resolved Hide resolved
liggitt marked this conversation as resolved.
Show resolved Hide resolved
This includes:
1. Trust distribution - how trust (ca bundles) are distributed.
2. Permitted subjects - (any? specific subtree?) and behavior when a disallowed subject is requested.
3. Permitted x509 extensions - (IP SANs? DNS SANs? Email SANs? URI SANs? others?) and behavior when a disallowed
extension is requested.
4. Permitted key usages / extended key usages - (client only? server only? any? signer-determined? CSR-determined?) and
behavior when usages different than the signer-determined usages are specified in the CSR.
5. Expiration/cert lifetime - (fixed by signer? configurable by admin? CSR-determined?) and behavior when an expiration
different than the signer-determined expiration is specified in the CSR.
6. CA bit allowed/disallowed - and behavior if a CSR contains a request a for a CA cert when the signer does not permit it.


sig-auth reserves all `kubernetes.io/*` `signerNames` and more may be added in the future.
Kubernetes provides the following well-known signers. Today, failures for all of these are only reported in kube-controller-manager logs:
1. kubernetes.io/kube-apiserver-client - signs certificates that will be honored as client-certs by the kube-apiserver.
Never auto-approved by kube-controller-manager.
liggitt marked this conversation as resolved.
Show resolved Hide resolved
1. Trust distribution: signed certificates must be honored as client-certificates by the kube-apiserver. The CA bundle
is not distributed by any other means.
2. Permitted subjects - no subject restrictions, but approvers and signers may choose not to approve or sign.
Certain subjects like cluster-admin level users or groups vary between distributions and installations, but deserve
additional scrutiny before approval and signing. An admission plugin is available to restrict system:masters, but
it is often not the only cluster-admin subject in a cluster.
Comment on lines +170 to +171
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this implemented in admission control vs approver or signer policy?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deads2k have you got any input here? We currently have this implemented as an admission plugin in kubernetes/kubernetes#86933, but is it better we move this to the approval flow?

3. Permitted x509 extensions - Non-critical extensions should be dropped.
4. Permitted key usages - must include `[]string{"client auth"}`. Must not include key usages beyond `[]string{"digital signature", "key encipherment", "client auth"}`
5. Expiration/cert lifetime - minimum of CSR signer or request. Sanity of the time is the concern of the signer.
6. CA bit allowed/disallowed - not allowed.
2. kubernetes.io/kube-apiserver-client-kubelet - signs client certificates that will be honored as client-certs by the kube-apiserver.
May be auto-approved by kube-controller-manager.
1. Trust distribution: signed certificates must be honored as client-certificates by the kube-apiserver. The CA bundle
is not distributed by any other means.
2. Permitted subjects - organizations are exactly `[]string{"system:nodes"}`, common name starts with `"system:node:"`
3. Permitted x509 extensions - none
4. Permitted key usages - exactly `[]string{"key encipherment", "digital signature", "client auth"}`
5. Expiration/cert lifetime - minimum of CSR signer or request. Sanity of the time is the concern of the signer.
6. CA bit allowed/disallowed - not allowed.
3. kubernetes.io/kubelet-serving - signs serving certificates that are honored as a valid kubelet serving certificate
by the kube-apiserver, but has no other guarantees. Never auto-approved by kube-controller-manager.
1. Trust distribution: signed certificates must be honored by the kube-apiserver as valid to terminate connections to a kubelet.
The CA bundle is not distributed by any other means.
2. Permitted subjects - organizations are exactly `[]string{"system:nodes"}`, common name starts with `"system:node:"`
3. Permitted x509 extensions - DNS and IP SANs are allowed
4. Permitted key usages - exactly `[]string{"key encipherment", "digital signature", "server auth"}`
5. Expiration/cert lifetime - minimum of CSR signer or request.
6. CA bit allowed/disallowed - not allowed.
4. kubernetes.io/legacy-unknown - has no guarantees for trust at all. Some distributions may honor these as client
certs, but that behavior is not standard kubernetes behavior. Never auto-approved by kube-controller-manager.
1. Trust distribution: None. There is no standard trust or distribution for this signer in a kubernetes cluster.
2. Permitted subjects - any
3. Permitted x509 extensions - honors SAN extensions and discards other extensions.
4. Permitted key usages - any
5. Expiration/cert lifetime - minimum of CSR signer or request. Sanity of the time is the concern of the signer.
6. CA bit allowed/disallowed - not allowed.

Distribution of trust happens out of band for these signers. Any trust outside of those described above are strictly
deads2k marked this conversation as resolved.
Show resolved Hide resolved
coincidental. For instance, some distributions may honor kubernetes.io/legacy-unknown as client-certificates for the
kube-apiserver, but this is not a standard.
None of these usages are related to ServiceAccount token secrets `.data[ca.crt]` in any way. That ca-bundle is only
guaranteed to verify a connection the kube-apiserver using the default service.

To support HA upgrades, the kube-controller-manager will duplicate defaulting code for an empty `signerName` for one
release.

#### Limiting approval and signer powers for certain signers.
Given multiple signers which may be implemented as "dumb" controllers that sign if the CSR is approved, there is benefit
to providing a simple way to subdivide approval powers through the API. We will introduce an admission plugin that requires
1. verb == `approve`
2. resource == `signers`
3. name == `<.spec.signerName>`
4. group == `certificates.k8s.io`

To support a use-case that wants a single rule to allow approving an entire domain (example.com in example.com/cool-signer),
there will be a second check for
1. verb == `approve`
2. resource == `signers`
3. name == `<.spec.signerName domain part only>/*`
4. group == `certificates.k8s.io`

There are congruent check for providing a signature that use the verb=="sign" instead of "approve" above.

For migration, we will provide three bootstrap cluster-roles defining authorization rules needed to approve CSRs for the kubernetes.io signerNames.
Cluster admins can either:
1. grant signer-specific approval permissions using roles they define
2. grant signer-specific approval permissions using the bootstrap roles starting in 1.18
3. disable the approval-authorizing admission plugin in 1.18 (if they don't care about partitioning approver rights)


### CertificateSigningRequest API Definition

```go
Expand All @@ -145,6 +246,22 @@ type CertificateSigningRequest struct {


type CertificateSigningRequestSpec struct {
// requested signer for the request up to 571 characters long. It is a qualified name in the form: `scope-hostname.io/name`.
// If empty, it will be defaulted for v1beta1:
// 1. If it's a kubelet client certificate, it is assigned "kubernetes.io/kubelet-client". This is determined by
// Seeing if organizations are exactly `[]string{"system:nodes"}`, common name starts with `"system:node:"`, and
// key usages are exactly `[]string{"key encipherment", "digital signature", "client auth"}`
// 2. Otherwise, it is assigned "kubernetes.io/legacy-unknown".
// In v1 it will be required. Distribution of trust for signers happens out of band.
// The following signers are known to the kube-controller-manager signer.
// 1. kubernetes.io/kube-apiserver-client - signs certificates that will be honored as client-certs by the kube-apiserver. Never auto-approved by kube-controller-manager.
// 2. kubernetes.io/kubelet-client - signs client certificates that will be honored as client-certs by the kube-apiserver. May be auto-approved by kube-controller-manager.
// 3. kubernetes.io/kubelet-serving - signs serving certificates that are honored as a valid kubelet serving certificate by the kube-apiserver, but has no other guarantees.
// 4. kubernetes.io/legacy-unknown - has no guarantees for trust at all. Some distributions may honor these as client certs, but that behavior is not standard kubernetes behavior.
// None of these usages are related to ServiceAccount token secrets `.data[ca.crt]` in any way.
liggitt marked this conversation as resolved.
Show resolved Hide resolved
// You can select on this field using `.spec.signerName`.
SignerName string

// Base64-encoded PKCS#10 CSR data
Request []byte

Expand Down Expand Up @@ -240,7 +357,12 @@ control plane.

### Beta -> GA Graduation

- TBD
Things to resolve for v1.
1. .spec.signerName should be non-defaulted and required
2. Should we disallow the legacy .spec.signerName in v1?
3. Maybe there should be a way to provide intermediate certificates. Note that this does not solve trust rotation overall
since clients still have to trust the certificates after the CSR instance has been reaped.
4. Define how signers indicate terminal failure in signing on a CSR. Fix status conditions perhaps?

## Implementation History

Expand Down