-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[epic] Secure communication with x509 certificates #76
Comments
@gecube could you please elaborate on this, and write user stories and user spec for certificates |
How to create certificates infrastructurePermalink to kamaji-etcd repo.
Generate CA
CSR for our etcd (if we need it with cert-manager):
Important points:
Questions:
Generate certificate for peer authenticationPeer authentication certs authenticate etcd nodes against etcd nodes and used as server certificates.
config.json - configuration for certificate requests (1 year validity):
peer-csr.json - certificate request config. We need to create it. Kamaji example:
Hostnames helm template:
Important points:
Questions:
Generate server certificate for client trust
server-csr.json - certificate request config. We need to create it. Kamaji example:
Hostnames helm template:
Important points:
Questions:
Generate root client certificate
root-client-csr.json request config. We need to create it. Kamaji example:
Important points:
Questions:
Etcd reference for v3.5What certificates and private keys to use for what configuration key: kamaji-etcd chart: Client-to-server communication:
Peer (server-to-server / cluster) communication: The peer options work the same way as the client-to-server options:
If either a client-to-server or peer certificate is supplied the key must also be set. All of these configuration options are also available through the environment variables, ETCD_CA_FILE, ETCD_PEER_CA_FILE and so on. Common options:
|
For now it was decided to create secret reference sections in the etcdCluster spec and expect secrets with certificates from the user of etcd-operator. Basic spec from @lllamnyp (TBD)
FYI @kvaps |
Peer.ca
Peer.cert
ClientServer section has the same fields as peer section.Rbac
spec:
security:
peer:
enabled: true # optional
ca:
# if not defined, then operator generates CA by the spec below
secretName: ext-peer-ca-tls-secret # oneof secretName or secretTemplate
secretTemplate: # oneof secretName or secretTemplate
annotations: {} # optional
labels: {} # optional
duration: 86400h # optional
renewBefore: 720h # optional
privateKey:
algorithm: RSA # optional
size: 4096 # optional
cert:
secretName: ext-peer-tls-secret
secretTemplate:
annotations: {}
labels: {}
duration: 720h
renewBefore: 180h
privateKey:
rotate: true # optional
algorithm: RSA
size: 4096
clientServer:
enabled: true
ca:
secretName: ext-server-ca-tls-secret
secretTemplate:
annotations: {}
labels: {}
duration: 86400h
renewBefore: 720h
privateKey:
algorithm: RSA
size: 4096
serverCert:
secretName: ext-server-tls-secret
secretTemplate:
annotations: {}
labels: {}
extraSANs: []
duration: 720h
renewBefore: 180h
privateKey:
rotate: true
algorithm: RSA
size: 4096
rootClientCert:
secretName: ext-client-tls-secret
secretTemplate:
annotations: {}
labels: {}
duration: 720h
renewBefore: 180h
privateKey:
rotate: true
algorithm: RSA
size: 4096
auth:
enabled: true # optional Important points:
|
I would suggest to reuse logic of resource templates we introduced before, eg for every specific resources configuration:
Instead of this:
|
Also I'd like to ask if we really need to mount CA secret to the etcd containers? - or is it just to manage it? |
@kvaps We discussed it with @Kirill-Garbar and there is no obvious reason to do it. As |
Ofcourse I do not want to mount CA secret to etcd container - it is not necessary. |
I don't like |
Client-to-server communication is written in etcd security documents: https://etcd.io/docs/v3.5/op-guide/security/ In this section root client and etcd server certificates are defined. |
Yeah, and I like this. However despite the fact it is guaranteed by cert-manager, it is not guaranteed by |
I do not want to make CA and CERTificate one secret despite the fact that certificate secret has CA.crt in one of the sections. It is not secure - if certificate secret is hijacked, then BAD ACTOR can place its own CA and etcd will automatically trust it. But I agree that in operator functionality CA secret (that is mounted to etcd) should not include ca.key. |
What is the difference between Also option If you want to manage additional users and their access to the cluster, it must be implemented later using CRs. Imao |
I don't agree as if hi-jacker has the access to the secrets with the certificates and private keys, the game is over already. |
ClientServer cert is server certificate. A agree with the topic anbout the CRs for users. But root cert is necessary to manage etcd cluster from the operator |
I may guess that it is for different use-cases, like patroni etc. |
You can't remove one security layer, because you have another security layers. Security layers are independent. It is not an argument. |
It's okay, client certificates usually contain CA certificate but not contain CA secret. Just keep in mind that this field introduced by cert-manager and not standartized in So it is widely used, but not standartized in Kubernetes yet. |
agree, it's a good remark, but it could be a part of our "API" or "convention" |
Ah got it! I would like to move this into the same level then, eg:
As this would make it closer to repeat etcd configuration keys. |
I personally like nesting structures more as usually hierarchical structure shows relations and usually more extendable. But I do not have an argument against "closer to etcd keys". Semantically closer to etcd is important. I prepared structure with your proposal.
spec:
security:
clientTLS: false # Disables client-server tls communication
rbac: false # Disables etcd role-based access control
peerCertificate:
secretName: ext-peer-tls-secret # oneof secretName or secretTemplate
secretTemplate: # oneof secretName or secretTemplate
annotations: {}
labels: {}
duration: 720h
renewBefore: 180h
privateKey:
rotate: true # optional
algorithm: RSA
size: 4096
peerTrustedCACertficate:
# if not defined, then operator generates CA by the spec below
secretName: ext-peer-ca-tls-secret
secretTemplate:
annotations: {}
labels: {}
duration: 86400h # optional
renewBefore: 720h # optional
privateKey:
algorithm: RSA # optional
size: 4096 # optional
serverCertificate:
secretName: ext-server-tls-secret
secretTemplate:
annotations: {}
labels: {}
duration: 720h
renewBefore: 180h
privateKey:
rotate: true
algorithm: RSA
size: 4096
trustedCACertificate:
secretName: ext-server-ca-tls-secret
secretTemplate:
annotations: {}
labels: {}
duration: 86400h
renewBefore: 720h
privateKey:
algorithm: RSA
size: 4096
clientCertificate:
secretName: ext-client-tls-secret
secretTemplate:
annotations: {}
labels: {}
duration: 720h
renewBefore: 180h
privateKey:
rotate: true
algorithm: RSA
size: 4096 |
Cool! Let's now add there
If it's specified, it means our operator manages secrets Similarly to volumeClaimTemplate in STS or podDisruptionBudgetTemplate logic we have |
I would say that operator should manage secrets by default without any additional configurations. So I would base this decision (manage by operator or attach external ones) on another field: P.S. Changed both specs by your request: first nested version and flat map. |
I would suggest:
In this way, to enable automatic certificate managedement user would need to add simple: ---
apiVersion: etcd.aenix.io/v1alpha1
kind: EtcdCluster
metadata:
name: test
namespace: default
spec:
replicas: 3
security: {} |
Another option is that we can introduce security:
clientAuth: true
clientTLS: true
tls:
peer:
secretName: ""
secretTemplate: {}
peerCA:
secretName: ""
secretTemplate: {}
ca:
secretName: ""
secretTemplate: {}
client:
secretName: ""
secretTemplate: {}
server:
secretName: ""
secretTemplate: {}
extraSANs: [] |
All right I agree with the variant from #76 (comment) |
Secure communication between etcd members and etcd clients with x509 certificates.
This is a last task remained before we start using etcd-operator in our cozystack platform.
The text was updated successfully, but these errors were encountered: