Certificate update path is not clear

[nyurik]

Could someone clarify what happens during the certificate update:

• When using k8s secret, will Ingress automatically pick up the new certificate when it gets updated in-place?
• For GCP certificates, it is not possible to update a cert in-place - a new certificate must first be created under a new name, then the https proxy is updated to use it, and then the old cert can be deleted. This means that k8s annotation would not be very practical - certs are much easier to update with Terraform, using the randomly generated name. Is there a recommended approach for this method?
  Thx!

[rramkumar1]

• When using k8s secret, will Ingress automatically pick up the new certificate when it gets updated in-place?

No, we currently do not watch secrets but it's something we can think about doing.

• For GCP certificates, it is not possible to update a cert in-place - a new certificate must first be created under a new name, then the https proxy is updated to use it, and then the old cert can be deleted. This means that k8s annotation would not be very practical - certs are much easier to update with Terraform, using the randomly generated name. Is there a recommended approach for this method?

I'm not familiar w/ how Terraform manages certs. Does it take care of updating the Ingress spec for you?

[nyurik]

• When using k8s secret, will Ingress automatically pick up the new certificate when it gets updated in-place?

No, we currently do not watch secrets but it's something we can think about doing.

That's strange - today I tested it by changing the tls secret for an existing ingress, and after a few minutes the LB has automatically began using it. I used this command to compare the old and the new:

echo | openssl s_client -showcerts -servername example.com -connect example.com:443 2>/dev/null | openssl x509 -inform pem -noout -text > info-1.txt

Could it be that Google's load balancer monitors it? My ingress definition:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: main-svc
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "named-ip-address"
spec:
  tls:
  - secretName: tls-cert
  backend:
    serviceName: varnish-svc
    servicePort: varnish-sv-prt

---

• For GCP certificates, it is not possible to update a cert in-place - a new certificate must first be created under a new name, then the https proxy is updated to use it, and then the old cert can be deleted. This means that k8s annotation would not be very practical - certs are much easier to update with Terraform, using the randomly generated name. Is there a recommended approach for this method?

I'm not familiar w/ how Terraform manages certs. Does it take care of updating the Ingress spec for you?

Sorry for not making it clear. Terraform does not know anything about k8s/Ingress. I used Terraform to initialize the k8s cluster and create a static IP resource. I could also create a certificate resource, and use its name with ingress.gcp.kubernetes.io/pre-shared-cert: 'cert-name', but there is no point in doing that because the certificate resource name is not permanent - I cannot upload a new cert into the same resource (the resource is readonly), and I cannot delete it until k8s stops using it.

For my other, non-k8s project, I used Terraform to also set up the loadbalancer. In that case, the certificate dynamic (random) name is ok - whenever cert is updated, it gets a new name, and the LB is updated with the new cert resource name before the old one is deleted. Problem is - I cannot set up a LB via Terraform that points to backend ports 32xxx, only to 80.

Hence the confusion - if i use ingress.gcp.kubernetes.io/pre-shared-cert: '...' annotation, it makes no sense because cert cannot be updated. The first method seems to work, but you said it shouldn't have... :)

[rramkumar1]

That's strange - today I tested it by changing the tls secret for an existing ingress, and after a few minutes the LB has automatically began using it. I used this command to compare the old and the new:

We periodically requeue every Ingress for processing (every 10 minutes I think). So its possible you updated the secret and then a couple minutes later the Ingress using it got requeued as expected, but not as a direct reaction to the secret update.

[nyurik]

@rramkumar1 if Ingress refreshes every 10 min, that's totally fine. LetsEncrypt certificates expire in 90 days, so as long as Ingress refreshes regularly, it will automatically pick up the updated secret. Could we document this behavior somehow? Not sure what the best place should be. I'm sure everyone who works with certs will face the same question. Thx!

[rramkumar1]

/kind documentation

[rramkumar1]

@nyurik How does this sound?

"Changes to secrets are picked up periodically so if you modify the
data inside of the secret, it will take a max of 10 minutes for those changes
to be applied to the load balancer."

[rramkumar1]

@nyurik Docs are published. If it's not clear enough, let me know so I can fix it.

Ref: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress#setting_up_https_tls_between_client_and_load_balancer

/close

[k8s-ci-robot]

@rramkumar1: Closing this issue.

In response to this:

@nyurik Docs are published. If it's not clear enough, let me know so I can fix it.

Ref: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress#setting_up_https_tls_between_client_and_load_balancer

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.