Managed certificates do not include intermediate certs

[cemerick]

Please provide us with the following information:

This issue is a: (mark with an x)

• bug report -> please search issues before submitting
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Issue description

The managed certificates supported in the current/forthcoming preview do not include the intermediate "GeoTrust Global TLS" cert, which causes connections to domains secured by those certificates to fail from non-browser clients (e.g. curl, wget, and probably HTTP client libraries in various programming languages as well, depending on how they handle resolution of intermediate certificates, etc).

Per Qualsys' SSL Test, "This server's certificate chain is incomplete." Specifically, in its view of the certificate chain, it shows GeoTrust Global TLS RSA4096 SHA256 2022 CA1 as requiring an extra download (which apparently browsers will do, but e.g. curl will not?). For reference, the cloudflare cert that we use for our ACA apps currently (with good results) carries all of its intermediate certificates.

Presumably the fix here would be for ACA's managed certificates to form a complete chain from the root CA. The big question is whether that's slated to happen, or if this is as far as managed certificates are intended to go, and those looking for non-browser compatibility (for APIs and such) should plan on continuing to use other solutions.

Steps to reproduce

1. Add and bind a hostname for a container app as described in Managed Certificates #607
2. Attempt to connect to it from e.g. curl.

Expected behavior [What you expected to happen.]

Typical connection and a response from the service in question.

Actual behavior [What actually happened.]

An error message, and a terminated connection attempt. e.g. curl:

$ curl -v https://app.container.service.com
*   Trying 20.75.45.63:443...
* Connected to app.container.service.com (20.75.45.63) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS header, Unknown (21):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

and wget:

ERROR: cannot verify app.container.service.com's certificate, issued by ‘CN=GeoTrust Global TLS RSA4096 SHA256 2022 CA1,O=DigiCert\\, Inc.,C=US’:
  Unable to locally verify the issuer's authority.
To connect to app.container.service.com insecurely, use `--no-check-certificate'.

[anthonychu]

We’ll have this fixed before the managed certs feature officially enters public preview. Thanks

[rhuanbarreto]

As in the board the feature entered public preview, so this is fixed?

[rhuanbarreto]

@anthonychu can you confirm whether this is fixed?

[mburumaxwell]

@rhuanbarreto I have about 16 managed certificates across different container apps and environments. I can confirm that the intermediate certificates are served in a similar manner as to when brining in your own certificate.

However, the order of certificates seems to still be wrong. This happens even when you bring your own certificate but not with the default *.azurecontainerapps.io domain.

You should not have an issue using these.

Maybe I should also add, that if you are using bicep for deployment it is likely to fail after almost 2 hours but the certificate has been provisioned. I solve this by cancelling the deployment once the provisioning state of the managed certificate showed Succeeded on the portal (requires you to view hidden resources) then redeploy. It succeeds the second time.

[marcindulak]

As for the incorrect order of certificates, is this Azure/azure-rest-api-specs#10637 what you see?

[mburumaxwell]

As for the incorrect order of certificates, is this Azure/azure-rest-api-specs#10637 what you see?

I never used the Azure Key Vault Provider for Secrets Store CSI Driver when I used AKS thus cannot provide any concrete information on it. However, it could be the case because certificates in KeyVault used for Application Gateway and Azure CDN which are served in the correct order while these are not. Though, I would have thought that the certificate for the default domain is served using the same mechanisms as other certificates; but I just realized the root and intermediate certificates are different from the ones on the default domain.

I wonder in which/whose KeyVault the managed certificate is stored (I thought it was a Kubernetes TLS Secret).

[anthonychu]

@anthonychu can you confirm whether this is fixed?

Yes, this was fixed before we announced public preview. Please let us know if you're still having issues.