-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to delete hosted zone with DNS verified certificate #7063
Comments
Encountered the same issue myself. The lambda creates a Any updates on this or how are we thinking of handling it? |
Possible this may be fixed by the following PR given that this will build this as native CloudFormation (though may need verifying once that has merged): #8552 |
Just ran into the same issue on Any suggested workarounds for now? |
Still present in It wouldn't be so bad, but the stack rollback eliminates anything but the hosted zone and then fails to complete the rollback. You can't then simply redeploy the stack since that creates a new zone (with the same problem), and you have to swap around NS records with each attempt to deploy a zone followed by some manual cleanup of the DNS validation records for the certificates. |
@moltar @chasemaier - marking this as a |
I can make a repro repo example, if that'd help. |
@moltar it would help! any minimal repro would expedite triage resolution :) |
It occurs to me this could well be a CloudFormation limitation but that doesn't mean CDK can't do one better maybe (though I don't know). It seems that when you manually delete the CloudFormation stack that created the cert it still leaves this CNAME entry in place, which is what causes the problem. Maybe CDK can recognise that if such a CNAME as would have been created still exists it deletes it, or at least can be pushed forward to a CloudFormation or beyond team if the problem lays there. |
Here you go: https://github.com/moltar/cdk-hosted-zone I also added repro steps in the README |
I was thinking of maybe attaching a Lambda "Custom Resource" that does a "cleanup" of DNS records within the hosted zone as part of the HostedZone deletion. It seems kind of risky though? What's the opinion of CDK maintainers on this topic? This approach is very similar to the s3.Bucket being often replaced by AutoDeleteBucket. Maybe we (the community) should develop a third party AutoDeleteHostedZone construct? |
Seems to me that the bug is in the ACM teardown. ACM setup that CNAME entry in the first place, when it's deleted it should do the cleanup too. I'd still expect the DNS hosted zone to fail deletion if I've manually added a record that wasn't cleaned up say. In the same way that with an S3 bucket I'd expect it to not clearup if there's still data in it. |
This is extremely frustrating in CI, because the cleanup cannot happen automatically. |
I was able to get around this issue by writing a script that automatically removes the generated CNAME record from a given hosted zone. https://gist.github.com/zacklitzsinger/26051489d05aca19e3c360324028717d |
I just want to highlight @Dzhuneyt: While the verification record should definitely be auto-deleted, we really need an option to delete all records when tearing down the hosted zone. This issue comes up often. |
Similar to @zacklitzsinger, we implemented a custom resource lambda that automatically deletes hosted zone records when the stack is being deleted. We only turn this on in CI/test/development environments though. https://gist.github.com/stevehodgkiss/01241f0ca57e22557f592ee87dc667f9
I'm not sure that's true. AFAIK automatically cleaning up these entries after validation might remove records for other certificates and break renewals for them. I added some references to relevant PRs when I implemented this ~2 months ago. https://gist.github.com/stevehodgkiss/01241f0ca57e22557f592ee87dc667f9#file-stack-usage-ts-L4 Having this option in HostedZone itself would be ideal 👍 |
We used this for the ElasticBeanstalk app which I've deleted __Testing done__ 1. `cdk deploy infraStackProd infraStackBeta` 1. See certifate gone from cfn console 1. Needed to manually delete the CNAME record from the hosted zone, which CDK doesn't manage. See aws/aws-cdk#7063
As a newcomer to this issue, I agree with @strottos that if the ACM setup adds the CNAME, it should delete it. Hopefully this issue can get fixed soon. |
Adds an option to DnsValidatedCertificate to automatically cleanup the related DNS validation records when the Certificate is deleted. This is an opt-in property and discouraged for production use, as there are edge cases that can cause unintended side effects. The most obvious is that if two or more certificates exist with the same domain, the same validation record is used for both. If one certificate is deleted (and deletes the validation record), the second certificate (with the same domain name) will be unable to automatically renew. closes #3333 closes #7063
Adds an option to DnsValidatedCertificate to automatically cleanup the related DNS validation records when the Certificate is deleted. This is an opt-in property and discouraged for production use, as there are edge cases that can cause unintended side effects. The most obvious is that if two or more certificates exist with the same domain, the same validation record is used for both. If one certificate is deleted (and deletes the validation record), the second certificate (with the same domain name) will be unable to automatically renew. closes #3333 closes #7063
…18311) Adds an option to DnsValidatedCertificate to automatically cleanup the related DNS validation records when the Certificate is deleted. This is an opt-in property and discouraged for production use, as there are edge cases that can cause unintended side effects. The most obvious is that if two or more certificates exist with the same domain, the same validation record is used for both. If one certificate is deleted (and deletes the validation record), the second certificate (with the same domain name) will be unable to automatically renew. closes #3333 closes #7063 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
|
…ws#18311) Adds an option to DnsValidatedCertificate to automatically cleanup the related DNS validation records when the Certificate is deleted. This is an opt-in property and discouraged for production use, as there are edge cases that can cause unintended side effects. The most obvious is that if two or more certificates exist with the same domain, the same validation record is used for both. If one certificate is deleted (and deletes the validation record), the second certificate (with the same domain name) will be unable to automatically renew. closes aws#3333 closes aws#7063 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
…ws#18311) Adds an option to DnsValidatedCertificate to automatically cleanup the related DNS validation records when the Certificate is deleted. This is an opt-in property and discouraged for production use, as there are edge cases that can cause unintended side effects. The most obvious is that if two or more certificates exist with the same domain, the same validation record is used for both. If one certificate is deleted (and deletes the validation record), the second certificate (with the same domain name) will be unable to automatically renew. closes aws#3333 closes aws#7063 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
I see this fix was added in v1.141.0, but I'm still experiencing this issue on v1.152.0. Any suggestions? |
@skkrail-amzn I've run into this myself and come across this thread. On your certificate E.g. const certificate = new acm.DnsValidatedCertificate(this, "SiteCertificate", {
domainName: props.domainName,
hostedZone: props.zone,
region: "us-east-1", // Cloudfront only checks this region for certificates.
cleanupRoute53Records: true
}); |
Thanks @runtooctober! I was using |
Ironically |
There should be a similar |
Are there any workaround or update on this? |
Having the same issue with |
This issue is closed as fixed by #18311. Can you please check if the referred fix covers your problem? |
It appears that the current solution only works for the deprecated |
+1 .... This issue is still occurring, please fix. |
Facing the same issue using CloudFormation... Definitely needs a fix |
Experiencing this issue in CF as well. Certificate created in CF yml, and added to a HostedZone, also created in the same yml. If the stack fails to update and has to rollback, the HostedZone cannot be deleted due to the validation CNAME record |
Experiencing this as well |
This is quite frustrating. Would highly appreciate a fix. |
@stevehodgkiss do you still have the example you shared, the links are no more accessible, it would be very much appreciated ! thx |
Also reporting in to say I'm experiencing this issue in CF and it's getting in the way of automating stack teardown. Would really appreciate a fix along the lines of the |
Any update on this? We have also been using a custom resource for this, but there's a bunch of edge cases when you have multiple certificates with the same domain name, and it gets trickier when having multiple SANs. This really should be built in. Someone clearly thought it should, since it was a part of the deprecated DnsValidatedConstruct. |
When I try to delete a stack that has a hosted zone with a certificate created through certificate manager, it has a CNAME entry created by certificate manager that prevents the hosted zone being deleted. Error report attached below.
My case is as follows though it's possible you don't need separate accounts, I haven't checked. I have two accounts, one that has the DNS hosted zone and name servers for my existing domain name (say Account A) and a separate one that I'm creating a DNS hosted zone underneath (say Account B). For example, account A has the Route53 records for my domain example.com and I want to be able to create a subdomain test.example.com in account B that has a valid certificate.
I've got 3 CDK stacks do this, one to create the hosted zone in Account B, one to add the nameservers from that account to Account A and finally one to create the certificate in Account B (see the code samples below). This all works really well for creation but for deletion the certificate manager stack has left a CNAME entry similar to the following in the hosted zone preventing deletion:
I would have expected the certificate manager that created this to remove it as well.
Reproduction Steps
Release the following stacks in order and delete them in reverse order, you'll need to pass in appropriate environment variables.
Stack 1:
Stack 2:
Stack 3:
Error Log
Environment
Other
This is 🐛 Bug Report
The text was updated successfully, but these errors were encountered: