Deleting an eks.Cluster may fail with DependencyViolation on nodeSecurityGroup

[t0yv0]

What happened?

In certain configurations pulumi destroy fails to delete the eks.Cluster with an error message resembling the following:

    aws:ec2:SecurityGroup (example-cluster-py-3-nodeSecurityGroup):
      error: deleting urn:pulumi:p-it-fv-az1210--cluster-py-37ea6979::example-cluster-py::eks:index:Cluster$aws:ec2/securityGroup:SecurityGroup::example-cluster-py-3-nodeSecurityGroup: 1 error occurred:
      	* deleting Security Group (sg-0e8b6d9244b79c723): DependencyViolation: resource sg-0e8b6d9244b79c723 has a dependent object
      	status code: 400, request id: 1b401e09-4dda-4402-8a4e-95c43e03dbec

The workaround is to retry the operation at a later time.

Example

The issue surfaces in this repo CI periodically see #1222 for example

Output of pulumi about

pulumi-eks 2.7.3 is confirmed to be affected (https://github.com/pulumi/pulumi-eks/releases/tag/v2.7.3)

Additional context

• Delete daemonset(s) as part of cluster deletion eksctl-io/eksctl#4214
• https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/faq.md#what-steps-can-be-taken-to-mitigate-this-issue
• Integrate with Karpenter #1114
• DependencyViolation: resource sg has a dependent object #194

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[flostadler]

Retrying at a later time won’t help if the ENI is dangling. Those will need to be manually cleaned up.

[flostadler]

Duplicate of: #779