docs: Use IRSA for Node Termination Handler IAM policy attachement in Instance Refresh example

[stevehipwell]

PR o'clock

Description

I've updated the example for how to use instance refresh.

• Removed policy attachment to worker security group
• Added spot instance termination support
• Enable lifecycle for all worker groups
• Rename Terraform resources to make it clearer

Checklist

• CI tests are passing
• README.md has been updated after any changes to variables and outputs. See https://github.com/terraform-aws-modules/terraform-aws-eks/#doc-generation

[stevehipwell]

/assign @barryib

[barryib]

Awesome ❤️ ! Thanks @stevehipwell for your contribution.

[bashims]

@stevehipwell I believe the example was already using IRSA according to aws-node-termination-handler's logs:

│ 2021/05/19 01:53:08 DBG AWS Credentials retrieved from provider: WebIdentityCredentials                                                             │
│ 2021/05/19 01:53:08 INF Started watching for interruption events                                                                                    │
│ 2021/05/19 01:53:08 INF Kubernetes AWS Node Termination Handler has started successfully!                                                             

Thanks for removing the unnecessary policy attachment!

[stevehipwell]

@bashims it was but the policy was also attached to the worker SG, I assumed it was a typo. Thanks for the great work on getting the instance refresh integrated and released.

[bashims]

This change seems to break the instance_refresh example:

Error: Invalid for_each argument

  on main.tf line 220, in resource "aws_autoscaling_lifecycle_hook" "aws_node_termination_handler":
 220:   for_each = toset(module.eks.workers_asg_names)

The "for_each" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the for_each depends on.

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.