aws_eks_cluster_auth should not cache token in plan

[llamahunter]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.24

Affected Resource(s)

• aws_eks_cluster_auth

Terraform Configuration Files

data "aws_eks_cluster" "example" {
  name = "example"
}

data "aws_eks_cluster_auth" "example" {
  name = "example"
}

provider "kubernetes" {
  host                   = "${data.aws_eks_cluster.example.endpoint}"
  cluster_ca_certificate = "${base64decode(data.aws_eks_cluster.example.certificate_authority.0.data)}"
  token                  = "${data.aws_eks_cluster_auth.example.token}"
  load_config_file       = false
}

Debug Output

From terraform apply:

Error: Unauthorized

  on .terraform/modules/prometheus_operator/modules/prometheus-operator/main.tf line 36, in resource "kubernetes_namespace" "this":
  36: resource "kubernetes_namespace" "this" {

From EKS authorization log:

time="2020-05-06T05:17:59Z" level=warning msg="access denied" client="127.0.0.1:55512" error="input token was not properly formatted: X-Amz-Date parameter is expired (15 minute expiration) 2020-05-06 01:09:00 +0000 UTC" method=POST path=/authenticate

Note the very stale X-Amz-Date token (2020-05-06 01:09:00Z) relative to the current time (2020-05-06T05:17:59Z). The token date corresponds to the time at which the terraform plan was run, several hours earlier.

Panic Output

n/a

Expected Behavior

data.aws_eks_cluster_auth.example.token should be refreshed on apply. Authentication tokens should not be cached as part of the plan.

Actual Behavior

data.aws_eks_cluster_auth.example.token is cached in the plan, and attempted to be reused later on apply. But, tokens only have validity for 15 minutes.

Steps to Reproduce

1. terraform plan and save plan output
2. wait more than 15 minutes
3. terraform apply existing cached plan

Important Factoids

We are using atlantis to apply terraform plans. From the command line, we didn't notice this problem because we don't cache the plans, but re-generate on every apply. Atlantis, however, caches the plans for a more rigorous workflow.

References

This issue is sort of related, but not really.

• Inconsistent behaviour of aws_eks_cluster_auth token #10362

This issue is sort of the reverse problem for the kubernetes provider, maybe?

• data.kubernetes_service should fetch values in "plan" phase rather than "apply" phase terraform-provider-kubernetes#353

[bflad]

Hi @llamahunter 👋 Just wanted to drop a quick note here that Terraform Providers (like the Terraform AWS Provider) cannot control this behavior themselves. Terraform Core, which manages the ordering of dependencies and handling of saved plan files, would need changes to optionally support the lack of caching for data sources in a saved plan file. You may want to consider checking that repository's issues to see if this has been requested or otherwise raise a new feature request.

If saved plans are required, then you may need to perform the authentication outside of Terraform and use a Terraform or environment variable.

[llamahunter]

For atlantis workflows, you kind of want saved plans. It is unfortunate that the aws provider eks data objects aren't compatible with that. It's undesirable to have to configured this outside of terraform, since all the data exists within terraform, but not in a useable way.

I'll go poke the terraform core people and see whether they are willing to fix this, or have a workaround that the aws provider can use to not cache credentials in the plan.

[bflad]

Closing this in preference of the upstream issue, since this is not something we can adjust in the Terraform AWS Provider itself: hashicorp/terraform#24886

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!