depends_on should wait for template_dir generation

[ghost]

This issue was originally opened by @aosman-tibco as hashicorp/terraform#23684. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Terraform Version

Terraform v0.12.18
+ provider.aws v2.41.0
+ provider.template v2.1.2

Terraform Configuration Files

resource "template_dir" "policies" {
  source_dir      = var.policies
  destination_dir = "${path.cwd}/policies"

  vars = {
    email   = var.recipient_email
    sqs_url = aws_sqs_queue.foo.id
  }
}

resource "aws_s3_bucket_object" "policies" {
  depends_on  = [template_dir.policies]
  for_each    = fileset(template_dir.policies.destination_dir, "**/*")

  bucket      = data.aws_s3_bucket.this.id
  key         = "policies/${each.value}"
  source      = "${template_dir.policies.destination_dir}/${each.value}"
  etag        = filemd5("${template_dir.policies.destination_dir}/${each.value}")

}

Expected Behavior

Templates are generated and aws_s3_bucket_object is aware of them (causing them to be uploaded)

Actual Behavior

Templates are generated and aws_s3_bucket_object is not aware of them, therefore no action is taken. A second apply is required to upload the generated files.
If the generated files are deleted, the aws_s3_bucket_object detects the source files as missing then then deletes the resources (even if the files are generated again). A second apply is required to upload the generated files.

Steps to Reproduce

1. terraform init
2. terraform apply

Additional Context

References

• depends_on should should defer interpolation terraform#22036

[eosman-tibco]

Moving relevant comment over:

I took a second look at the template provider documentation for template_dir and it indicates that it should be populating the directory before another resource consumes it:

destination_dir - The destination directory given in configuration. Interpolate this attribute into other resource configurations to create a dependency to ensure that the destination directory is populated before another resource attempts to read it.

There is a note prior to that to not use the file function due to the risk of evaluating the path before the destination directory has been updated, in this instance fileset is being used so it would be good to update the doc to include that if it falls under this limitation.

That said, I would very much like to not have this limitation as otherwise it's not clear how to even use template_dir in conjunction with other terraform resources.

[eosman-tibco]

As a test I rewrote it to use local_file to generate it in conjunction with templatefile and the issue was reproduced:

resource "local_file" "policies" {
  for_each = fileset(var.policies, "**/*")
  filename = "${path.cwd}/policies/${each.value}"

  sensitive_content = templatefile("${var.policies}/${each.value}", {
    email   = var.recipient_email
    sqs_url = aws_sqs_queue.foo.id
  })
}

resource "aws_s3_bucket_object" "policies" {
  depends_on = [local_file.policies]
  for_each    = fileset("${path.cwd}/policies/", "**/*")

  bucket      = data.aws_s3_bucket.this.id
  key         = "policies/${each.value}"
  source      = "${path.cwd}/policies/${each.value}"
  etag        = filemd5("${path.cwd}/policies/${each.value}")

}

[eosman-tibco]

This is a decent workaround:

resource "aws_s3_bucket_object" "policies" {
  for_each    = fileset(var.policy_templates, "**/*")

  bucket      = data.aws_s3_bucket.this.id
  key         = "policies/${each.value}"
  content     = templatefile("${var.policy_templates}/${each.value}", {
    email   = var.recipient_email
    sqs_url = aws_sqs_queue.foo.id
  })

}