Data Source template_file causes resource to be recreated each execution.

[kristjanelias]

Hi,

I have upgraded to Terraform v0.7.0 and now refactoring some of my code.
Changed template_file resource to data source but now i am seeing that my resource get regenerated each execution. I will try to illustrate my example the best i can for my concearn to come through.
Use-case: Create a reverse lookup zone in aws route53 using a domain name generated within a template.

• Firstly same use-case using resource template_file to create a string for the name of a route53 domain:

Code:

    variable "vpc_cidr_base" { default = "172.30" }

    resource "template_file" "private_reverse_zone_name" {
      template = "${second}.${first}.in-addr.arpa"
      vars {
        first = "${element(split(".", var.vpc_cidr_base), 0)}"
        second = "${element(split(".", var.vpc_cidr_base), 1)}"
      }
    }

    resource "aws_route53_zone" "reverse_private" {
      name = "${template_file.private_reverse_zone_name.rendered}"
      comment = "${var.vpc_cidr_base} - Reverse Zone"
      vpc_id = "${var.aws_default_vpc_id}"
      tags { Name = "${template_file.private_reverse_zone_name.rendered}" }
    }

Statefile after apply:

    "template_file.private_reverse_zone_name": {
        "type": "template_file",
        "primary": {
            "id": "a25c5ce7e9be18bd4e16d6e1b0915542c441a477b309031aab2fc2c8bc00719d",
            "attributes": {
                "id": "a25c5ce7e9be18bd4e16d6e1b0915542c441a477b309031aab2fc2c8bc00719d",
                "rendered": "30.172.in-addr.arpa",
                "template": "${second}.${first}.in-addr.arpa",
                "vars.%": "2",
                "vars.first": "172",
                "vars.second": "30"
            }
        }
    }

Plan after initial apply:

    My-MacBook-Pro:global kristjan$ ./terraform_run.sh plan
    There are warnings and/or errors related to your configuration. Please
    fix these before continuing.

    Warnings:

      * template_file.private_reverse_zone_name: using template_file as a resource is deprecated; consider using the data source instead

    No errors found. Continuing with 1 warning(s).

    Refreshing Terraform state in-memory prior to plan...

    ...

    No changes. Infrastructure is up-to-date. This means that Terraform
    could not detect any differences between your configuration and
    the real physical resources that exist. As a result, Terraform
    doesn't need to do anything.

As you can see the route53 zone resource will not be regenerated with the next apply.

• Secondly same use-case using data template_file to create a string for the name of a route53 domain:

Code:

      variable "vpc_cidr_base" { default = "172.30" }

      data "template_file" "private_reverse_zone_name" {
        template = "${second}.${first}.in-addr.arpa"
        vars {
          first = "${element(split(".", var.vpc_cidr_base), 0)}"
          second = "${element(split(".", var.vpc_cidr_base), 1)}"
        }
      }

      resource "aws_route53_zone" "reverse_private" {
        name = "${data.template_file.private_reverse_zone_name.rendered}"
        comment = "${var.vpc_cidr_base} - Reverse Zone"
        vpc_id = "${var.aws_default_vpc_id}"
        tags { Name = "${data.template_file.private_reverse_zone_name.rendered}" }
      }

Statefile after apply:

      "data.template_file.private_reverse_zone_name": {
          "type": "template_file",
          "primary": {
              "id": "a25c5ce7e9be18bd4e16d6e1b0915542c441a477b309031aab2fc2c8bc00719d",
              "attributes": {
                  "id": "a25c5ce7e9be18bd4e16d6e1b0915542c441a477b309031aab2fc2c8bc00719d",
                  "rendered": "30.172.in-addr.arpa",
                  "template": "${second}.${first}.in-addr.arpa",
                  "vars.%": "2",
                  "vars.first": "172",
                  "vars.second": "30"
              }
          }
      }

Plan after initial apply:

      My-MacBook-Pro:global kristjan$ ./terraform_run.sh plan
      Refreshing Terraform state in-memory prior to plan...

      ...

      Note: You didn't specify an "-out" parameter to save this plan, so when
      "apply" is called, Terraform can't guarantee this is what will execute.

      -/+ aws_route53_zone.reverse_private
          comment:        "172.30 - Reverse Zone" => "172.30 - Reverse Zone"
          name:           "30.172.in-addr.arpa" => "${data.template_file.private_reverse_zone_name.rendered}" (forces new resource)
          name_servers.#: "4" => "<computed>"
          tags.%:         "2" => "<computed>"
          tags.Name:      "30.172.in-addr.arpa" => ""
          tags.Terraform: "1" => ""
          vpc_id:         "vpc-f31a439a" => "vpc-f31a439a"
          vpc_region:     "eu-central-1" => "<computed>"
          zone_id:        "Z195T1X77U2MBH" => "<computed>"

      <= data.template_file.private_reverse_zone_name
          rendered:    "<computed>"
          template:    "${second}.${first}.in-addr.arpa"
          vars.%:      "2"
          vars.first:  "172"
          vars.second: "30"


      Plan: 1 to add, 0 to change, 1 to destroy.

So you see after switching to data source my route53 zone gets recreated on every apply.

Is this expected behaviour and i am doing something wrong or is this not supposed to be so?
I hope you can help resolving this conundrum...

[apparentlymart]

Hi @kristjanelias! Sorry for this strange behavior.

I think what you've run into here is the issue discussed in #8077. If so, the issue is that your template argument to the template_file data source contains interpolation expressions that Terraform core thinks it needs to resolve in order to instantiate the data source, but in fact your goal was for the Template data source to deal with them. Due to some weird interactions in Terraform core this does eventually work but it leaves Terraform thinking that it has some further work to do in order to resolve the ${first} and ${second} expressions.

The fix is to escape the interpolations in the template string so that Terraform core won't try to parse them. By the time they get passed to the template data source itself, they will have been unescaped and should work as expected:

      variable "vpc_cidr_base" { default = "172.30" }

      data "template_file" "private_reverse_zone_name" {
        template = "$${second}.$${first}.in-addr.arpa"
        vars {
          first = "${element(split(".", var.vpc_cidr_base), 0)}"
          second = "${element(split(".", var.vpc_cidr_base), 1)}"
        }
      }

      resource "aws_route53_zone" "reverse_private" {
        name = "${data.template_file.private_reverse_zone_name.rendered}"
        comment = "${var.vpc_cidr_base} - Reverse Zone"
        vpc_id = "${var.aws_default_vpc_id}"
        tags { Name = "${data.template_file.private_reverse_zone_name.rendered}" }
      }

I'm going to close this issue to consolidate discussion around #8077, but do feel free to re-open it if the above workaround doesn't work or you think that there's more here than what #8077 covers.

[kristjanelias]

Thanks @apparentlymart, workaround worked like a charm!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.