String templates as values

[apparentlymart]

This is a design prototype of one possible way to meet the need of defining a template in a different location from where it will ultimately be evaluated, as discussed in #26838. For example, a generic module might accept a template as an input variable and then evaluate that template using values that are private to the module, thus separating the concern of providing the data from the concern of assembling the data into a particular string shape.

The idea here is to establish a new type of value in the Terraform language that can store a template that has been parsed but not yet evaluated. This is similar in principle to a normal string containing template syntax passed into the deprecated template_file data source, but allows use of the real template syntax at the definition site and allows for early evaluation of syntax. It also, unlike a string passed to template_file, allows the template to refer to a mixture of template arguments and other values from the surrounding scope.

---

For a small example I put together something admittedly-contrived around constructing log line prefixes. This particular use-case probably doesn't make sense to be done at the Terraform level because log lines tend to incorporate very dynamic data about the specific system generating them, but it serves to show the overall mechanics of this design nonetheless.

The module which consumes the template looks like this:

variable "log_prefix_template" {
  type = template({
    level    = string
    app_name = string
    pid      = number
  })
}

output "log_prefix" {
  value = evaltemplate(var.log_prefix_template, {
      app_name = "awesome-app"
      level    = "DEBUG"
      pid      = 2546
  })
}

This shows two different aspects of this proposed design:

• The new template type constraint syntax, which represents accepting a template that will have a particular set of arguments available to it.
• The new evaltemplate function, which takes a value of a template type and an object whose attributes conform to the template's argument types and evaluates the template, returning the resulting string.

The actual definition of the template is left to the calling module, whose source code might look something like this:

variable "env_name" {
  type    = string
  default = "PROD"
}

module "child" {
  source = "./child"

  log_prefix_template = maketemplate("${template.app_name}.${var.env_name} ${template.pid}: [${template.level}]")
}

output "result" {
  value = module.child
}

Again, this shows an aspect of the proposed design:

• The new maketemplate function, which parses its argument as a string template (using a custom argument parser technique, similar to how Terraform currently implements the try and can functions) and saves it along with the evaluation scope inside a value of a template type, which it returns.

The maketemplate function treats its argument as a normal string template, except that it additionally supports special symbols with the template. prefix, which serve as the template's arguments. Notice that the log_prefix_template template above includes both template.-prefixed references and a normal var.env_name prefix, which allows it to create a blend of values known both in the calling module and in the called module.

The result of my contrived configuration here is the following:

Outputs:

result = {
  "log_prefix" = "awesome-app.PROD 2546: [DEBUG]"
}

---

Although this valid example doesn't really show it, having an explicit type for templates (as opposed to just passing them as strings with special characters inside) allows various validity checks on both the side of the template definer and the template evaluator, helping both to cooperate to produce a working result. For example, if I change the template to include a reference to a template argument that the module doesn't provide, we can catch that early at the call site:

╷
│ Error: Invalid value for module argument
│ 
│   on templatevals.tf line 10, in module "child":
│   10:   log_prefix_template = maketemplate("${template.appp_name}: ")
│ 
│ The given value is not suitable for child module variable "log_prefix_template" defined at child/templatevals-child.tf:2,1-31: incompatible
│ template arguments.
╵

Or, if I change the template to use one of the arguments in a way that isn't compatible with the argument types the module defines:

╷
│ Error: Invalid value for module argument
│ 
│   on templatevals.tf line 10, in module "child":
│   10:   log_prefix_template = maketemplate("${template.app_name["boop"]}: ")
│ 
│ The given value is not suitable for child module variable "log_prefix_template" defined at child/templatevals-child.tf:2,1-31: incompatible
│ usage of template arguments.
╵

On the evaluator side, similarly we can catch if the templateeval call is missing one of the expected arguments or if one has been assigned an incompatible type, even if the currently-passed template value doesn't make use of all of the arguments or if we're just validating the shared module in isolation:

╷
│ Error: Invalid function argument
│ 
│   on child/templatevals-child.tf line 11, in output "log_prefix":
│   11:   value = evaltemplate(var.log_prefix_template, {
│   12:       app_name = "awesome-app"
│   13:       level    = "DEBUG"
│   14:   })
│ 
│ Invalid value for "args" parameter: attribute "pid" is required.
╵

(The basic error messages above are just placeholders for more detailed messages we should implement if we decide to move forward with this.)

---

This is only just enough to try out the behavior and see how it feels and how intuitive it seems. If we were to move forward with something like this then a real implementation would need some extra care to some other concerns, including but not limited to:

• Generating good error messages when templates don't make valid use of their arguments.

• Properly blocking template values from being used in places where they don't make sense, such as as input to jsonencode(...), as a root module output value, or directly as an argument to a provider. (In all of these situations, it's necessary to first evaluate the template and pass the resulting string.)

• Possibly another function similar to templatefile which can create a template value from a file on disk, to allow factoring out larger templates while still being compatible with modules that expect template values.

• Defining default values for variables with template type constraints given that functions aren't available in the default argument, and thus it wouldn't naturally work to call maketemplate from in there.

There are undoubtedly more things which we'd find with further prototyping and experimentation, if this design seems to have promise.

Before deciding whether to move forward with anything like this we'll need to evaluate it against various use-cases and see which ones it supports and which it doesn't. It doesn't necessarily need to solve all use-cases related to templating, but in order to be a satisfying replacement for template_file it should at least address use-cases involving a template being defined in a different module than where it's being evaluated.

[hashicorp-cla]

All committers have signed the CLA.

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.