WI: ensure Consul hook and WID manager interpolate services

[tgross]

Services can have some of their string fields interpolated. The new Workload Identity flow doesn't interpolate the services before requesting signed identities or using those identities to get Consul tokens.

Add support for interpolation to the WID manager and the Consul tokens hook by providing both with a taskenv builder. Add an "interpolate workload" field to the WI handle to allow passing the original workload name to the server so the server can find the correct service to sign.

This changeset also makes two related test improvements:

• Remove the mock WID manager, which was only used in the Consul hook tests and isn't necessary so long as we provide the real WID manager with the mock signer and never call Run on it. It wasn't feasible to exercise the correct behavior without this refactor, as the mocks were bypassing the new code.
• Fixed swapped expect-vs-actual assertions on the consul_hook tests.

Fixes: #20025

---

Smoke tested with the following jobspec:

jobspec

job "example" {

  group "web" {

    network {
      mode = "bridge"
      port "metrics" {
        to = 9000
      }
      port "www" {
        to = 8001
      }
    }

    service {
      name = "${NOMAD_GROUP_NAME}"
      port = "metrics"
    }

    task "http" {

      driver = "docker"

      service {
        name = "${NOMAD_TASK_NAME}"
        port = "www"
      }

      config {
        image   = "busybox:1"
        command = "httpd"
        args    = ["-vv", "-f", "-p", "8001", "-h", "/local"]
        ports   = ["www"]
      }

      resources {
        cpu    = 100
        memory = 100
      }

    }
  }
}

[schmichael]

The security implications of this do make me nervous. As you and @gulducat discussed offline there doesn't appear to be any immediate concern, but security issues often arise from unintended side effects down the road.

I wonder if we should deprecate this form of interpolation in this place in favor of HCL variables? I think an HCL variable with a default value could specify both the group and service names in a way that would make them static by the time they're submitted to Nomad. It would be nice to have a path away from this extremely buggy phased interpolation approach.

[schmichael]

This is the second time allocrunner uses taskenv.Builder in a unique way compared to taskrunner (nil Task, in the other place it's used in a closure). Might be nice to reevaluate the old Builder design to see if we can split up the alloc and task concerns and make it less dangerous to hold.

[schmichael]

I love to see a mock removed!

[tgross]

I wonder if we should deprecate this form of interpolation in this place in favor of HCL variables? I think an HCL variable with a default value could specify both the group and service names in a way that would make them static by the time they're submitted to Nomad. It would be nice to have a path away from this extremely buggy phased interpolation approach.

I love this idea! Right now interpolation can include data that really is allocation-specific or node-specific, like node metadata or the IP address. But there's probably some subsets of fields that we could say "you're allowed to interpolate these fields with node metadata, and these fields only with HCL2". Being able to document which those are would be nice for users and for us developers. I'll open a new issue for that.

[tgross]

@schmichael I've opened #20362 to follow up on the design of interpolation more generally.