Multiple instances of service job with different variables?

[sbrl]

Proposal

I have a job file that has a number of variables. It is not a batch job - for example it's a file browser job with a variable for the folder to use. At the moment, Nomad won't let me run multiple instances of the same service job with different var files:

nomad job run -var-file=a.vars filebrowser.nomad
nomad job run -var-file=b.vars filebrowser.nomad # Overwrites the previous one :-(

Use-cases

See above. For example, a file browser template that we want to re-use to serve multiple different directories.

Attempted Solutions

I looked at parameterising my job, but that only works for batch jobs, and this is a service job.

I tried interpolating a variable into the job name and the group name, but Nomad didn't like either of these at all

[jrasell]

Hi @sbrl and thanks for raising this issue. The problem you are facing it seems is that you need to deploy a version of the job specification with substituted variables, but most importantly, with a variable job name so you avoid updating the job in-place that is already running. The inability to use variables within HCL block labels is a design constraint within the HCL language; additional information can be found within issue #9522.

It would be possible to run some form of text manipulation prior to submitting the job using the Nomad CLI; sed or awk would be the likely choices here. Alternatively, the new nomad-pack project would solve this problem by allowing you to create a single "pack" which holds your fully templated job. Among a number of templating features, nomad-pack supports templating every aspect of the job specification. You could then house this template within a private registry or locally for use. The nomad-pack-community-registry has links to useful documentation on how to write packs, as well as a number of good packs to use as inspiration.

[Ramblurr]

Now with nomad-pack in preview, will this issue lose priority? It seems nomad-pack provides an opinionated workaround to this issue.

We really love terraform + nomad + hcl2, it gets us mostly everything we need, except the ability to re-use a job spec for different jobs.

[jrasell]

Hi @Ramblurr; the issue as I mentioned in my previous comment is that HCL does not allow interpolation of block labels which therefore impacts the Nomad job specification job name and group names. This is a core HCL parser constraint and therefore would take a fair amount of work and testing to change (if we could) in order to ensure it does not impact any project that utilises HCL, particularly Terraform.

The existence of nomad-pack does not impact any priority we originally have on this downstream issue.

I will close this issue out in favour of #9522. Please feel free to add any relevant comments there.

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.