BUG: Inconsistent behaviour variable precendence / merge behaviour

[stefreak]

Bug

According to https://docs.garden.io/using-garden/variables-and-templating#module-variables we have a clear variable precedence, but we also have some merge behaviour that we explain by linking to an RFC (Can we also have some examples?)

I find the behaviour strange when comparing it with the documentation. Also, we are merging variables in the varfile with project-level, but we are not merging variables in the module-level with project-level.

Reproducible example

project.garden.yml

kind: Project
name: incident-repro
variables:
  one:
    config:
      foo: foo
      bar: bar
environments:
  - name: plain
  - name: varfile
    varfile: varfile
  - name: varfile-yml
    varfile: varfile.yml
  - name: varfile-yml-obj
    varfile: varfile-obj.yml
  - name: varfile-yml-empty
    varfile: varfile-empty.yml

varfile

one:
  config:
    foo: "baz"

varfile.yml

one:
  config:
    foo: "baz"

varfile-empty.yml

one:
  config:

varfile-obj.yml

one:
  config: {}

module.garden.yml

kind: Module
type: exec
name: vars
build:
  command: [echo, "${var.one.config.foo}", "${var.one.config.bar}"]
#variables:
#  one:
#    config:
#      foo: baz

Current Behavior

% garden build -f -l3
ℹ vars                      → [echo]: foo bar
✔ vars                      → [echo]: foo bar → Done (took 0 sec)

% garden build -f --env varfile
Failed resolving one or more modules:

vars: Invalid template string (${var.one.config.foo}): Attempted to look up key "config" on a string.

% garden build -f --env varfile-yml -l3
ℹ vars                      → [echo]: baz bar
✔ vars                      → [echo]: baz bar → Done (took 0 sec)

% garden build -f --env varfile-yml-obj -l3
ℹ vars                      → [echo]: foo bar
✔ vars                      → [echo]: foo bar → Done (took 0 sec)

% garden build -f --env varfile-yml-empty -l3
Failed resolving one or more modules:

vars: Invalid template string (${var.one.config.foo}): Attempted to look up key "foo" on a object.

Compare this to the behaviour when uncommenting the variables in module.garden.yml

-#variables:
-#  one:
-#    config:
-#      foo: baz
+variables:
+  one:
+    config:
+      foo: baz

% garden build -f -l3
ℹ graph                     → Resolving 1 modules...
Failed resolving one or more modules:

vars: Invalid template string (${var.one.config.foo}): Could not find key bar under var.one.config. Available keys: foo.

Expected behavior

Clear and consistent variable precendence behaviour that can be explained in a few sentences without ambiguity

Especially if we allow merging vars from variable files with project-level vars, I think we should also allow merge of module-level vars with project-level vars.

Workaround

Suggested solution(s)

Additional context

Your environment

garden version 0.12.48-edge-2d8bf03e5

[stefreak]

User question in Discord community about this: https://discord.com/channels/817392104711651328/1067641774731956235/1067641774731956235

[stefreak]

Might be a good idea to fix the behaviour in 0.13 actions as it might be a breaking change.

[salotz]

A fix for this would be nice :)

I have the problem that for an action I have to specify a necessary varfile for injecting a secret (which is handled automatically in my exec provider), but there is also a site-configurable aspect I want to have in another varfile.

My options are:

1. add it to the secret varfile e.g. _tls.yaml
2. add another varfile and require it always be there (not good since defaults are set in my action already)
3. always use the --var command (this quickly becomes impractical in interactive work, e.g garden deploy --var a=1 thing; garden test test-thing will redeploy thing without the vars)