Variable all the things!

[jeff-mccoy]

Move component-level variables to the package level and allow package string-templating for the Zarf.yaml and package variables templated out manifests.

Create-time variables

Zarf.yaml string templating

• Validation must require uppercase and snake case
• special type of string replace that can only be specified during package create
• do not appear in the variables or constants lists
• can only template string values in a zarf.yaml
• run after component composition so cannot change component import paths or names
• check for all matching variables and error if one is not set by the user on package create

components:
  - name: zarf-agent
    required: true
    images:
      - "###ZARF_VAR_AGENT_IMAGE###"
    manifests:
      - name: zarf-agent
        namespace: zarf

Deploy-time constants

Constant declaration by package authors

• validation must require uppercase and snake case for all variables
• specified as an array under the constants key
• only set during package create
• cannot be overriden during package deploy
• are used with ZARF_CONST prefix

constants:
  # Must be all-caps, snake-case
  - name: AGENT_IMAGE
    value: "defenseunicorns/zarf-agent:v0.19.5"

In the manifest will appear as ###ZARF_CONST_AGENT_IMAGE###

Deploy-time variables

Variable declaration by package deployers

• validation must require uppercase and snake case for all variables
• specified as an array under the variables key
• set via default key
• set via--set flags on package deploy
• user prompt on package deploy if prompt: true
• are used with ZARF_VAR_ prefix

variables:
  # Must be all-caps, snake-case
  - name: AGENT_IMAGE
    default: "defenseunicorns/zarf-agent:v0.19.5"

  # This variable will template manifests on deploy and will prompt if the user doesn't specify by flag
  - name: PROMPT_ON_DEPLOY
    prompt: true

In the manifest will appear as ###ZARF_VAR_AGENT_IMAGE### and ###ZARF_VAR_PROMPT_ON_DEPLOY###

Issue planning coordinated with @RothAndrew @YrrepNoj @Racer159

[jeff-mccoy]

@Racer159 added some more details to the issue if you want to pick this one up

[Racer159]

@jeff-mccoy @RothAndrew @YrrepNoj is this worthy of and ADR as to why we chose to go with variables at the package level (vs component level) and why we changed from simple key-value pair map to a list of sub-objects?

[RothAndrew]

I think the quick and easy answer is if it was worth asking that question then the answer likely should be yes, but happy to hear other opinions

[jeff-mccoy]

I don't think it warrants an adr as we haven't done one for any other package structure changes (of which there have been dozens) since we aren't GA. It certainly should be explained in the PR (or issue) why we changed the variable placement.

We currently have ADRs for large architecture decisions such as the injector system, major test replacement and the mutating webhook. This doesn't seem like that level of change to me.

[Racer159]

@jeff-mccoy @RothAndrew @YrrepNoj how did you want to handle variables in nested packages/components?

I see two potential paths:

1. Force package developers to declare all variables within the root package
2. Dynamically build a list of variables and allow each imported package to provide its own list of variables (and if the package that imported it provided an override the imported package would not change that)

[Racer159]

Path 2 has been chosen

[Racer159]

I am guessing that this is "expected" behavior for zarf prepare find-images? (to show the ZARF_VAR instead of what it by default templates to?

[jeff-mccoy]

Yeah it's funny looking, but I don't think we'd want to change that. I would note though, the package should not include the REGISTRY variables as that is specified in the manifest. If it is specified in the zarf.yaml, Zarf will try to pull from a non-existent image to create the package.

[Racer159]

Yeah it makes sense since you don't know what that might be overridden with later (and it is nice to know that it is dynamic in the output). Was mostly just poking around/asking because there are no tests covering it right now. And yes those builtins I left prefixed with ZARF_ alone (no VAR) so they cannot be overridden by a zarf.yaml change. The PR on this is pretty close. I just have to handle prompts on deploy and the validation checks in code (they are already in the JSON Schema)

[jeff-mccoy]

Yeah I don't think we have hardly any test coverage for zarf prepare or zarf tools...though most of zarf tools is just vendored code too.

[jeff-mccoy]

Updated the issue with all the tweaks we discussed during our meeting earlier @Racer159 @YrrepNoj @Madeline-UX

[Racer159]

@jeff-mccoy @Madeline-UX @YrrepNoj close to having this ready I think, but thoughts on differentiating the create variables even further with something like ###ZARF_PKG_AGENT_IMAGE### or ###ZARF_PKG_VAR_AGENT_IMAGE### so that people know these variables are different from deploy time ###ZARF_CONST_AGENT_IMAGE###

[jeff-mccoy]

Yeah I'm certainly happy with being more explicit with this. Not sure which format is better, no strongly held opinion as I still see it as an overall edge case.

[Racer159]

Going with PKG_VAR for now then

[Madeline-UX]

@Racer159 @YrrepNoj and myself got together to document the current pain points, desired outcomes, Solutions, and updated user flows for this enhancement. This design references functional users (user defined by action they are trying to complete) Not to be confused with personas. There may be multiple personas that fit each functional role.

For reference to the living document see Miro >> https://miro.com/app/board/uXjVOUJ4f2s=/?moveToWidget=3458764529916551885&cot=14

*living documents may be updated in the future to reflect new information.

Here is a screen capture of that miro board at the time of this PR being reviewed.
PUBLIC - Zarf Project - PR 621 - Move Variable definitions from components to packages.pdf