Error with expr from parameter output another step

[DiegoFlop]

Checklist

• Double-checked my configuration.
• Tested using the latest version.
• Used the Emissary executor.

Summary

What happened/what you expected to happen?

When on a when field add an expr with output task I get the next error:

Failed: invalid spec: templates.main when doesn't support 'expr' expression '{{='. 'When' expression is only support govaluate format {{

the expr I'm using is:

when: "{{=jsonpath(tasks.checkout.outputs.parameters.custom_config, '$.steps.test_unit.skip')}}"

However when I use and input parameter with expr everything works correctly, but I don't have use an input parameter because, I need use the output parameter to another task and I get a json file.

when: "{{=inputs.parameters.BRANCH == 'false'}}"

Which this expr works correctly

What version are you running?

Argo-workflows: 3.3.6
Kubernetes: 1.22.9

Diagnostics

Paste the smallest workflow that reproduces the bug. We must be able to run the workflow.

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: example
  entrypoint: main
  templates:
    - name: main
      inputs:
        parameters:
          - name: URL
          - name: BRANCH 
      dag:
        tasks:
        - name: checkout
          templateRef:
            name: checkout
            template: main
          arguments:
            parameters:
              - name: git_ssh_url
                value: "git@{{= sprig.replace('net/', 'net:', sprig.substr( 8, -1, inputs.parameters.URL)) }}"
              - name: BRANCH
                value: "{{inputs.parameters.BRANCH}}"
...
...
        - name: test-unit
          templateRef:
            name: unit-test
            template: node-web
          arguments:
            parameters:
              - name: PROJECT_NAME
                value: "{{workflow.parameters.PROJECT_NAME}}"
# Get value for testing
              - name: SKIP
                value: "{{=jsonpath(tasks.checkout.outputs.parameters.custom_config, '$.steps.test_unit.skip')}}"
          when: "{{=jsonpath(tasks.checkout.outputs.parameters.custom_config, '$.steps.test_unit.skip') == 'false'}}"

...

kubectl logs -n argo-workflows deploy/workflow-controller | grep example-db6tm

# If the workflow's pods have not been created, you can skip the rest of the diagnostics.

# The workflow's pods that are problematic:
kubectl get pod -o yaml -l workflows.argoproj.io/workflow=example-db6tm,workflow.argoproj.io/phase!=Succeeded

# Logs from in your workflow's wait container, something like:
kubectl logs -c wait -l workflows.argoproj.io/workflow=example-db6tm,workflow.argoproj.io/phase!=Succeeded
time="2022-06-03T20:18:11.654Z" level=info msg="Processing workflow" namespace=argo-events workflow=example-db6tm
time="2022-06-03T20:18:11.789Z" level=info msg="Updated phase  -> Failed" namespace=argo-events workflow=example-db6tm
time="2022-06-03T20:18:11.789Z" level=info msg="Updated message  -> invalid spec: templates.main when doesn't support 'expr' expression '{{='. 'When' expression is only support govaluate format {{" namespace=argo-events workflow=example-db6tm
time="2022-06-03T20:18:11.789Z" level=info msg="Marking workflow completed" namespace=argo-events workflow=example-db6tm
time="2022-06-03T20:18:11.789Z" level=info msg="Marking workflow as pending archiving" namespace=argo-events workflow=example-db6tm
time="2022-06-03T20:18:11.790Z" level=error msg="Unable to set ExecWorkflow" error="templates.main when doesn't support 'expr' expression '{{='. 'When' expression is only support govaluate format {{" namespace=argo-events workflow=example-db6tm
time="2022-06-03T20:18:11.790Z" level=info msg="Checking daemoned children of " namespace=argo-events workflow=example-db6tm
time="2022-06-03T20:18:11.795Z" level=info msg="cleaning up pod" action=deletePod key=argo-events/example-db6tm-1340600742-agent/deletePod
time="2022-06-03T20:18:11.813Z" level=info msg="Workflow update successful" namespace=argo-events phase=Failed resourceVersion=49713004 workflow=example-db6tm
time="2022-06-03T20:18:11.824Z" level=info msg="archiving workflow" namespace=argo-events uid=b2e5b49b-db7d-427c-9859-dec6d546fc40 workflow=example-db6tm
time="2022-06-03T20:18:11.871Z" level=info msg="Queueing Failed workflow argo-events/example-db6tm for delete in 1m0s due to TTL"
apiVersion: v1
items: []
kind: List
metadata:
  resourceVersion: ""
No resources found in argo-events namespace.

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

[sarabala1979]

@DiegoFlop can you try without {{=

when: "jsonpath(tasks.checkout.outputs.parameters.custom_config, '$.steps.test_unit.skip') == 'false'"

[DiegoFlop]

Hi, @sarabala1979
Thanks for your answer. I try with you said me but when execute workflow, on the step that use that output. Now I get this error:

Invalid 'when' expression 'jsonpath(tasks.checkout.outputs.parameters.custom_config, '$.steps.unit_test.skip') == 'false'': Unable to access unexported field 'checkout' in token 'tasks.checkout.outputs.parameters.custom_config'

The field 'checkout' is the first step of workflow.

I changed my workflow for execute a python script to export multiple file as output parameters and so this I can continue to another step, but this make that more complex.

Really, I would like the use of output parameters from other steps to work correctly in the "when" clause.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

[tooptoop4]

crunchy

[DiegoFlop]

Any idea how to solve this?

[sarabala1979]

@DiegoFlop let me look this and update you

[sarabala1979]

@dpadhiar Can you look at this issue?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

[DiegoFlop]

Any news on this issue?

[vermaxik]

Is it any plan fix this issue? 🙏

[vermaxik]

@DiegoFlop could you please share with us workaround that you mentioned above? Thank you in advance 🤗

[DiegoFlop]

@vermaxik In my case only use this feature to skip steps.

During firts step I create a config file, however with expr doesn't work for my case. But how I only need get if boolean attribute is True or False, I create others files only with this attribute value.

Example:
I create the file /tmp/step1_skip.txt, on this file on write True or False.
and with outputs parameters get this value of file and next steps Use this output on when field to skip this step.

step0

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: step0
spec:
  entrypoint: main
  templates:
    - name: main
      inputs:
        parameters:
          - name: git_ssh_url
          - name: BRANCH
        artifacts:
          - name: git-clone
            path: "/workspace/project"
            git:
              revision: "{{inputs.parameters.BRANCH}}"
              repo: "{{inputs.parameters.git_ssh_url}}"
              singleBranch: false
              sshPrivateKeySecret:
                name: github
                key: id_rsa
              insecureIgnoreHostKey: true
            optional: true
      outputs:
        parameters:
        - name: custom
          valueFrom:
            path: "/workspace/custom.json"
        - name: step1_skip
          description: "Skip step1 step"
          valueFrom:
            path: "/tmp/step1_skip"
        - name: step2_skip
          description: "Skip step2 step"
          valueFrom:
            path: "/tmp/step2_skip"
      container:
        imagePullPolicy: Always
        image: xxxxxxx:1.0.0
        workingDir: "/workspace/project"
        resources:
          limits:
            cpu: 500m
            memory: 512Mi
          requests:
            cpu: 200m
            memory: 256Mi
        command:
          - "python"
        args:
          - "/my-script.py"
          - "param1"

main workflow

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: pipeline
spec:
  entrypoint: main 
  templates:
    - name: main
      dag:
        tasks:
        - name: step0
          templateRef:
            name: step0
            template: main
          arguments:
            parameters:
              - name: git_ssh_url
                value: "XXXXXXXXXX"
              - name: BRANCH
                value: "XXXXXXXXX"
        - name: step1
          templateRef:
            name: step1
            template: main
          depends: "step0"
          when: "{{tasks.checkout.outputs.parameters.step0_skip}} =~ '^(false|False)$'"
....

[sarabala1979]

@DiegoFlop Core Argo team is busy with 3.4 feature completion. We will take a look at this issue once we have completed the feature work. do you like to contribute to fixing this issue?

[DiegoFlop]

@sarabala1979
I'd like to contribute with this project, but unfortunately, my knowledge with golang are very limited at this moment

[sarabala1979]

@DiegoFlop are you working on it? Can I get ETA for this?

[juliev0]

@DiegoFlop Can you provide me a full set of files I can use to reproduce this on my end? I might be missing something but in the first example, I don't believe it includes the WorkflowTemplate "checkout" or "unit-test" being referenced. Also, "tasks.checkout.outputs.parameters.custom_config" is referenced but I don't see that output parameter defined. In the second example I don't think your "step1" WorkflowTemplate is included, right? And "tasks.checkout" is referenced but I don't see that task.

[juliev0]

I spoke to my colleague, and currently expr expressions are not supported for when conditions. We will open an Enhancement Issue related to that instead, and I can post the link here.

Please let us know if there is something that you can only do with expr and can't do with govaluate.

[juliev0]

Added this issue, which as I've been told was the actual root cause behind why the validation logic prevents expr expressions in the "when" condition.

[alexec]

Issue: #6257
PR: fix(controller): Randomly expr expression fail to resolve (#6410)

[alexec]

I think the fix for this would be to revert c29b275. It is that validation that is preventing its use.
The downside of this would be the removal of a helpful warning, but I can't see another way to fix this.

[alexec]

It is trivial git revert to fix this, but you would need to test the fix works for you.

[juliev0]

I think the fix for this would be to revert c29b275. It is that validation that is preventing its use. The downside of this would be the removal of a helpful warning, but I can't see another way to fix this.

Please see the PR - I have reverted it and fixed the underlying cause based on discussion with Bala