[TEP-0145] CEL in WhenExpression #1074

Yongxuanzhang · 2023-09-28T15:40:32Z

This commit adds the proposal of adding CEL in WhenExpression

Signed-off-by: Yongxuan Zhang [email protected]

Yongxuanzhang · 2023-09-28T15:59:33Z

Hi @tektoncd/core-maintainers ptal at this proposal.
cc @vdemeester @pritidesai thought you might be interested

teps/0145-cel-in-whenexpression.md

JeromeJu · 2023-10-02T15:27:30Z

/assign

jerop · 2023-10-02T16:04:23Z

/kind tep

JeromeJu · 2023-10-02T16:15:36Z

/assign @pritidesai

JeromeJu · 2023-10-02T16:15:46Z

/assign @vdemeester

QuanZhang-William

Thanks @Yongxuanzhang, great TEP! I left some comments, but lgtm in general.

I believe having this feature and Param Enum can improve UX significantly.

teps/0145-cel-in-whenexpression.md

QuanZhang-William · 2023-10-02T22:04:15Z

teps/0145-cel-in-whenexpression.md

+
+The new fields will be gated by a new feature flag, `enable-cel-whenexpression`, which will defaults to `"false"` while the feature is in alpha.
+
+The validation webhook will validate either the current when `input`+`operator`+`values` or `cel` is used, users cannot use both at the same time for one `WhenExpression`.


I'm wondering if the validation webhook can validate if the given CEL expression is syntactically valid or not?

Oh this is a great suggestion! Thanks!

teps/0145-cel-in-whenexpression.md

Yongxuanzhang · 2023-10-12T19:57:53Z

teps/0145-cel-in-whenexpression.md

+    - cel: '$(params.branch)'.matches('release/.*')"
+```
+
+### Variables Substitution


Hi @vdemeester, I added one more section since last time you reviewed. This section will discuss how we handle variable substitution in CEL expression and how we can prevent the attack. PTAL 🙏

Yongxuanzhang · 2023-10-12T19:58:26Z

Hi @pritidesai, @JeromeJu could you review this TEP? Thanks!

teps/0145-cel-in-whenexpression.md

pritidesai · 2023-10-13T00:20:58Z

teps/0145-cel-in-whenexpression.md

+    - input: "$(params.param1)$(params.param2)"
+      operator: notin
+      values: [""]
+```


Right, this is a workaround but @durera was hopping to implement more complex logic but chose to go this route since their desired conditional logic is not supported.

The desired conditional logic is:

whenAny: - input: "$(params.param1)" operator: notin values: [""] - input: "$(params.param2)" operator: in values: ["8.5", "8.6"]

which I think can supported with a single CEL expression:

when: cel: "'$(params.param1)' != '' || '$(params.param2)' == '8.5' || $(params.param2)' == '8.6'"

Do we know if there is any limitation on the length of the cel expression? If so, its worth documenting here.

Thanks! I missed the original one the user wants to use. Updated.

For the length limitation, I have searched a bit but could not find answers...

teps/0145-cel-in-whenexpression.md

pritidesai · 2023-10-13T00:31:09Z

teps/0145-cel-in-whenexpression.md

+
+### Variables Substitution
+
+The CEL in When Expressions should support current Tekton’s Params and Results [string substitutions](https://github.com/tektoncd/pipeline/blob/main/docs/variables.md#variables-available-in-a-pipeline). Array and object substitutions are not supported because Tekton doesn’t support whole array/object replacements in a string.


is this true? values is an array and it does honor the star notation, isn't it?

Its reasonable to implement only string params with CEL but are we supporting indexing into an array or referencing an object param using a key in CEL expressions?

Yes, I should clarify that we cannot reference to a whole array or object.

I remember there is an issue that we cannot reference param in "script", the case where a long string contains param reference.

I will remove this statement, and rephrase that we will support string replacements first

Removed this statement

teps/0145-cel-in-whenexpression.md

pritidesai · 2023-10-13T00:50:52Z

Thanks a bunch @Yongxuanzhang for all the efforts on this. This will be greatly appreciated and utilized by the pipeline authors. I have left a couple of comments. Please address them before we can merge this since this is marked as implementable. Thanks again!

/approve

JeromeJu

The proposal LGTM, just some small questions.

teps/0145-cel-in-whenexpression.md

JeromeJu · 2023-10-13T14:45:05Z

teps/0145-cel-in-whenexpression.md

+- Explicitly document that passing CEL expression from param is not allowed and won’t be executed.
+
+The solution we proposed is to let CEL handle the variable substitution:
+1. Add params, results, context variables to CEL's environment, similar like [Tekton Triggers](https://github.com/tektoncd/triggers/blob/main/pkg/interceptors/cel/cel.go#L104C1-L112)


I think the tekton trigger examples are nice to add. Would you mind also clarifying the environment here?

I added link to the cel environment, it would add too many details here if we want to add code snippets.

teps/0145-cel-in-whenexpression.md

tekton-robot · 2023-10-13T16:09:18Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JeromeJu, pritidesai, vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~teps/OWNERS~~ [pritidesai,vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

teps/0145-cel-in-whenexpression.md

This commit adds the proposal of adding CEL in WhenExpression Signed-off-by: Yongxuan Zhang [email protected]

jerop

Merging offline as we have all the approvals 🚀

/lgtm

Thank you @Yongxuanzhang 🎉

tekton-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 28, 2023

tekton-robot requested review from vtereso and wlynch September 28, 2023 15:40

tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 28, 2023

Yongxuanzhang force-pushed the tep-144 branch 2 times, most recently from a68eaab to 6fcec4a Compare September 28, 2023 15:48

Yongxuanzhang changed the title ~~[TEP-0144] CEL in WhenExpression~~ [TEP-0145] CEL in WhenExpression Sep 28, 2023

Yongxuanzhang force-pushed the tep-144 branch from 6fcec4a to e721063 Compare September 28, 2023 15:52

Yongxuanzhang marked this pull request as ready for review September 28, 2023 15:58

tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 28, 2023

tekton-robot requested review from nikhil-thomas and pritidesai September 28, 2023 15:58

jerop reviewed Sep 29, 2023

View reviewed changes

teps/0145-cel-in-whenexpression.md Outdated Show resolved Hide resolved

Yongxuanzhang force-pushed the tep-144 branch from a005004 to 09a2723 Compare October 1, 2023 16:08

tekton-robot assigned JeromeJu Oct 2, 2023

tekton-robot added the kind/tep Categorizes issue or PR as related to a TEP (or needs a TEP). label Oct 2, 2023

tekton-robot assigned pritidesai Oct 2, 2023

tekton-robot assigned vdemeester Oct 2, 2023

QuanZhang-William mentioned this pull request Oct 2, 2023

[TEP-0138]: Decouple API and Feature Versioning Proposal #1034

Merged

QuanZhang-William reviewed Oct 2, 2023

View reviewed changes

Yongxuanzhang force-pushed the tep-144 branch from 09a2723 to d8e5fe2 Compare October 3, 2023 15:11

vdemeester approved these changes Oct 10, 2023

View reviewed changes

teps/0145-cel-in-whenexpression.md Outdated Show resolved Hide resolved

teps/0145-cel-in-whenexpression.md Outdated Show resolved Hide resolved

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 10, 2023

wlynch removed their request for review October 11, 2023 15:05

Yongxuanzhang force-pushed the tep-144 branch from d8e5fe2 to 239302d Compare October 12, 2023 15:24

Yongxuanzhang force-pushed the tep-144 branch from 239302d to 739fffd Compare October 12, 2023 19:56

Yongxuanzhang commented Oct 12, 2023

View reviewed changes

pritidesai reviewed Oct 12, 2023

View reviewed changes

teps/0145-cel-in-whenexpression.md Outdated Show resolved Hide resolved

jerop reviewed Oct 12, 2023

View reviewed changes

teps/0145-cel-in-whenexpression.md Outdated Show resolved Hide resolved

pritidesai reviewed Oct 13, 2023

View reviewed changes

teps/0145-cel-in-whenexpression.md Show resolved Hide resolved

pritidesai reviewed Oct 13, 2023

View reviewed changes

teps/0145-cel-in-whenexpression.md Outdated Show resolved Hide resolved

pritidesai reviewed Oct 13, 2023

View reviewed changes

teps/0145-cel-in-whenexpression.md Show resolved Hide resolved

pritidesai reviewed Oct 13, 2023

View reviewed changes

teps/0145-cel-in-whenexpression.md Show resolved Hide resolved

Yongxuanzhang force-pushed the tep-144 branch from 739fffd to d75bb83 Compare October 13, 2023 13:39

JeromeJu reviewed Oct 13, 2023

View reviewed changes

Yongxuanzhang force-pushed the tep-144 branch 6 times, most recently from ead28c8 to a035184 Compare October 13, 2023 16:06

JeromeJu approved these changes Oct 13, 2023

View reviewed changes

jerop reviewed Oct 13, 2023

View reviewed changes

teps/0145-cel-in-whenexpression.md Outdated Show resolved Hide resolved

[TEP-0145] CEL in WhenExpression

e27777f

This commit adds the proposal of adding CEL in WhenExpression Signed-off-by: Yongxuan Zhang [email protected]

Yongxuanzhang force-pushed the tep-144 branch from a035184 to e27777f Compare October 13, 2023 17:36

jerop reviewed Oct 13, 2023

View reviewed changes

tekton-robot assigned jerop Oct 13, 2023

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 13, 2023

tekton-robot merged commit 3725139 into tektoncd:main Oct 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TEP-0145] CEL in WhenExpression #1074

[TEP-0145] CEL in WhenExpression #1074

Yongxuanzhang commented Sep 28, 2023

Yongxuanzhang commented Sep 28, 2023

JeromeJu commented Oct 2, 2023

jerop commented Oct 2, 2023

JeromeJu commented Oct 2, 2023

JeromeJu commented Oct 2, 2023

QuanZhang-William left a comment

QuanZhang-William Oct 2, 2023

Yongxuanzhang Oct 3, 2023

Yongxuanzhang Oct 12, 2023

Yongxuanzhang commented Oct 12, 2023

pritidesai Oct 13, 2023

Yongxuanzhang Oct 13, 2023

Yongxuanzhang Oct 13, 2023

pritidesai Oct 13, 2023

Yongxuanzhang Oct 13, 2023

Yongxuanzhang Oct 13, 2023

Yongxuanzhang Oct 13, 2023

pritidesai commented Oct 13, 2023

JeromeJu left a comment

JeromeJu Oct 13, 2023

Yongxuanzhang Oct 13, 2023

tekton-robot commented Oct 13, 2023

jerop left a comment


		The new fields will be gated by a new feature flag, `enable-cel-whenexpression`, which will defaults to `"false"` while the feature is in alpha.

		The validation webhook will validate either the current when `input`+`operator`+`values` or `cel` is used, users cannot use both at the same time for one `WhenExpression`.


		### Variables Substitution

		The CEL in When Expressions should support current Tekton’s Params and Results [string substitutions](https://github.com/tektoncd/pipeline/blob/main/docs/variables.md#variables-available-in-a-pipeline). Array and object substitutions are not supported because Tekton doesn’t support whole array/object replacements in a string.

[TEP-0145] CEL in WhenExpression #1074

[TEP-0145] CEL in WhenExpression #1074

Conversation

Yongxuanzhang commented Sep 28, 2023

Yongxuanzhang commented Sep 28, 2023

JeromeJu commented Oct 2, 2023

jerop commented Oct 2, 2023

JeromeJu commented Oct 2, 2023

JeromeJu commented Oct 2, 2023

QuanZhang-William left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Yongxuanzhang commented Oct 12, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pritidesai commented Oct 13, 2023

JeromeJu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tekton-robot commented Oct 13, 2023

jerop left a comment

Choose a reason for hiding this comment