Proposal: Tekton Workflows

[dibyom]

This is a proposal for an experimental higher level resource that would group the lower level Tekton primitives into a single entity. Conceptually, this would be similar to workflow in GitHub Actions and Argo Workflows, and Pipeline in Jenkins, CircleCI, and Gitlab.

The problem the project will solve

The main goal is to simplify the CI/CD configuration for end users. Tekton today has different un-opinionated primitives. While this is very flexible, it adds additional burden on end users looking for simpler CI/CD use cases. Platform developers tend to abstract away this complexity by adding abstraction layers that only expose basic inputs to the app developers.

Within Tekton itself, there have been a few experiments in this area. The Generators project combines trigger and pipeline definitions into a higher level opinionated syntax that is aware of external systems such as GitHub. The Pipelines as Code project uses PipelineRuns with annotations to describe Triggers. Similarly, Tekton Integrations proposed adding annotations for notifications and status updates to external systems. There was also a proposal for a concept of "Project" that will combine Triggers and Pipelines into a single view.

Similarly, many other CI/CD systems have an abstraction where primitives such as Triggers, and Pipelines are combined into one resource e.g. workflow in GitHub Actions and Argo Workflows, and Pipeline in Jenkins, CircleCI, and Gitlab.

Who Will Own It

@dibyom
@bobcatfish

[vdemeester]

@dibyom @bobcatfish few questions

• Would this be an opiniated setup/use of the different TektonCD components ?
• Should we aim to be the solution we use for dogfooding or not ? (cc @afrittoli)
• Can this be started as an experimental project ? If not (or even if), should we have a TEP for it ? If we go experimental, I wouldn't require it, if we want to make it a project on its own, I would require it I think.
• How does it relates to "Pipelines as Code", or others experimentation in that sense ? Also, would it be a full "distribution" (aka you install it, it installs everything) or a component that sits on top of the others and let the "installation/configuration" part up to the user (or something else, like the operator 🙃 )

[nikhil-thomas]

/unassign
and subscribe instead 😌

[dibyom]

Would this be an opiniated setup/use of the different TektonCD components ?

Yes, some similarities with https://github.com/openshift-pipelines/pipelines-as-code

Should we aim to be the solution we use for dogfooding or not ? (cc @afrittoli)

Absolutely, dogfooding would be a great use case!

Can this be started as an experimental project ? If not (or even if), should we have a TEP for it ? If we go experimental, I wouldn't require it, if we want to make it a project on its own, I would require it I think.

Yes! this is for an experimental project but I do plan on opening up a TEP so that we can discuss the actual API in more detail

How does it relates to "Pipelines as Code", or others experimentation in that sense ? Also, would it be a full "distribution" (aka you install it, it installs everything) or a component that sits on top of the others and let the "installation/configuration" part up to the user (or something else, like the operator 🙃 )

Good question -- pipelines as code also groups triggers + other options together (using annotations). So in that sense it is similar. The concept of repo CRD is also very interesting -- would be interesting to explore a generic eventSource type thing perhaps for Workflows. I think this would be less of a distribution and more of a component (something like the operator could do the install/configuration).

I think the pipelines as code WG that @chmouel proposed would be a good forum to explore these questions.

[dibyom]

According to the Proposing Projects process, we need two governing board members to approve before we can create an experimental project.

/cc @vdemeester @afrittoli @abayer @bobcatfish

[bobcatfish]

👍 from me!

[vdemeester]

👍🏼

[dibyom]

Awesome! I'll go ahead with setting up the experimental folder!

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[afrittoli]

/remove-lifecycle stale

[afrittoli]

/lifecycle frozen This is an ongoing experimental project

[vdemeester]

/lifecycle frozen

[dibyom]

Status Update - we started a Working Group leading to a proposed design (join mailing list for access) and an ongoing experimental implementation in https://github.com/tektoncd/experimental/tree/main/workflows

Join the #workflows channel on our Slack if you are interested in Workflows!