TEP-0087: Custom Tasks Graduation #523

jerop · 2021-09-28T20:28:58Z

This change adds the Tekton Enhancement Proposal for Custom Tasks Graduation.

Today, Custom Tasks shared by the Tekton community are all experimental, and we don't have a process to promote
them beyond experimental.

As such:

Users can't depend on the Custom Tasks because they can change any time, they may not reflect the Tekton roadmap and they may not meet the Tekton standards because they haven't been reviewed in a Tekton Enhancement Proposal.
Contributors don't have a process to stabilize their Custom Tasks or integrate them to the Tekton Pipelines API.

In this TEP, we aim to:

Define the graduation requirements path for Custom Tasks.
Provide Custom Tasks that are officially supported by Tekton and have stability guarantees.

Custom Tasks will have four stability levels in their graduation path:

Experimental: Custom Tasks can change any time, alpha API policy applies, as we are iterating on
them.
Stable: Custom Tasks are stable, beta API policy applies, and have been approved in a TEP.
Packaged: Custom Tasks are shipped with Tekton Pipelines releases, so are available for use out of the box.
Integrated: Custom Tasks' functionalities are natively supported in the Tekton Pipelines API.

The Tekton Pipelines owners have the discretion to expedite the graduation process of a given Custom Task, such as
when they agree early on that they want to integrate the functionality provided by the Custom Task directly to the
Tekton Pipelines API.

See also TEP-0002: Enable Custom Tasks.

jerop · 2021-09-28T20:29:46Z

/kind tep

teps/0087-custom-tasks-graduation.md

afrittoli

Thanks @jerop , great work! I like the two tier approach, with lower bar for the first level.
I have a few questions on the integrated level.

teps/0087-custom-tasks-graduation.md

jerop · 2021-10-04T16:09:41Z

/assign @afrittoli @sbwsg @vdemeester @pritidesai

jerop · 2021-10-04T16:20:47Z

/assign @mattmoor

tekton-robot · 2021-10-04T16:20:48Z

@jerop: GitHub didn't allow me to assign the following users: mattmoor.

Note that only tektoncd members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @mattmoor

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bobcatfish · 2021-10-05T19:07:06Z

/assign

teps/0087-custom-tasks-graduation.md

This change adds the Tekton Enhancement Proposal for `Custom Tasks` Graduation. Today, `Custom Tasks` shared by the Tekton community are all *experimental*, and we don't have a process to promote them beyond *experimental*. As such: - Users can't depend on the `Custom Tasks` because they can change any time, they may not reflect the Tekton roadmap and they may not meet the Tekton standards because they haven't been reviewed in a Tekton Enhancement Proposal. - Contributors don't have a process to stabilize their `Custom Tasks` or integrate them to the *Tekton Pipelines API*. In this TEP, we aim to: - Define the graduation requirements path for `Custom Tasks`. - Provide `Custom Tasks` that are officially supported by Tekton and have stability guarantees. `Custom Tasks` will have four stability levels in their graduation path: 1. *Experimental*: `Custom Tasks` can change any time, [*alpha* API policy][api-policy] applies, as we are iterating on them. 2. *Stable*: `Custom Tasks` are stable, [*beta* API policy][api-policy] applies, and have been approved in a TEP. 3. *Packaged*: `Custom Tasks` are shipped with *Tekton Pipelines* releases, so are available for use out of the box. 4. *Integrated*: `Custom Tasks`' functionalities are natively supported in the *Tekton Pipelines API*. The *Tekton Pipelines* owners have the discretion to expedite the graduation process of a given `Custom Task`, such as when they agree early on that they want to integrate the functionality provided by the `Custom Task` directly to the *Tekton Pipelines API*. See also [TEP-0002: Enable Custom Tasks](https://github.com/tektoncd/community/blob/main/teps/0002-custom-tasks.md). [api-policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md#alpha-beta-and-ga

jerop · 2021-10-21T16:08:32Z

@sbwsg @afrittoli @bobcatfish, thank you for the initial reviews -- this tep is ready for another look :)

afrittoli

Nice, thank you for all the updates!
/approve

ghost · 2021-10-21T19:07:53Z

Strong +1 to this given dependence on custom tasks by downstream projects like kubeflow. Changes since my last read all look good to me. Thanks @jerop !

/approve

tekton-robot · 2021-10-21T19:08:02Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: afrittoli, sbwsg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~teps/OWNERS~~ [afrittoli]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

vdemeester

Reading this, I have a slight trouble understanding if it's about a set of particular Custom Task we want to bring in Tekton Pipelines or about a process around graduation.

Do we assume all Custom Task that are proposed in the tekton org (starting in the experimental repository) aim to be shipped as part of tektoncd/pipeline releases or integrated directly in the API (and thus fade away) ? I don't think we should make such an assumption, but maybe I am wrong ? 🤔
As commented below, I also have a little bit of trouble with stable and graduation here — for me, there are slightly unrelated, at least if we speak about "API stability" and graduation process. One (API stability) is about if the API will not break the next release, the other is about the "community" commitment on supporting a given piece of code (Custom Task here)

vdemeester · 2021-10-25T15:53:12Z

teps/0087-custom-tasks-graduation.md

+- Users can't depend on the `Custom Tasks` because they can change any time, they may not reflect the Tekton roadmap and
+  they may not meet the Tekton standards because they haven't been reviewed in a Tekton Enhancement Proposal.
+- Contributors don't have a process to stabilize their `Custom Tasks` or integrate them to the *Tekton Pipelines API*.


I think one main aspect is that, being in experimental, it's tricky to get a release out. If we were able to easily release custom task, with "stuck in time" payloads, it would be a little bit more decent for user to use them (but still, doesn't prevent from us to define a process)

vdemeester · 2021-10-25T15:54:26Z

teps/0087-custom-tasks-graduation.md

+
+In this TEP, we aim to:
+- Define the graduation requirements path for `Custom Tasks`.
+- Provide `Custom Tasks` that are officially supported by Tekton and have stability guarantees.


I am not sure what this means ? We aim to provide a list of Custom Tasks that are officially supported ?
If yes, I find it weird as this will change over time.

vdemeester · 2021-10-25T15:56:28Z

teps/0087-custom-tasks-graduation.md

+1. *Experimental*: `Custom Tasks` can change any time, [*alpha* API policy][api-policy] applies, as we are iterating on 
+them.
+2. *Stable*: `Custom Tasks` are stable, [*beta* API policy][api-policy] applies, and have been approved in a TEP.
+3. *Packaged*: `Custom Tasks` are shipped with *Tekton Pipelines* releases, so are available for use out of the box.
+4. *Integrated*: `Custom Tasks`' functionalities are natively supported in the *Tekton Pipelines API*.


This gives the impression that all Custom Tasks will or need to end-up in Tekton Pipelines API to be seen as stable and really usable, which, I think, is not what we should aim for.

vdemeester · 2021-10-25T16:00:13Z

teps/0087-custom-tasks-graduation.md

+We need to promote some `Custom Tasks` to the top level such that they are shipped with *Tekton Pipelines* releases, 
+or even more, that their functionality is supported natively in the *Tekton Pipelines API*. On the other hand, 
+we may decide not to natively support some functionalities provided in some `Custom Tasks` but we need to provide 
+stability guarantees. Thus, we need to provide incremental stability levels and graduation path for `Custom Tasks`. 


From my point of view, Custom Tasks do not need to be shipped with Tekton Pipelines to reach "stability". Some might, but most wouldn't need to. To reach stability, they need to be able to do releases as they want and have a stable API (**/v1 for example).
Be shipped as part of Tekton Pipelines (or even be "phased out" in the API) is yet another level of stability or integration.

(note: to be able to do releases, they might just need the proper infra or their own repository)

vdemeester · 2021-10-25T16:04:18Z

teps/0087-custom-tasks-graduation.md

+1. As a contributor, I want to stabilize my `Custom Tasks`, and possibly integrate them into the *Tekton Pipelines API*. 
+As such, I need a process that I can follow with clear requirements for graduation to the next stability level.
+2. As a user, I need to use `Custom Tasks` that I can rely on. The stability expectations can vary depending on the 
+requirements based on the use cases. 


I have that weird feeling we are talking stability and graduation on the same level. Nothing prevents any Custom Task to be stable, today, on tektoncd/experimental or outside of it. As stated above, you just need a stable API, most likely a k8s-like api policy some automated tests set-up and a way to do release — but all this is already achievable today.

Nothing prevents any Custom Task to be stable, today, on tektoncd/experimental or outside of it.

one caveat is that if the authors of the Custom Task want the custom task to graduate, they may need to change the API, so in that sense, even if the Custom Task has been declared GA within experimental, the API might need to change once it's considered for graduation. So to me that makes it feel like stability and graduation are at least related - or can influence each other?

They defenitely can influence each other, but they are not cause/consequence. If by graduation we mean having its own repository, maybe they wouldn't change their API at all. Also, if they went v1beta1 or v1 on the API while being in experimental, I don't really see why they would want to make breaking change in there (because they went beta or stable for a reason).

vdemeester · 2021-10-25T17:03:53Z

teps/0087-custom-tasks-graduation.md

+1. It has a TEP that has been approved.
+2. It is tested and has nightly releases.


vdemeester · 2021-10-25T17:05:06Z

teps/0087-custom-tasks-graduation.md

+
+1. Add documentation for installation and provide examples of usage.
+2. Make nightly releases of the `Custom Task` to catch any failures.
+3. Make monthly releases with notes to drive adoption of the `Custom Task`.


This goes with the rest of the assumptions, but, similarly, I don't see any "obvious" reason to decide a release cadence for a Custom Task.

vdemeester · 2021-10-25T17:05:52Z

teps/0087-custom-tasks-graduation.md

+1. Add documentation for installation and provide examples of usage.
+2. Make nightly releases of the `Custom Task` to catch any failures.
+3. Make monthly releases with notes to drive adoption of the `Custom Task`.
+4. To make changes to the *stable* `Custom Task`:


What kind of changes are we talking about here ? API changes, or any changes (bugfix, refactoring, behaviour, …) ?

vdemeester · 2021-10-25T17:09:49Z

teps/0087-custom-tasks-graduation.md

+### Packaged
+
+When we find that a given *stable* `Custom Task` is necessary for common Continuous Delivery use cases, we can consider 
+making it *packaged*. That is, when *Tekton Pipelines* is installed, the `Custom Task` is available with no extra step. 
+The *packaged* `Custom Task` and its functionality is available out of the box with *Tekton Pipelines* installation. 
+Not all *stable* `Custom Tasks` have to be *packaged* - a *stable* `Custom Task` that is not necessary in 
+*Tekton Pipelines* can stay as a *stable* `Custom Task` in the long term. 


We should try to put the operator more into our stories — but that's on us (Operator owners). It is relatively easy to ship a "tiny-bit" opiniated setup of the tekton components with the Operator. We could decide to package Custom Task as part of the Operator so that we don't need to add more complexity to the Pipeline release process and payload.

For example, if we add a given Custom Task to the pipeline releases, it will be installed by default even — consuming resources — even if the user has no need for it (because its pipeline usage doesn't require it in any way). A user would have to "manually" remove the custom task part, and do that on each updates — or we would have to provide different payloads. The operator here is at a slightly higher level and allow to easily setup and manage Pipeline instance as well as other components, Custom Task being seen as components here.

vdemeester · 2021-10-25T17:13:57Z

teps/0087-custom-tasks-graduation.md

+
+###### Cons
+
+- Limiting for users who don't install and manage Tekton installations through the *Tekton Operator*.


That's a good point, but, probably a question of perspective, I wouldn't put this in a Cons (or a very limited Cons). If the Operator becomes the way to easily install and managed a Tekton instance — Pipeline being just one, main, part of the overall setup — then it's not a Cons, it just is.

bobcatfish · 2021-10-25T21:48:04Z

Some thoughts after reading @vdemeester 's feedback:

I'm not sure it makes sense for experimental projects to become stable - the goal of the experimental repo was to allow projects to "incubate" (https://github.com/tektoncd/community/blob/main/process.md#proposing-projects) - if a project in experimental wants to provide stability guarantees I'm not sure it should continue to live there. I wonder if there might be a different "graduation path" where the project moves out of Tekton entirely?
I agree a Custom Task can be complete and stable without ever "graduating" (but maybe not in the experimental repo ^^) - I think "graduating" is about accepting a Custom Task as part of Tekton in some way; and since experimental project APIs don't require TEP review, I think there is a relationship between the stability of the Custom Task's API and Graduation - if the goal of the Custom Task is to graduate

vdemeester · 2021-10-26T05:36:20Z

I'm not sure it makes sense for experimental projects to become stable - the goal of the experimental repo was to allow projects to "incubate" (https://github.com/tektoncd/community/blob/main/process.md#proposing-projects) - if a project in experimental wants to provide stability guarantees I'm not sure it should continue to live there. I wonder if there might be a different "graduation path" where the project moves out of Tekton entirely?

I agree it makes little sense to have experimental projects to become stable. Usually, even before they become stable, if we think they have value (to maintain), we "graduate" them. A good example here is the Hub 👼🏼. One of the idea of the experimental repository, in my head, is to experiment on something the community think it might be worth exploring and maybe maintaining (in any kind of form) — so the general path to graduation would be to end up in a tektoncd project (in the org or a "friend" org if we had that). If, after experimentation, we fill, we don't really want to maintain it, but it still has value from users, they can definitely extract it into their own org and maintain it (and most likely mark it as deprecated in the experimental repository) — nothing should prevent anyone to do that. Extracting to "outside of tekton entirely" is not really a graduation path, just a "the community might not want to maintain this so we (owners) will elsewhere".

I agree a Custom Task can be complete and stable without ever "graduating" (but maybe not in the experimental repo ^^) - I think "graduating" is about accepting a Custom Task as part of Tekton in some way; and since experimental project APIs don't require TEP review, I think there is a relationship between the stability of the Custom Task's API and Graduation - if the goal of the Custom Task is to graduate

There is a relation for sure but, as of written, it makes it (at least how I read it) like if the goal of any Custom Task added in experimental has a aim to be shipped as part of tektoncd/pipeline ; and I disagree with this 😅 . I think the aim is, as you said, be part of tekton in some way, aka be maintained by the Tekton community — it doesn't have to be integrated into tektoncd/pipeline to be useful, successful or valuable. Going from experimental to its own repository and going from somewhere (experimental, another-repo, it-s-own-repo) to be shipped and/or integrated in the tekton pipeline API are two different thing, and in my eyes two different "graduation" process. One says "we value this, and the tekton community wil maintain it and drive it towards stability", the other is "this will be shipped and integrate into tekton pipeline" — you can want one without the other.

In all honesty, this means the first graduation I am talking about (from experimental to somewhere), is not really tied to Custom Task, and is a general experimental graduation process. The other is Custom Task based but it should be clear that you don't have to aim for being integrated or shipped as part of tektoncd/pipeline to become a fully successful Custom Task 👼🏼 .

bobcatfish · 2021-10-26T17:39:16Z

like if the goal of any Custom Task added in experimental has a aim to be shipped as part of tektoncd/pipeline ; and I disagree with this 😅 .

Makes sense to me @vdemeester and I agree! I wonder if part of the reason the TEP is coming across that way is because the motivation has been to progress custom tasks that we (at least @jerop and i haha) are interested in adding to the Pipelines API XD

So 👍 to updating the proposal to make this clear

pritidesai · 2021-11-01T16:13:24Z

API WG - @jerop is updating the TEP - in progress

tekton-robot · 2021-11-14T12:13:27Z

@jerop: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot · 2022-02-12T13:10:28Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

pritidesai · 2022-02-14T17:13:46Z

API WG - @jerop to revisit this soon

tekton-robot · 2022-03-16T17:14:29Z

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

jerop · 2022-03-16T17:18:18Z

/lifecycle frozen

jerop · 2022-05-02T20:01:51Z

hoping to revisit someday when I have the capacity, closing for now

tekton-robot requested review from dlorenc and hrishin September 28, 2021 20:29

tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 28, 2021

tekton-robot added the kind/tep Categorizes issue or PR as related to a TEP (or needs a TEP). label Sep 28, 2021

jerop force-pushed the custom-tasks-graduation branch 4 times, most recently from dc2d254 to 776f0aa Compare September 29, 2021 19:50

jerop changed the title ~~TEP-0087: Custom Tasks Graduation [Problem Statement]~~ TEP-0087: Custom Tasks Graduation Sep 29, 2021

jerop force-pushed the custom-tasks-graduation branch 3 times, most recently from 627ff1d to 46e1184 Compare September 30, 2021 00:43

ghost reviewed Sep 30, 2021

View reviewed changes

teps/0087-custom-tasks-graduation.md Outdated Show resolved Hide resolved

teps/0087-custom-tasks-graduation.md Outdated Show resolved Hide resolved

teps/0087-custom-tasks-graduation.md Outdated Show resolved Hide resolved

teps/0087-custom-tasks-graduation.md Outdated Show resolved Hide resolved

jerop force-pushed the custom-tasks-graduation branch 4 times, most recently from a34f9f9 to 5dfaae7 Compare October 1, 2021 01:41

afrittoli reviewed Oct 1, 2021

View reviewed changes

teps/0087-custom-tasks-graduation.md Outdated Show resolved Hide resolved

tekton-robot assigned afrittoli, pritidesai, ghost and vdemeester Oct 4, 2021

tekton-robot assigned bobcatfish Oct 5, 2021

jerop force-pushed the custom-tasks-graduation branch from 5dfaae7 to 4ad1c3f Compare October 7, 2021 23:01

bobcatfish reviewed Oct 8, 2021

View reviewed changes

tekton-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 21, 2021

jerop force-pushed the custom-tasks-graduation branch from fce4227 to f051bd6 Compare October 21, 2021 16:07

afrittoli reviewed Oct 21, 2021

View reviewed changes

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 21, 2021

vdemeester reviewed Oct 25, 2021

View reviewed changes

tekton-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 14, 2021

tekton-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Feb 12, 2022

otaviof mentioned this pull request Mar 18, 2022

SHIP-0031: Shipwright Trigger shipwright-io/community#68

Merged

jerop closed this May 2, 2022

jerop deleted the custom-tasks-graduation branch June 11, 2022 01:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TEP-0087: Custom Tasks Graduation #523

TEP-0087: Custom Tasks Graduation #523

jerop commented Sep 28, 2021 •

edited

Loading

jerop commented Sep 28, 2021

afrittoli left a comment

jerop commented Oct 4, 2021

jerop commented Oct 4, 2021

tekton-robot commented Oct 4, 2021

bobcatfish commented Oct 5, 2021

jerop commented Oct 21, 2021 •

edited

Loading

afrittoli left a comment

ghost commented Oct 21, 2021

tekton-robot commented Oct 21, 2021

vdemeester left a comment

vdemeester Oct 25, 2021

vdemeester Oct 25, 2021

vdemeester Oct 25, 2021

vdemeester Oct 25, 2021

vdemeester Oct 25, 2021

vdemeester Oct 25, 2021

bobcatfish Oct 25, 2021

vdemeester Oct 26, 2021

vdemeester Oct 25, 2021

vdemeester Oct 25, 2021

vdemeester Oct 25, 2021

vdemeester Oct 25, 2021

vdemeester Oct 25, 2021

bobcatfish commented Oct 25, 2021

vdemeester commented Oct 26, 2021 •

edited

Loading

bobcatfish commented Oct 26, 2021

pritidesai commented Nov 1, 2021

tekton-robot commented Nov 14, 2021

tekton-robot commented Feb 12, 2022

pritidesai commented Feb 14, 2022

tekton-robot commented Mar 16, 2022

jerop commented Mar 16, 2022

jerop commented May 2, 2022

		1. It has a TEP that has been approved.
		2. It is tested and has nightly releases.


		###### Cons

		- Limiting for users who don't install and manage Tekton installations through the Tekton Operator.

TEP-0087: Custom Tasks Graduation #523

TEP-0087: Custom Tasks Graduation #523

Conversation

jerop commented Sep 28, 2021 • edited Loading

jerop commented Sep 28, 2021

afrittoli left a comment

Choose a reason for hiding this comment

jerop commented Oct 4, 2021

jerop commented Oct 4, 2021

tekton-robot commented Oct 4, 2021

bobcatfish commented Oct 5, 2021

jerop commented Oct 21, 2021 • edited Loading

afrittoli left a comment

Choose a reason for hiding this comment

ghost commented Oct 21, 2021

tekton-robot commented Oct 21, 2021

vdemeester left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bobcatfish commented Oct 25, 2021

vdemeester commented Oct 26, 2021 • edited Loading

bobcatfish commented Oct 26, 2021

pritidesai commented Nov 1, 2021

tekton-robot commented Nov 14, 2021

tekton-robot commented Feb 12, 2022

pritidesai commented Feb 14, 2022

tekton-robot commented Mar 16, 2022

jerop commented Mar 16, 2022

jerop commented May 2, 2022

jerop commented Sep 28, 2021 •

edited

Loading

jerop commented Oct 21, 2021 •

edited

Loading

vdemeester commented Oct 26, 2021 •

edited

Loading