[TEP-0074] Deprecate PipelineResources ☠️ #480
Conversation
tekton-robot
added
the
do-not-merge/work-in-progress
tekton-robot
added
the
size/L
bobcatfish
force-pushed
the
tep_deprecate_pipelineresources
branch
from
1270a3e to
c82f79c
Compare
|
Note that we are still using cc @afrittoli |
Good point @vdemeester - maybe I can include in here that as part of this proposal we try updating our dogfooding to use alternatives like the experimental pipeline -> taskrun custom task as a POC that this is a valid alternative suggestion? AND we get to get some use of the experimental custom task at the same time :D |
bobcatfish
force-pushed
the
tep_deprecate_pipelineresources
branch
2 times, most recently
from
28a9445 to
40980de
Compare
tekton-robot
removed
the
do-not-merge/work-in-progress
|
It's ready for review!!!! 🎉 |
|
@bobcatfish CI failing with below error |
|
/test pull-community-teps-lint |
|
/assign |
tekton-robot
added
the
needs-rebase
| * When we create v1 versions of Tasks, Pipelines, TaskRuns and PipelineRuns, all alpha features should be behind | ||
| [the alpha api flag](https://github.com/tektoncd/pipeline/blob/main/docs/install.md#customizing-the-pipelines-controller-behavior) | ||
| * Our documentation and examples should clearly recommend best practices for how to use Tekton Pipelines components | ||
| * Our dogfooding should provide guidelines for how we recommend using Tekton |
There was a problem hiding this comment.
Do we want to have as requirements that any TEP that is listed above (and implementing parts of what PipelineResource was providing) should be implemented and out of alpha ? (aka enabled by default)
There was a problem hiding this comment.
That's a good question - would you want that to be a requirement before we announce our intention to deprecate PipelineResources, or would that be a requirement for their final removal?
- One path could be that we approve this TEP as "implementable", announce the deprecation, add it to our v1 plans, but we make their actual removal (and therefore the v1 release!) contingent on these other TEPs making it to at least alpha. (Perhaps we include that the TEPs should at least be accepted as proposed? which TEP-0044 (decoupling scheduling) is, but the other two (dict + array support) are not)
- The other path would be to maintain our current state (PipelineResources not recommended but with their future somewhat ambiguous) until those other TEPs make it to at least Alpha
I feel like (1) has the disadvantage of giving our users somewhat conflicting info: this thing is deprecated, the replacement(s) are a WIP, but (2) has the disadvantage of continuing this limbo state where we're not giving users a clear idea of our plans. So my preference (and there are definitely some selfish reasons here but I'll try to be impartial XD) would be to go with (1)
There was a problem hiding this comment.
+1 to option (1) - if we know this is the direction we want to go it seems reasonable to warn users ASAP so they don't adopt something we plan to remove in the near future (even if the alternative isn't as available / as friendly yet).
There was a problem hiding this comment.
option (1) seems more appropriate indeed, and announcing the PipelineResources are deprecated is more or less what we unofficially say (at least I do 😅 ) so this would make it official. But I think we should aim for condition the "removing them" (CRD not in new release / fresh installs) to "having any feature required to support the use case PipelineResource were out of alpha" (aka available on a default install) — what weird sentence lol. If we don't have that, indeed the message is going to be weird : "you could do that before, now you have to enable alpha api to do the same".
So I vote for marking this implementable, announcing the deprecation, but adding a condition to the actual removal, which would be "the required set of feature (list of TEP) is implemented, and out-of beta for X release".
There was a problem hiding this comment.
Okay after a big delay, I've tried to update this explicitly in the TEP in a "Requirements to mark as implementable" section!
As part of TEP-0074 (tektoncd/community#480) I'm trying to update some of the plumbing use of PipelineResources to not use them anymore. I picked this yamllint pipeline as a simple example; I thought I'd need the experimental pipeline to taskrun controller, but in this case conditions are in the mix - when expressions are the way to update those but they aren't supported in the pipeline to taskrun controller yet. Also the original version would run 3 separate pods, and each would separately clone the git resource. So in this updated version, even though it will run 4 separate pods, it will only clone the git repo once which seems like an optimization. And once we support when expressions in the pipeline to taskrun controller (seems like it could be easy enough? they execute sequentially after all!) this could all happen in one pod. Anyway this seems like an okay first step; next I'll tackle something that can use the experimental pipeline to taskrun controller. I tested this by setting up my own eventlistener with the same bindings and trigger template on my own repo and it seems to work - I didn't have all of the event triggering setup tho so the real test will be live :D
As part of TEP-0074 (tektoncd/community#480) I'm trying to update some of the plumbing use of PipelineResources to not use them anymore. I picked this yamllint pipeline as a simple example; I thought I'd need the experimental pipeline to taskrun controller, but in this case conditions are in the mix - when expressions are the way to update those but they aren't supported in the pipeline to taskrun controller yet. Also the original version would run 3 separate pods, and each would separately clone the git resource. So in this updated version, even though it will run 4 separate pods, it will only clone the git repo once which seems like an optimization. And once we support when expressions in the pipeline to taskrun controller (seems like it could be easy enough? they execute sequentially after all!) this could all happen in one pod. Anyway this seems like an okay first step; next I'll tackle something that can use the experimental pipeline to taskrun controller. I tested this by setting up my own eventlistener with the same bindings and trigger template on my own repo and it seems to work - I didn't have all of the event triggering setup tho so the real test will be live :D
bobcatfish
force-pushed
the
tep_deprecate_pipelineresources
branch
from
40980de to
a542253
Compare
tekton-robot
removed
the
needs-rebase
bobcatfish
force-pushed
the
tep_deprecate_pipelineresources
branch
from
a542253 to
cb140e1
Compare
|
/approve LGTM. I suspect that this will be something we come back to and take another look at, but removing to simplify the v1 surface while we figure out the rest SGTM. |
tekton-robot
added
the
approved
|
/assign @sbwsg |
|
/assign |
tekton-robot
added
the
needs-rebase
As part of TEP-0074 (tektoncd/community#480) I'm trying to update some of the plumbing use of PipelineResources to not use them anymore. I picked this yamllint pipeline as a simple example; I thought I'd need the experimental pipeline to taskrun controller, but in this case conditions are in the mix - when expressions are the way to update those but they aren't supported in the pipeline to taskrun controller yet. Also the original version would run 3 separate pods, and each would separately clone the git resource. So in this updated version, even though it will run 4 separate pods, it will only clone the git repo once which seems like an optimization. And once we support when expressions in the pipeline to taskrun controller (seems like it could be easy enough? they execute sequentially after all!) this could all happen in one pod. Anyway this seems like an okay first step; next I'll tackle something that can use the experimental pipeline to taskrun controller. I tested this by setting up my own eventlistener with the same bindings and trigger template on my own repo and it seems to work - I didn't have all of the event triggering setup tho so the real test will be live :D
vdemeester
added
the
kind/tep
|
I'm a bit unclear - is this TEP ready for another review or are there more changes that still need to be incorporated? FWIW this looks good to me and I'm in favor of PipelineResources' removal. Looks like it needs a non-Googler approval at this point. |
As part of TEP-0074 (tektoncd/community#480) I'm trying to update some of the plumbing use of PipelineResources to not use them anymore - and even further I want to start dogfooding [the experimental pipeline to taskrun custom task](https://github.com/tektoncd/experimental/tree/main/pipeline-to-taskrun). So to that end I've found a candiate Task that was using PipelineResources and I've converted it to use a Pipeline instead. I've also been able to run this successfully as a single pod on my own cluster so once I have the pipeline to taskrun controller installed in dogfooding I can come back and update this to run as a pod as well!
bobcatfish
force-pushed
the
tep_deprecate_pipelineresources
branch
from
cb140e1 to
e19b46f
Compare
tekton-robot
removed
the
needs-rebase
bobcatfish
force-pushed
the
tep_deprecate_pipelineresources
branch
from
e19b46f to
172e3a2
Compare
|
Sorry for the delay and confusion @sbwsg and everyone! This is finally ready for another look:
PTAL |
bobcatfish
force-pushed
the
tep_deprecate_pipelineresources
branch
from
172e3a2 to
3327aa9
Compare
| * As a user of Tekton Pipelines v1, I want to be able to: | ||
| * Understand clearly the stability level of all components I'm using |
There was a problem hiding this comment.
nit: If we use "Tekton Pipelines" there, it's pretty much "one component" isn't it ? (or component here isn't tektoncd/… but tektoncd/pipeline resources ?)
There was a problem hiding this comment.
maybe instead of "component" explicitly saying something like "resource" or "CRD" is more clear
| * Won't be as obvious or easy how to use common CI/CD integrations such as git when looking at Tekton Pipelines docs | ||
| (will need to be directed to the catalog and/or [tektoncd/images](#new-repo-tektoncdimages)) |
There was a problem hiding this comment.
nit: Isn't this up to us on docs side ? Nothing should prevent us from having example, walkthrough, and use-cases parts in our documentation that would "use" those images.
There was a problem hiding this comment.
makes sense! I've tried to make this more clear
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jerop, vdemeester, wlynch The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This TEP proposes _officially_ deprecating PipelineResources. From some perspectives they have been unoffically deprecated for quite a while, ever since [we decided not to make them beta](https://github.com/tektoncd/pipeline/blob/e76d4132ab2ecfbedc45a964f08a01022e2d4c14/docs/resources.md#why-arent-pipelineresources-in-beta). By officially making them deprecated, we can give clearer guidance to our community on how to use Tekton Pipelines going forward instead of being in a place where we recommend folks don't use PipelineResources but can't clearly say what will happen next with them. This proposal still keeps open the possibility that we can benefit from a concept similar to PipelineResources, which could be worked on in parallel by anyone who feels motivated.
bobcatfish
force-pushed
the
tep_deprecate_pipelineresources
branch
from
3327aa9 to
0abdffd
Compare
There was a problem hiding this comment.
I've seen PipelineResource like requirements popping up in a few places, but I agree we should not insist on keeping the existing PipelineResources around and focus on getting to v1 instead. Thanks for all the work around this and collating all the relevant info in one TEP 🙏
Since this has been already approved and discussed in the API WG, it's fine to lgtm.
/lgtm
As part of TEP-0074 (tektoncd/community#480) I'm trying to update some of the plumbing use of PipelineResources to not use them anymore - and even further I want to start dogfooding [the experimental pipeline to taskrun custom task](https://github.com/tektoncd/experimental/tree/main/pipeline-to-taskrun). So to that end I've found a candiate Task that was using PipelineResources and I've converted it to use a Pipeline instead. I've also been able to run this successfully as a single pod on my own cluster so once I have the pipeline to taskrun controller installed in dogfooding I can come back and update this to run as a pod as well!
This TEP proposes officially deprecating PipelineResources. From some
perspectives they have been unoffically deprecated for quite a while,
ever since we decided not to make them beta.
By officially making them deprecated, we can give clearer guidance to
our community on how to use Tekton Pipelines going forward instead of
being in a place where we recommend folks don't use PipelineResources
but can't clearly say what will happen next with them.
This proposal still keeps open the possibility that we can benefit from
a concept similar to PipelineResources, which could be worked on in
parallel by anyone who feels motivated.