TEP-0063: Workspace Dependencies #413
Conversation
|
@jerop: GitHub didn't allow me to request PR reviews from the following users: souleb. Note that only tektoncd members and repo collaborators can review this PR, and authors cannot review their own PRs. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
tekton-robot
added
the
size/L
tekton-robot
added
the
approved
|
/assign |
jerop
force-pushed
the
workspace-dependencies
branch
from
a33aa5f
to
0208d5c
Compare
|
@sbwsg @bobcatfish @souleb thank you for the reviews! 😁 it's ready for another review from a non-googler (cc @tektoncd/core-maintainers) |
jerop
force-pushed
the
workspace-dependencies
branch
from
0208d5c
to
3be1116
Compare
bobcatfish
added
the
kind/tep
|
/kind tep |
|
/assign @pritidesai |
tekton-robot
added
the
needs-rebase
There was a problem hiding this comment.
Thanks for this @jerop.
I agree that we need a way to express these kind of dependencies - as describe in your example runAfter is not sufficient.
The only concern I have is the overlap with TEP-0030 Workspace Paths, as I that TEP will introduce the same kind of dependency, only path specific.
It may be that this TEP is a first step towards TEP-0030, in which case I'd like to make sure that design decisions in this TEP are mindful of TEP-0030 too.
/approve
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: afrittoli, sbwsg The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
In this PR, we discuss the motivation, goals, non-goals and requirements for `Workspace` dependencies so that we can align on the problem before exploring the proposal and alternatives later `Tasks` can have either resource dependencies or ordering dependencies between them. Resource dependencies are based on `Results` and `Workspaces`, while ordering dependencies are defined using `runAfter` to sequence `Tasks`. Today, users cannot specify resource dependencies based on `Workspaces`, that is, a `Task` should execute and use a given `Workspace` before another `Task` executes and uses the same `Workspace`. We need to provide a way for users to specify resource dependencies based on `Workspaces` to ensure that failure and skipping strategies for common CI/CD use cases work and users don't get unexpected `Pipeline` failures when we roll out those features. Failure and skipping strategies: - [TEP-0059: Skip Guarded Task Only](https://github.com/tektoncd/community/blob/main/teps/0059-skip-guarded-task-only.md) - [TEP-0050: Ignore Task Failures](https://github.com/tektoncd/community/blob/main/teps/0050-ignore-task-failures.md) Issue: tektoncd/pipeline#3109
jerop
force-pushed
the
workspace-dependencies
branch
from
3be1116
to
6474293
Compare
tekton-robot
removed
the
needs-rebase
thanks for the review @afrittoli 😁 had considered TEP-0030 as a solution to this issue, but there were drawbacks to it -- will discuss in the alternatives will make sure to also consider / discuss how the TEPs would work together (cc @sbwsg) |
|
Updating my approve to an lgtm. Agree that there are considerable drawbacks to Workspace Paths vis-a-vis workspace resource dependency. I'd be happy to spend a bit of time re-drafting parts of Workspace Paths in response to this TEP as well, then bring those modifications up for review here. Thanks a lot @jerop ! /lgtm |
In tektoncd#413, we added the problem statement for [TEP-0063: Workspace Dependencies](https://github.com/tektoncd/community/blob/main/teps/0063-workspace-dependencies.md) that discusses the need to give users a way to specify resource dependencies based on `Workspaces`. That will ensure that failure and skipping strategies for common CI/CD use cases work and users don't get unexpected failures. In this change, we add the proposal and discuss the alternatives for solving that problem. To enable users to specify resource dependencies based on `Workspaces`, we will provide a field - `useAfter` - that can be used to specify that a given `Task` should use a specific `Workspace` after another `Task` has already used it. It will be used to construct the `Directed Acyclic Graph` that represents the `Pipeline` to enforce the execution order.
In tektoncd#413, we added the problem statement for [TEP-0063: Workspace Dependencies](https://github.com/tektoncd/community/blob/main/teps/0063-workspace-dependencies.md) that discusses the need to give users a way to specify resource dependencies based on `Workspaces`. That will ensure that failure and skipping strategies for common CI/CD use cases work and users don't get unexpected failures. In this change, we add the proposal and discuss the alternatives for solving that problem. To enable users to specify resource dependencies based on `Workspaces`, we will provide a field - `useAfter` - that can be used to specify that a given `Task` should use a specific `Workspace` after another `Task` has already used it. It will be used to construct the `Directed Acyclic Graph` that represents the `Pipeline` to enforce the execution order.
In tektoncd#413, we added the problem statement for [TEP-0063: Workspace Dependencies](https://github.com/tektoncd/community/blob/main/teps/0063-workspace-dependencies.md) that discusses the need to give users a way to specify resource dependencies based on `Workspaces`. That will ensure that failure and skipping strategies for common CI/CD use cases work and users don't get unexpected failures. In this change, we add the proposal and discuss the alternatives for solving that problem. To enable users to specify resource dependencies based on `Workspaces`, we will provide a field - `useAfter` - that can be used to specify that a given `Task` should use a specific `Workspace` after another `Task` has already used it. It will be used to construct the `Directed Acyclic Graph` that represents the `Pipeline` to enforce the execution order.
In this PR, we discuss the motivation, goals, non-goals and requirements
for
Workspacedependencies so that we can align on the problem beforeexploring the proposal and alternatives later
Taskscan have either resource dependencies or ordering dependenciesbetween them. Resource dependencies are based on
ResultsandWorkspaces, while ordering dependencies are defined usingrunAfterto sequence
Tasks.Today, users cannot specify resource dependencies based on
Workspaces,that is, a
Taskshould execute and use a givenWorkspacebeforeanother
Taskexecutes and uses the sameWorkspace. We need toprovide a way for users to specify resource dependencies based on
Workspacesto ensure that failure and skipping strategies forcommon CI/CD use cases work and users don't get unexpected
Pipelinefailures when we roll out those features.
Failure and skipping strategies:
Issue: tektoncd/pipeline#3109
/cc @souleb @sbwsg @pritidesai @bobcatfish