Allow steps to run regardless of previous step errors

[ghost]

Fixes #1559

Changes

Outstanding TODOs:

• Implement defaultErrorStrategy behaviour: its value should be copied into any steps that dont include their own errorStrategy field.
• Confirm how this interacts with existing output pipeline resources.

This commit introduces an errorStrategy field to Steps and a defaultErrorStrategy field to Tasks. These fields can currently accept one of two possible strategies. The first is SkipOnPriorStepErrors and tells the Step to skip its work if a prior step has failed. This is how Tekton works today and is therefore the default value if the field is omitted or left blank. The second strategy is IgnorePriorStepErrors and tells a step to run regardless of whether a previous step has already errored out.

This feature is intended to serve two similar use-cases:

1. Unit test failures in a step can be written to the task workspace and then uploaded to persistent storage in a subsequent step, even though the unit test failure resulted in a non-zero exit code.
2. Pipeline resources that need to report on the failed status of a step can do so. For example the pull request resource could be updated to add a final step to a Task that updates a PR with the status of a build step. Currently if a build step fails then any later steps will be skipped.

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

• Includes tests (if functionality changed/added)
• Includes docs (if user facing)
• Commit messages follow commit message best practices

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

• If you are adding a new binary/image to the cmd dir, please update
  the release Task to build and release this image.

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

The steps of a task can now be run regardless of previous steps' errors. A step can now upload test results even if the test runner step before it failed.

[tekton-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign sbwsg
You can assign the PR to them by writing /assign @sbwsg in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tekton-robot]

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/waiter.go	63.6%	70.0%	6.4
pkg/apis/pipeline/v1alpha1/task_defaults.go	80.0%	88.9%	8.9
pkg/apis/pipeline/v1alpha1/task_validation.go	81.5%	80.3%	-1.2
pkg/entrypoint/error_strategies.go	Do not exist	80.0%
pkg/entrypoint/skip_error.go	Do not exist	0.0%
pkg/reconciler/taskrun/entrypoint/entrypoint.go	76.9%	77.5%	0.6

[tekton-robot]

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/waiter.go	63.6%	70.0%	6.4
pkg/apis/pipeline/v1alpha1/task_defaults.go	80.0%	88.9%	8.9
pkg/apis/pipeline/v1alpha1/task_validation.go	81.5%	80.3%	-1.2
pkg/entrypoint/error_strategies.go	Do not exist	80.0%
pkg/entrypoint/skip_error.go	Do not exist	0.0%
pkg/reconciler/taskrun/entrypoint/entrypoint.go	76.9%	77.5%	0.6

[tekton-robot]

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/waiter.go	63.6%	70.0%	6.4
pkg/apis/pipeline/v1alpha1/task_defaults.go	80.0%	88.9%	8.9
pkg/apis/pipeline/v1alpha1/task_validation.go	81.5%	80.3%	-1.2
pkg/entrypoint/error_strategies.go	Do not exist	80.0%
pkg/entrypoint/skip_error.go	Do not exist	0.0%
pkg/reconciler/taskrun/entrypoint/entrypoint.go	76.9%	77.5%	0.6

[tekton-robot]

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/waiter.go	63.6%	70.0%	6.4
pkg/apis/pipeline/v1alpha1/task_defaults.go	80.0%	88.9%	8.9
pkg/apis/pipeline/v1alpha1/task_validation.go	81.5%	80.3%	-1.2
pkg/entrypoint/error_strategies.go	Do not exist	100.0%
pkg/entrypoint/skip_error.go	Do not exist	0.0%
pkg/reconciler/taskrun/entrypoint/entrypoint.go	76.9%	77.5%	0.6

[tekton-robot]

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/waiter.go	63.6%	70.0%	6.4
pkg/apis/pipeline/v1alpha1/task_defaults.go	80.0%	88.9%	8.9
pkg/apis/pipeline/v1alpha1/task_validation.go	81.5%	80.9%	-0.6
pkg/entrypoint/error_strategies.go	Do not exist	100.0%
pkg/entrypoint/skip_error.go	Do not exist	0.0%
pkg/reconciler/taskrun/entrypoint/entrypoint.go	76.9%	77.5%	0.6

[tekton-robot]

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/waiter.go	63.6%	70.0%	6.4
pkg/apis/pipeline/v1alpha1/task_validation.go	81.5%	80.9%	-0.6
pkg/entrypoint/error_strategies.go	Do not exist	100.0%
pkg/entrypoint/skip_error.go	Do not exist	0.0%
pkg/reconciler/taskrun/entrypoint/entrypoint.go	76.9%	77.5%	0.6
pkg/reconciler/taskrun/taskrun.go	72.5%	73.2%	0.7
test/builder/step.go	10.0%	9.5%	-0.5
test/builder/task.go	82.6%	81.8%	-0.8

[tekton-robot]

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
cmd/entrypoint/waiter.go	63.6%	70.0%	6.4
pkg/apis/pipeline/v1alpha1/task_validation.go	81.5%	80.9%	-0.6
pkg/entrypoint/error_strategies.go	Do not exist	100.0%
pkg/entrypoint/skip_error.go	Do not exist	0.0%
pkg/reconciler/taskrun/entrypoint/entrypoint.go	76.9%	77.5%	0.6
pkg/reconciler/taskrun/taskrun.go	72.5%	73.2%	0.7
test/builder/step.go	10.0%	9.5%	-0.5
test/builder/task.go	82.6%	81.8%	-0.8

[ghost]

With respect to output pipeline resources I don't observe any obvious change in their behaviour with this PR. I modified examples/pipelineruns/output-pipelinerun.yaml so that the first Task's first step errored out and the subsequent step ignores the error. Apart from the second executing there were no perceivable differences in the execution results of the pipelinerun. Both before and after the error strategy support was introduced the PipelineRun failed because the first TaskRun failed. The only difference was that within that first TaskRun the second Step was allowed to finish executing despite the first step's failure.

[ghost]

ok i think this is ready to review now.

[bobcatfish]

/hold

humble request to be able to review this before it goes in - but am swamped with kubecon until after next week 🙏 which im guessing is the case for most OWNERS

[ghost]

i'm going to close this until after kubecon when it can be discussed further. will just keep it as part of my resources branch.