From b7defb625a7a66d9b28dbb91db9937374eab371e Mon Sep 17 00:00:00 2001 From: Jerop Date: Fri, 10 Jun 2022 08:22:36 -0400 Subject: [PATCH] TEP-0090: Matrix - Status [TEP-0090: Matrix][tep-0090] proposed executing a `PipelineTask` in parallel `TaskRuns` and `Runs` with substitutions from combinations of `Parameters` in a `Matrix`. In this change, we describe how the status of a `PipelineRun` with a matrixed `PipelineTask` would be in full and minimal modes. Previously, we had included `matrixId` to the `PipelineRun` status so that it can help with identifying `Results` from fanned out `TaskRuns` or `Runs`. However, we ended up descoping that problem so that it can be addressed when TEP-0075 and TEP-0076 are implemented. See [`Results` from fanned-out `PipelineTasks`][results] for details. In this change, we remove the `matrixId` in `PipelineRun` status so that the problem can be addressed completely later. [tep-0090]: https://github.com/tektoncd/community/blob/main/teps/0090-matrix.md [results]: https://github.com/tektoncd/community/blob/main/teps/0090-matrix.md#results-from-fanned-out-pipelinetasks --- teps/0090-matrix.md | 74 +++++++++++++++++++++++++++------------------ teps/README.md | 2 +- 2 files changed, 45 insertions(+), 31 deletions(-) diff --git a/teps/0090-matrix.md b/teps/0090-matrix.md index 44eaf978c..50e5fe35d 100644 --- a/teps/0090-matrix.md +++ b/teps/0090-matrix.md @@ -2,7 +2,7 @@ status: implementable title: Matrix creation-date: '2021-10-13' -last-updated: '2022-02-14' +last-updated: '2022-06-06' authors: - '@jerop' - '@pritidesai' @@ -46,7 +46,6 @@ see-also: - [Parameters](#parameters) - [Substituting String Parameters in the Tasks](#substituting-string-parameters-in-the-tasks) - [Substituting Array Parameters in the Tasks](#substituting-array-parameters-in-the-tasks) - - [Combinations of Parameters in the Matrix](#combinations-of-parameters-in-the-matrix) - [Results](#results) - [Specifying Results in the Matrix](#specifying-results-in-the-matrix) - [Results from Fanned Out PipelineTasks](#results-from-fanned-out-pipelinetasks) @@ -61,6 +60,7 @@ see-also: - [When Expressions](#when-expressions) - [Retries](#retries) - [Timeouts](#timeouts) + - [Status](#status) - [Design Evaluation](#design-evaluation) - [API Conventions](#api-conventions) - [Reusability](#reusability) @@ -1021,34 +1021,6 @@ We plan to provide minimum feature set needed to meet the use cases, as describe [simplicity][simplicity] design principle. However, this remains an option we can explore later if we have use cases for it and already support `Arrays of Arrays` in Tekton Pipelines. -#### Combinations of Parameters in the Matrix - -We need a way to identify a specific combination of `Parameters` in the `Matrix` so that -we can map it to a given `TaskRun` and its components, such as `Results`. We cannot rely -on the ordering of the `Parameters` in the `Matrix` to be maintained as specified in the -`Pipeline`. We propose that we add the generated combinations identifications to the -`PipelineRunStatusFields` in `PipelineRunStatus`. For example: - -```yaml - taskRuns: - matrixed-pr-foo-0: - pipelineTaskName: foo - matrixId: 0 - status: - ... - matrixed-pr-foo-1: - pipelineTaskName: foo - matrixId: 1 - status: - ... -``` - -The `Parameters` were used for a given `TaskRun` or `Runs` would available in the `TaskRun` -status itself. The `matrixId` in the `PipelineRunStatus` is useful in identifying that the -`TaskRun` is part of a `Matrix`, especially if the `TaskRun` name was too long so got hashed. -This is also in line with providing the minimal information needed in the `PipelineRunStatus` -as discussed in [TEP-0100: Embedded TaskRuns and Runs Status in PipelineRuns][tep-0100]. - ### Results #### Specifying Results in the Matrix @@ -1340,6 +1312,48 @@ spec: name: build-push ``` +### Status + +The status of `PipelineRuns` with fanned-out `PipelineTasks` will list all the `TaskRuns` and `Runs` created. + +In [TEP-0100][tep-0100] we proposed changes to `PipelineRun` status to reduce the amount of information stored about +the status of `TaskRuns` and `Runs` to improve performance, reduce memory bloat and improve extensibility. Now that +those changes have been implemented, the `PipelineRun` status is set up to handle `Matrix` without exacerbating the +performance and storage issues that were there before. + +We will populate `ChildReferences` for all fanned out `TaskRuns` and `Runs`, as shown below: + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: PipelineRun +metadata: + name: matrixed-pr + ... +spec: + ... +status: + completionTime: "2020-05-04T02:19:14Z" + conditions: + - lastTransitionTime: "2020-05-04T02:19:14Z" + message: "Tasks Completed: 4, Skipped: 0" + reason: Succeeded + status: "True" + type: Succeeded + startTime: "2020-05-04T02:00:11Z" + childReferences: + - apiVersion: tekton.dev/v1beta1 + kind: TaskRun + name: matrixed-pr-foo-0 + pipelineTaskName: foo + - apiVersion: tekton.dev/v1beta1 + kind: TaskRun + name: matrixed-pr-foo-1 + pipelineTaskName: foo +``` + +The `ChildReferences` will be populated for matrixed `PipelineTasks` regardless of the embedded status flags +because that is the API behavior we're migrating to. + ## Design Evaluation ### API Conventions diff --git a/teps/README.md b/teps/README.md index 556817f4b..948983dd9 100644 --- a/teps/README.md +++ b/teps/README.md @@ -234,7 +234,7 @@ This is the complete list of Tekton teps: |[TEP-0086](0086-changing-the-way-result-parameters-are-stored.md) | Changing the way result parameters are stored | proposed | 2022-04-07 | |[TEP-0088](0088-result-summaries.md) | Tekton Results - Record Summaries | proposed | 2021-10-01 | |[TEP-0089](0089-nonfalsifiable-provenance-support.md) | Non-falsifiable provenance support | implementable | 2022-01-18 | -|[TEP-0090](0090-matrix.md) | Matrix | implementable | 2022-02-14 | +|[TEP-0090](0090-matrix.md) | Matrix | implementable | 2022-06-06 | |[TEP-0092](0092-scheduling-timeout.md) | Scheduling Timeout | implementable | 2022-04-11 | |[TEP-0094](0094-configuring-resources-at-runtime.md) | Configuring Resources at Runtime | implemented | 2022-03-11 | |[TEP-0095](0095-common-repository-configuration.md) | Common Repository Configuration | proposed | 2021-11-29 |