Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TEP-0118: Add validation for Matrix.Include.Params of type string #6219

Closed

Conversation

EmmaMunley
Copy link
Contributor

@EmmaMunley EmmaMunley commented Feb 23, 2023

Changes

This commit validates that:

  1. Parameters in Matrix.Include are of type string
  2. The parameter only exist in either matrix.Params or pipelineTask.Params
  3. The matrix params and include params does not exceed the max combination count
    Note: Other forms of validation and implementation logic will be added in subsequent commits.

/kind feature

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

  • Has Docs included if any changes are user facing
  • Has Tests included if any functionality added or changed
  • Follows the commit message standard
  • Meets the Tekton contributor standards (including
    functionality, content, code)
  • Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
  • Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
  • Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

NONE

@tekton-robot tekton-robot added kind/feature Categorizes issue or PR as related to a new feature. release-note-none Denotes a PR that doesnt merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 23, 2023
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1/param_types.go 98.3% 98.3% 0.1
pkg/apis/pipeline/v1/pipeline_types.go 97.2% 97.2% 0.0
pkg/apis/pipeline/v1beta1/param_types.go 98.2% 98.3% 0.1
pkg/apis/pipeline/v1beta1/pipeline_types.go 96.7% 96.8% 0.0

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1/param_types.go 98.3% 95.3% -2.9
pkg/apis/pipeline/v1/pipeline_types.go 97.2% 97.2% 0.0
pkg/apis/pipeline/v1beta1/param_types.go 98.2% 95.3% -3.0
pkg/apis/pipeline/v1beta1/pipeline_types.go 96.7% 96.8% 0.0

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1/param_types.go 98.3% 95.3% -2.9
pkg/apis/pipeline/v1/pipeline_types.go 97.2% 97.2% 0.0
pkg/apis/pipeline/v1beta1/param_types.go 98.2% 95.3% -3.0
pkg/apis/pipeline/v1beta1/pipeline_types.go 96.7% 96.8% 0.0

@tekton-robot tekton-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 23, 2023
Comment on lines 299 to 306
// MatrixHasInclude return whether matrix has Include
func (matrix *Matrix) MatrixHasInclude() bool {
return len(matrix.Include) > 0
}

// MatrixHasParams return whether matrix has Params
func (matrix *Matrix) MatrixHasParams() bool {
return len(matrix.Params) > 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do these functions have to be exported? it seems that they are only used within this package

@@ -314,22 +314,51 @@ func validatePipelineParametersVariablesInTaskParameters(params []Param, prefix
return errs
}

// validatePipelineParametersVariablesInMatrixParameters validates matrix param value
// validatePipelineParametersVariablesInMatrixParameters validates all Matrix Params including Matrix.Params and Matrix.Include.Params
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// validatePipelineParametersVariablesInMatrixParameters validates all Matrix Params including Matrix.Params and Matrix.Include.Params
// validatePipelineParametersVariablesInMatrixParameters validates all params in matrix.params and matrix.include.params

// ValidateParamsTypesInMatrix validates the type of parameter
// for Matrix.Params and Matrix.Include.Params
// Matrix.Params must be of type array. Matrix.Include.Params must be of type string.
func ValidateParamsTypesInMatrix(matrix *Matrix) (errs *apis.FieldError) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this function changed from unexported to exported?

if matrix.MatrixHasInclude() {
for _, include := range matrix.Include {
for _, param := range include.Params {
// Validate Matrix.Include.Params are of type string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment is not necessary because the code is easy to read

Comment on lines 457 to 514

func TestValidateParamsTypesInMatrix(t *testing.T) {
tests := []struct {
name string
matrix *v1.Matrix
wantErrs *apis.FieldError
}{{
name: "parameters in matrix are strings",
matrix: &v1.Matrix{
Params: []v1.Param{{
Name: "foo", Value: v1.ParamValue{Type: v1.ParamTypeString, StringVal: "foo"},
}, {
Name: "bar", Value: v1.ParamValue{Type: v1.ParamTypeString, StringVal: "bar"},
}}},
wantErrs: &apis.FieldError{
Message: "invalid value: parameters of type array only are allowed in matrix",
Paths: []string{"matrix[foo]", "matrix[bar]"},
},
}, {
name: "parameters in matrix are arrays",
matrix: &v1.Matrix{
Params: []v1.Param{{
Name: "foobar", Value: v1.ParamValue{Type: v1.ParamTypeArray, ArrayVal: []string{"foo", "bar"}},
}, {
Name: "barfoo", Value: v1.ParamValue{Type: v1.ParamTypeArray, ArrayVal: []string{"bar", "foo"}},
}}},
}, {
name: "parameters in include matrix are strings",
matrix: &v1.Matrix{
Include: []v1.MatrixInclude{
{Name: "build-1"},
{Params: []v1.Param{
{Name: "IMAGE", Value: v1.ParamValue{Type: v1.ParamTypeString, StringVal: "image-1"}},
{Name: "DOCKERFILE", Value: v1.ParamValue{Type: v1.ParamTypeString, StringVal: "path/to/Dockerfile1"}}}},
}},
}, {
name: "parameters in include matrix are arrays",
matrix: &v1.Matrix{
Include: []v1.MatrixInclude{
{Name: "build-1"},
{Params: []v1.Param{
{Name: "foobar", Value: v1.ParamValue{Type: v1.ParamTypeArray, ArrayVal: []string{"foo", "bar"}}},
{Name: "barfoo", Value: v1.ParamValue{Type: v1.ParamTypeArray, ArrayVal: []string{"bar", "foo"}}}}},
}},
wantErrs: &apis.FieldError{
Message: "invalid value: parameters of type string only are allowed in matrix",
Paths: []string{"matrix[barfoo]", "matrix[foobar]"},
},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if d := cmp.Diff(tc.wantErrs.Error(), v1.ValidateParamsTypesInMatrix(tc.matrix).Error()); d != "" {
t.Errorf("ValidateParamsTypesInMatrix() errors diff %s", diff.PrintWantGot(d))
}
})
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have a test for matrix validations in

func TestPipelineTask_validateMatrix(t *testing.T) {
tests := []struct {
name string
pt *PipelineTask
wantErrs *apis.FieldError
}{{
name: "parameter duplicated in matrix and params",
pt: &PipelineTask{
Name: "task",
Matrix: &Matrix{
Params: []Param{{
Name: "foobar", Value: ParamValue{Type: ParamTypeArray, ArrayVal: []string{"foo", "bar"}},
}}},
Params: []Param{{
Name: "foobar", Value: ParamValue{Type: ParamTypeArray, ArrayVal: []string{"foo", "bar"}},
}},
},
wantErrs: apis.ErrMultipleOneOf("matrix[foobar]", "params[foobar]"),
}, {
name: "parameters unique in matrix and params",
pt: &PipelineTask{
Name: "task",
Matrix: &Matrix{
Params: []Param{{
Name: "foobar", Value: ParamValue{Type: ParamTypeArray, ArrayVal: []string{"foo", "bar"}},
}}},
Params: []Param{{
Name: "barfoo", Value: ParamValue{Type: ParamTypeArray, ArrayVal: []string{"bar", "foo"}},
}},
},
}, {
name: "parameters in matrix are strings",
pt: &PipelineTask{
Name: "task",
Matrix: &Matrix{
Params: []Param{{
Name: "foo", Value: ParamValue{Type: ParamTypeString, StringVal: "foo"},
}, {
Name: "bar", Value: ParamValue{Type: ParamTypeString, StringVal: "bar"},
}}},
},
wantErrs: &apis.FieldError{
Message: "invalid value: parameters of type array only are allowed in matrix",
Paths: []string{"matrix[foo]", "matrix[bar]"},
},
}, {
name: "parameters in matrix are arrays",
pt: &PipelineTask{
Name: "task",
Matrix: &Matrix{
Params: []Param{{
Name: "foobar", Value: ParamValue{Type: ParamTypeArray, ArrayVal: []string{"foo", "bar"}},
}, {
Name: "barfoo", Value: ParamValue{Type: ParamTypeArray, ArrayVal: []string{"bar", "foo"}},
}}},
},
}, {
name: "parameters in matrix contain results references",
pt: &PipelineTask{
Name: "task",
Matrix: &Matrix{
Params: []Param{{
Name: "a-param", Value: ParamValue{Type: ParamTypeArray, ArrayVal: []string{"$(tasks.foo-task.results.a-result)"}},
}}},
},
}, {
name: "count of combinations of parameters in the matrix exceeds the maximum",
pt: &PipelineTask{
Name: "task",
Matrix: &Matrix{
Params: []Param{{
Name: "platform", Value: ParamValue{Type: ParamTypeArray, ArrayVal: []string{"linux", "mac", "windows"}},
}, {
Name: "browser", Value: ParamValue{Type: ParamTypeArray, ArrayVal: []string{"chrome", "firefox", "safari"}},
}}},
},
wantErrs: &apis.FieldError{
Message: "expected 0 <= 9 <= 4",
Paths: []string{"matrix"},
},
}, {
name: "count of combinations of parameters in the matrix equals the maximum",
pt: &PipelineTask{
Name: "task",
Matrix: &Matrix{
Params: []Param{{
Name: "platform", Value: ParamValue{Type: ParamTypeArray, ArrayVal: []string{"linux", "mac"}},
}, {
Name: "browser", Value: ParamValue{Type: ParamTypeArray, ArrayVal: []string{"chrome", "firefox"}},
}}},
},
}}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
featureFlags, _ := config.NewFeatureFlagsFromMap(map[string]string{
"enable-api-fields": "alpha",
})
defaults := &config.Defaults{
DefaultMaxMatrixCombinationsCount: 4,
}
cfg := &config.Config{
FeatureFlags: featureFlags,
Defaults: defaults,
}
ctx := config.ToContext(context.Background(), cfg)
if d := cmp.Diff(tt.wantErrs.Error(), tt.pt.validateMatrix(ctx).Error()); d != "" {
t.Errorf("PipelineTask.validateMatrix() errors diff %s", diff.PrintWantGot(d))
}
})
}
}

adds duplication, please add new test cases to the existing validations test

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1/param_types.go 98.3% 95.3% -2.9
pkg/apis/pipeline/v1/pipeline_types.go 97.2% 90.9% -6.3
pkg/apis/pipeline/v1beta1/param_types.go 98.2% 95.3% -3.0
pkg/apis/pipeline/v1beta1/pipeline_types.go 96.7% 97.2% 0.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1/param_types.go 98.3% 95.3% -2.9
pkg/apis/pipeline/v1/pipeline_types.go 97.2% 90.9% -6.3
pkg/apis/pipeline/v1beta1/param_types.go 98.2% 95.3% -3.0
pkg/apis/pipeline/v1beta1/pipeline_types.go 96.7% 97.2% 0.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1/param_types.go 98.3% 95.3% -2.9
pkg/apis/pipeline/v1/pipeline_types.go 97.2% 90.9% -6.3
pkg/apis/pipeline/v1beta1/param_types.go 98.2% 95.3% -3.0
pkg/apis/pipeline/v1beta1/pipeline_types.go 96.7% 97.2% 0.4

Copy link
Member

@jerop jerop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @EmmaMunley!

  • Update the title that it's doing all validations, not just checking the type
  • Update the commit message with some context about what matrix.include is, before explaining the validations for matrix.include
  • Improve the test coverage
  • Add relevant documentation
  • I'd prefer if the different validations were in separate PRs because this is now XL (but feel free to have all validations in one PR if you prefer)

…ferences to pipeline parameter variables, and combination count

This commit validates that:
1) Parameters in Matrix.Include are of type string
2) The parameter only exist in either matrix.Params or pipelineTask.Params
3) The matrix params and include params does not exceed the max combination count
Note: Other forms of validation and implementation logic will be added in subsequent commits.
@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign wlynch after the PR has been reviewed.
You can assign the PR to them by writing /assign @wlynch in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Feb 24, 2023
@EmmaMunley
Copy link
Contributor Author

/hold Will break up into smaller PRs

@tekton-robot tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 24, 2023
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1/param_types.go 98.3% 93.2% -5.0
pkg/apis/pipeline/v1/pipeline_types.go 97.2% 97.6% 0.4
pkg/apis/pipeline/v1beta1/param_types.go 98.2% 98.4% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 96.7% 97.2% 0.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1/param_types.go 98.3% 93.2% -5.0
pkg/apis/pipeline/v1/pipeline_types.go 97.2% 97.6% 0.4
pkg/apis/pipeline/v1beta1/param_types.go 98.2% 98.4% 0.2
pkg/apis/pipeline/v1beta1/pipeline_types.go 96.7% 97.2% 0.4

@tekton-robot
Copy link
Collaborator

tekton-robot commented Feb 24, 2023

@EmmaMunley: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-tekton-pipeline-build-tests 7ecf8a4 link true /test pull-tekton-pipeline-build-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@tekton-robot tekton-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 24, 2023
@tekton-robot
Copy link
Collaborator

@EmmaMunley: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@EmmaMunley
Copy link
Contributor Author

Closing this out. This PR has been broken up into the following PRs that have been opened:

TEP-0118: Added Matrix.Include field in preview mode #6188
TEP-0118: Add exported functions for validating Matrix.Include and Matrix.Params #6229
TEP-0118: Add validation for Matrix.Include.Params of type string [#6230]
TEP-0118: Add validation for matrix include pipeline parameter variables #6235
TEP-0118: Add validation for matrix combination count with matrix.include params [#6237]

@EmmaMunley EmmaMunley closed this Feb 27, 2023
@EmmaMunley EmmaMunley deleted the ValidateParamsTypesInMatrix branch February 27, 2023 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. kind/feature Categorizes issue or PR as related to a new feature. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note-none Denotes a PR that doesnt merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants