Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Missing coverage of stored resources in upgrade tests #6868

Closed
Tracked by #6870
lbernick opened this issue Jun 26, 2023 · 2 comments · Fixed by #6855
Closed
Tracked by #6870

Missing coverage of stored resources in upgrade tests #6868

lbernick opened this issue Jun 26, 2023 · 2 comments · Fixed by #6855
Assignees
@JeromeJu
Labels
area/testing Issues or PRs related to testing kind/bug Categorizes issue or PR as related to a bug.

Comments

@lbernick
Copy link
Member

Expected Behavior

Upgrade tests cover scenarios where Tekton CRDs exist on the cluster prior to upgrading the Pipelines version, as described in their comments.

Actual Behavior

Upgrade tests install the older pipeline server, upgrade to the newer server, run tests, clean up the environment, and repeat this process.

The function apply_resources, which was responsible for this functionality, was renamed in #1351, breaking the upgrade tests, and the defunct function call was removed in #6723.
@lbernick lbernick added the kind/bug Categorizes issue or PR as related to a bug. label Jun 26, 2023
@lbernick lbernick added the area/testing Issues or PRs related to testing label Jun 26, 2023
This was referenced Jun 23, 2023
Merged
Closed
Open
@JeromeJu
Copy link
Member

During the work of #6855, and by comparing how the upgrade test scenario 2 is introduced in #1162, we could find it is problematic in the way of the following: when we are trying to create taskRuns and pipelineRuns resources in the same namespace:

Regarding the resources we choose, we would have the following options instead of restoring the scenario ii of the upgrade test back to its earlier stage when it's been created:

  • choose a list of 'stable' examples as resources to be created
  • choose a list of resources that does not have duplicate naming

@lbernick
Copy link
Member Author

@JeromeJu that's a great point about not wanting to run all our integration tests to avoid flakiness.
One thing we could try would be to create a simple pipeline and simple task, upgrade the pipelines version, and then create a simple pipelinerun and taskrun using the pipeline and task that existed before the upgrade and verify that they run successfully.

JeromeJu added a commit to JeromeJu/pipeline that referenced this issue Jun 28, 2023
@JeromeJu
Restore Upgrade Test Scenario2 by creating resources
56b2e17 
Scenario 2 of the upgrade test aims to install pipeline server from the
preivous release, create resources from that version and test on those
resources against the client of the current release.

This commit restores the resources creation of simple Task and Pipeline
resources to enable the 2nd scenario of the upgrade test to work by
applying the resources from the old server version of pipeline. It was
preivously renamed and removed in tektoncd#2685.

It fixes the missing piece where the required resources from the previous
release have not been created to accomplish scenario 2 of the upgrade
test. More specifically, it creates a simple Task and Pipeline resources
and then runs the two simple resources with a TaskRun and PipelineRun
created after upgrading to the current version and verify that they both
run successfully.

/kind misc
Closes tektoncd#6868
JeromeJu added a commit to JeromeJu/pipeline that referenced this issue Jun 28, 2023
@JeromeJu
Restore Upgrade Test Scenario2 by creating resources
3ee0608 
Scenario 2 of the upgrade test aims to install pipeline server from the
preivous release, create resources from that version and test on those
resources against the client of the current release.

This commit restores the resources creation of simple Task and Pipeline
resources to enable the 2nd scenario of the upgrade test to work by
applying the resources from the old server version of pipeline. It was
preivously renamed and removed in tektoncd#2685.

It fixes the missing piece where the required resources from the previous
release have not been created to accomplish scenario 2 of the upgrade
test. More specifically, it creates a simple Task and Pipeline resources
and then runs the two simple resources with a TaskRun and PipelineRun
created after upgrading to the current version and verify that they both
run successfully.

/kind misc
Closes tektoncd#6868
JeromeJu added a commit to JeromeJu/pipeline that referenced this issue Jun 29, 2023
@JeromeJu
Restore Upgrade Test Scenario2 by creating simple Task and Pipeline r…
e553150 
…esources

Scenario 2 of the upgrade test aims to install pipeline server from the
preivous release, create resources from that version and test on those
resources against the client of the current release.

This commit restores the resources creation of simple Task and Pipeline
resources to enable the 2nd scenario of the upgrade test to work by
applying the resources from the old server version of pipeline. It was
preivously renamed in tektoncd#1351 and then removed in tektoncd#2685.

It fixes the missing piece where the required resources from the previous
release have not been created to accomplish scenario 2 of the upgrade
test. More specifically, it creates a simple Task and Pipeline resources
and then runs the two simple resources with a TaskRun and PipelineRun
created after upgrading to the current version and verify that they both
run successfully.

/kind misc
Closes tektoncd#6868
JeromeJu added a commit to JeromeJu/pipeline that referenced this issue Jun 29, 2023
@JeromeJu
Restore Upgrade Test Scenario2 by creating simple Task and Pipeline r…
209d268 
…esources

Scenario 2 of the upgrade test aims to install pipeline server from the
preivous release, create resources from that version and test on those
resources against the client of the current release.

This commit restores the resources creation of simple Task and Pipeline
resources to enable the 2nd scenario of the upgrade test to work by
applying the resources from the old server version of pipeline. It was
preivously renamed in tektoncd#1351 and then removed in tektoncd#2685.

It fixes the missing piece where the required resources from the previous
release have not been created to accomplish scenario 2 of the upgrade
test. More specifically, it creates a simple Task and Pipeline resources
and then runs the two simple resources with a TaskRun and PipelineRun
created after upgrading to the current version and verify that they both
run successfully.

/kind misc
Closes tektoncd#6868
JeromeJu added a commit to JeromeJu/pipeline that referenced this issue Jun 29, 2023
@JeromeJu
Restore Upgrade Test Scenario2 by creating simple Task and Pipeline r…
61f3ad7 
…esources

Scenario 2 of the upgrade test aims to install pipeline server from the
preivous release, create resources from that version and test on those
resources against the client of the current release.

This commit restores the resources creation of simple Task and Pipeline
resources to enable the 2nd scenario of the upgrade test to work by
applying the resources from the old server version of pipeline. It was
preivously renamed in tektoncd#1351 and then removed in tektoncd#2685.

It fixes the missing piece where the required resources from the previous
release have not been created to accomplish scenario 2 of the upgrade
test. More specifically, it creates a simple Task and Pipeline resources
and then runs the two simple resources with a TaskRun and PipelineRun
created after upgrading to the current version and verify that they both
run successfully.

/kind misc
Closes tektoncd#6868
JeromeJu added a commit to JeromeJu/pipeline that referenced this issue Jun 29, 2023
@JeromeJu
Restore Upgrade Test Scenario2 by creating simple Task and Pipeline r…
21419ba 
…esources

Scenario 2 of the upgrade test aims to install pipeline server from the
preivous release, create resources from that version and test on those
resources against the client of the current release.

This commit restores the resources creation of simple Task and Pipeline
resources to enable the 2nd scenario of the upgrade test to work by
applying the resources from the old server version of pipeline. It was
preivously renamed in tektoncd#1351 and then removed in tektoncd#2685.

It fixes the missing piece where the required resources from the previous
release have not been created to accomplish scenario 2 of the upgrade
test. More specifically, it creates a simple Task and Pipeline resources
and then runs the two simple resources with a TaskRun and PipelineRun
created after upgrading to the current version and verify that they both
run successfully.

/kind misc
Closes tektoncd#6868
JeromeJu added a commit to JeromeJu/pipeline that referenced this issue Jun 30, 2023
@JeromeJu
Restore Upgrade Test Scenario2 by creating simple Task and Pipeline r…
241e82b 
…esources

Scenario 2 of the upgrade test aims to install pipeline server from the
preivous release, create resources from that version and test on those
resources against the client of the current release.

This commit restores the resources creation of simple Task and Pipeline
resources to enable the 2nd scenario of the upgrade test to work by
applying the resources from the old server version of pipeline. It was
preivously renamed in tektoncd#1351 and then removed in tektoncd#2685.

It fixes the missing piece where the required resources from the previous
release have not been created to accomplish scenario 2 of the upgrade
test. More specifically, it creates a simple Task and Pipeline resources
and then runs the two simple resources with a TaskRun and PipelineRun
created after upgrading to the current version and verify that they both
run successfully.

/kind misc
Closes tektoncd#6868
tekton-robot pushed a commit that referenced this issue Jul 10, 2023
@JeromeJu @tekton-robot
Restore Upgrade Test Scenario2 by creating simple Task and Pipeline r…
161ff7d 
…esources

Scenario 2 of the upgrade test aims to install pipeline server from the
preivous release, create resources from that version and test on those
resources against the client of the current release.

This commit restores the resources creation of simple Task and Pipeline
resources to enable the 2nd scenario of the upgrade test to work by
applying the resources from the old server version of pipeline. It was
preivously renamed in #1351 and then removed in #2685.

It fixes the missing piece where the required resources from the previous
release have not been created to accomplish scenario 2 of the upgrade
test. More specifically, it creates a simple Task and Pipeline resources
and then runs the two simple resources with a TaskRun and PipelineRun
created after upgrading to the current version and verify that they both
run successfully.

/kind misc
Closes #6868
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JeromeJu JeromeJu

Labels
area/testing Issues or PRs related to testing kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Restore Upgrade Test Scenario2 by creating simple Task and Pipeline resources JeromeJu/pipeline
2 participants
@lbernick @JeromeJu