Unifying test arguments and test workflows #503

saratvemulapalli · 2021-09-16T15:30:29Z

Description

Unifying test arguments across test suites and have a single runner via test.sh.
Restructured BWC tests to pull manifests from S3 and kick off the test suite.
Adding Jenkins job support for BWC tests

Issues Resolved

Check List

Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Sarat Vemulapalli <[email protected]>

codecov-commenter · 2021-09-16T15:35:40Z

Codecov Report

Merging #503 (c896849) into main (3a6dbae) will decrease coverage by 0.85%.
The diff coverage is 18.60%.

@@            Coverage Diff             @@
##             main     #503      +/-   ##
==========================================
- Coverage   69.31%   68.45%   -0.86%     
==========================================
  Files          58       59       +1     
  Lines        1509     1525      +16     
==========================================
- Hits         1046     1044       -2     
- Misses        463      481      +18

Impacted Files	Coverage Δ
bundle-workflow/src/run_bwc_test.py	`0.00% <0.00%> (ø)`
bundle-workflow/src/run_integ_test.py	`0.00% <0.00%> (ø)`
bundle-workflow/src/test_workflow/test_args.py	`0.00% <0.00%> (ø)`
...kflow/src/test_workflow/bwc_test/bwc_test_suite.py	`90.00% <100.00%> (-0.63%)`	⬇️
...ow/src/ci_workflow/ci_check_gradle_dependencies.py	`100.00% <0.00%> (ø)`
...src/manifests_workflow/component_opensearch_min.py	`100.00% <0.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3a6dbae...c896849. Read the comment docs.

Signed-off-by: Sarat Vemulapalli <[email protected]>

dblock

Some nits, but feel free to defer to another PR, there's enough here.

By making bundle-workflow/src/run_integ_test.py and friends runnable it becomes rather difficult to test them (hence lack of tests for them today). You could refactor them into classes such as IntegTestSuiteRunner(TestSuite), sinking any common args logic into TestSuite and moving the runners into the test suite subfolders as well, and then calling .run. Then test.sh wouldn't need to do the switching of what test suite to run in bash and become a dumb caller for run.sh like the other runners.

dblock · 2021-09-16T16:07:51Z

bundle-workflow/README.md

+export AWS_ROLE_ARN=arn:aws:iam::<AWS_JENKINS_ACCOUNT>:role/opensearch-test
+export AWS_ROLE_SESSION_NAME=dummy-session
+
+Next, configure temporary credentials in environment w/


Quote only the shell commands, this line should not be between ```.

dblock · 2021-09-16T16:08:48Z

bundle-workflow/jenkins_workflow/test/Jenkinsfile

@@ -34,6 +34,11 @@ pipeline {
                                            name: 'test_run_id',
                                            trim: true
                                    ),
+                                    string(
+                                            defaultValue: '',
+                                            name: 'test_suite_type',


Nit, test_suite_type -> test_suite, I think it's equally clear but shorter.

Sure, I like it. I have a follow up PR, will clean it up there.

dblock · 2021-09-16T16:09:06Z

bundle-workflow/jenkins_workflow/test/README.md

saratvemulapalli · 2021-09-16T16:25:40Z

Some nits, but feel free to defer to another PR, there's enough here.

By making bundle-workflow/src/run_integ_test.py and friends runnable it becomes rather difficult to test them (hence lack of tests for them today). You could refactor them into classes such as IntegTestSuiteRunner(TestSuite), sinking any common args logic into TestSuite and moving the runners into the test suite subfolders as well, and then calling .run. Then test.sh wouldn't need to do the switching of what test suite to run in bash and become a dumb caller for run.sh like the other runners.

Yeah absolutely, I didnt want to do major restructuring. Will take care of this in a follow up PR.
Issue: #505

bundle-workflow/README.md

bundle-workflow/src/test_workflow/test_args.py

owaiskazi19 · 2021-09-16T17:25:42Z

bundle-workflow/src/run_bwc_test.py

+    args = TestArgs()
+    with TemporaryDirectory(keep=args.keep) as work_dir:
+        logging.info("Switching to temporary work_dir: " + work_dir)
+        os.chdir(work_dir)


Can use with WorkingDirectory(work_dir) as cur_dir here

Sure will take care of this in a quick follow up PR.

saratvemulapalli added 5 commits September 15, 2021 23:31

Unifying all test arguments and test runners

5ae95a3

Signed-off-by: Sarat Vemulapalli <[email protected]>

Fixing typos

f492524

Signed-off-by: Sarat Vemulapalli <[email protected]>

Removing usage of work_dir

fb4152c

Signed-off-by: Sarat Vemulapalli <[email protected]>

Removing Jenkins Job from this change

8c5a784

Signed-off-by: Sarat Vemulapalli <[email protected]>

Updating README.md

d0961cd

Signed-off-by: Sarat Vemulapalli <[email protected]>

saratvemulapalli requested review from dblock, peternied and setiah September 16, 2021 15:31

Adding Jenkins job support for BWC tests

c896849

Signed-off-by: Sarat Vemulapalli <[email protected]>

saratvemulapalli changed the title ~~Unifying test arguments and restructuring BWC tests to pull manifests~~ Unifying test arguments and test workflows Sep 16, 2021

dblock approved these changes Sep 16, 2021

View reviewed changes

saratvemulapalli mentioned this pull request Sep 16, 2021

Restructure tests and inhert a common structure #505

Closed

gaiksaya reviewed Sep 16, 2021

View reviewed changes

bundle-workflow/README.md Show resolved Hide resolved

bundle-workflow/src/test_workflow/test_args.py Show resolved Hide resolved

owaiskazi19 reviewed Sep 16, 2021

View reviewed changes

owaiskazi19 approved these changes Sep 16, 2021

View reviewed changes

saratvemulapalli merged commit 596fcfc into opensearch-project:main Sep 16, 2021

saratvemulapalli mentioned this pull request Sep 16, 2021

Adding default script for bwctest.sh #508

Merged

1 task

peterzhuamazon mentioned this pull request Nov 8, 2024

[Feature] Publish plugins SNAPSHOT artifacts (along with SNASPHOT distribution) #5096

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unifying test arguments and test workflows #503

Unifying test arguments and test workflows #503

saratvemulapalli commented Sep 16, 2021 •

edited

Loading

codecov-commenter commented Sep 16, 2021 •

edited

Loading

dblock left a comment •

edited

Loading

dblock Sep 16, 2021

dblock Sep 16, 2021

saratvemulapalli Sep 16, 2021

dblock Sep 16, 2021

saratvemulapalli commented Sep 16, 2021 •

edited

Loading

owaiskazi19 Sep 16, 2021

saratvemulapalli Sep 16, 2021

Unifying test arguments and test workflows #503

Unifying test arguments and test workflows #503

Conversation

saratvemulapalli commented Sep 16, 2021 • edited Loading

Description

Issues Resolved

Check List

codecov-commenter commented Sep 16, 2021 • edited Loading

Codecov Report

dblock left a comment • edited Loading

Choose a reason for hiding this comment

dblock Sep 16, 2021

Choose a reason for hiding this comment

dblock Sep 16, 2021

Choose a reason for hiding this comment

saratvemulapalli Sep 16, 2021

Choose a reason for hiding this comment

dblock Sep 16, 2021

Choose a reason for hiding this comment

saratvemulapalli commented Sep 16, 2021 • edited Loading

owaiskazi19 Sep 16, 2021

Choose a reason for hiding this comment

saratvemulapalli Sep 16, 2021

Choose a reason for hiding this comment

saratvemulapalli commented Sep 16, 2021 •

edited

Loading

codecov-commenter commented Sep 16, 2021 •

edited

Loading

dblock left a comment •

edited

Loading

saratvemulapalli commented Sep 16, 2021 •

edited

Loading