-
Notifications
You must be signed in to change notification settings - Fork 304
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CART-831 TEST: add support for arbitrary envariables in yaml files #4445
Conversation
Add special other_env_vars param, which takes a newline separated list of env-var=value pairs to add to the python testing environment. Include a simple example use-case in nopmix_multictx yaml file. Skip-build-leap15-rpm: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-func-test-leap15: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4445/1/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules
FYI: Errors found in lines not modified in the patch:
src/tests/ftest/cart/no_pmix/cart_nopmix_multictx_one_node.py:91:
(pylint-undefined-variable) Undefined variable 'main'
Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4445/1/execution/node/66/log |
Test stage Unit Test with memcheck completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4445/1/execution/node/474/log |
Test stage Functional on CentOS 7 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4445/1/display/redirect |
Use Yaml array syntax, instead of multi-line string param. Test-tag: cart Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4445/2/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules
FYI: Errors found in lines not modified in the patch:
src/tests/ftest/cart/no_pmix/cart_nopmix_multictx_one_node.py:91:
(pylint-undefined-variable) Undefined variable 'main'
Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4445/2/execution/node/66/log |
Test-tag: cart Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4445/3/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules
FYI: Errors found in lines not modified in the patch:
src/tests/ftest/cart/no_pmix/cart_nopmix_multictx_one_node.py:91:
(pylint-undefined-variable) Undefined variable 'main'
Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4445/3/execution/node/66/log |
* unused vars * trailing whitespace * remove trace line Skip-build-leap15-rpm: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-unit-tests: true Skip-nlt: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4445/4/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules
FYI: Errors found in lines not modified in the patch:
src/tests/ftest/cart/no_pmix/cart_nopmix_multictx_one_node.py:91:
(pylint-undefined-variable) Undefined variable 'main'
Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4445/4/execution/node/66/log |
Test stage Functional on CentOS 7 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4445/4/display/redirect |
Address this pylint message: (pylint-no-self-use) Method could be a function Method doesn't use instance data vars, so make it a function. Skip-build-leap15-rpm: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-unit-tests: true Skip-nlt: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. No errors found by checkpatch.
FYI: Errors found in lines not modified in the patch:
src/tests/ftest/cart/no_pmix/cart_nopmix_multictx_one_node.py:91:
(pylint-undefined-variable) Undefined variable 'main'
Test stage Functional on CentOS 7 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4445/5/display/redirect |
Back out demonstration code in cart_nopmix_multictx_one_node Skip-build-leap15-rpm: true Skip-func-test-leap15: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-unit-tests: true Skip-nlt: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4445/6/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules
Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4445/6/execution/node/66/log |
Test stage Functional on CentOS 7 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4445/6/display/redirect |
Remedy this pylint error: (pylint-undefined-variable) Undefined variable 'main' Skip-build-leap15-rpm: true Skip-func-test-leap15: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-unit-tests: true Skip-nlt: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
Address these pylint issues: (pylint-super-with-arguments) Consider using Python 3 style super() without arguments Skip-build-leap15-rpm: true Skip-func-test-leap15: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-unit-tests: true Skip-nlt: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
Need 4 spaces, not just 2 space for indentation. Skip-build-leap15-rpm: true Skip-func-test-leap15: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-unit-tests: true Skip-nlt: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. No errors found by checkpatch.
FYI: Errors found in lines not modified in the patch:
src/tests/ftest/cart/corpc/cart_corpc_five_node.py:51:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/corpc/cart_corpc_one_node.py:51:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/corpc/cart_corpc_two_node.py:51:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/ctl/cart_ctl_one_node.py:67:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/ghost_rank_rpc/cart_ghost_rank_prc_one_node.py:51:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/group_test/group_test.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/iv/cart_iv_one_node.py:539:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/iv/cart_iv_two_node.py:300:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/nopmix_launcher/cart_nopmix_launcher_one_node.py:66:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/rpc/cart_rpc_one_node.py:51:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/rpc/cart_rpc_two_node.py:51:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/selftest/cart_selftest_three_node.py:71:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/util/cart_utils.py:56:
(pylint-no-else-break) Unnecessary "else" after "break"
src/tests/ftest/cart/util/cart_utils.py:98:
(pylint-no-else-break) Unnecessary "else" after "break"
src/tests/ftest/cart/util/cart_utils.py:223:
(pylint-no-else-return) Unnecessary "else" after "return"
src/tests/ftest/util/apricot/apricot/test.py:77:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments
src/tests/ftest/util/apricot/apricot/test.py:140:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments
src/tests/ftest/util/apricot/apricot/test.py:257:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments
src/tests/ftest/util/apricot/apricot/test.py:277:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments
src/tests/ftest/util/apricot/apricot/test.py:302:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments
src/tests/ftest/util/apricot/apricot/test.py:381:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments
src/tests/ftest/util/apricot/apricot/test.py:423:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments
src/tests/ftest/util/apricot/apricot/test.py:488:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments
src/tests/ftest/util/apricot/apricot/test.py:965:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments
Per "ESAD Build and Test Automation" Skype channel: @mjean308: While python 3 test content update is in progress: #4615 @ashleypitman: we'll just have to live with or suppress that warning until the test code is updated. @rpadma2: ... ignore Python 3 related errors for now... If some pylint errors are already in the script, you can leave it... There is an effort to move our scripts to python3. Presently, CI testing uses python 2 version. Skip-build-leap15-rpm: true Skip-func-test-leap15: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-unit-tests: true Skip-nlt: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
Sync with CI/Jenkins avocado version (69.2). Stringify all keys/vals in dict, to resolve this error: `execve() arg 3 contains a non-string value`. Skip-build-leap15-rpm: true Skip-func-test-leap15: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-unit-tests: true Skip-nlt: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
Background: `pylint` printed the following error: `(pylint-super-with-arguments) Consider using Python 3 style super() without arguments` #4445 (review) But when we fix the pylint error using Python 3 super w/o arguments, we get this error: `ERROR: super() takes at least 1 argument` https://build.hpdd.intel.com/job/daos-stack/job/daos/view/change-requests/job/PR-4445/27/testReport/junit/FTEST_cart/CartCoRpcFiveNodeTest/Test___Functional_on_CentOS_7___1___cart_cart_corpc_five_node_py_CartCoRpcFiveNodeTest_test_cart_corpc_ENV_no_sep_hosts_1_corpc_prefwd_2a08/ Q: How to resolve both? A: (@ashleypitman) ... live with or suppress that warning until the test code is updated. Skip-build-leap15-rpm: true Skip-func-test-leap15: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-unit-tests: true Skip-nlt: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4445/28/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules
FYI: Errors found in lines not modified in the patch:
src/tests/ftest/cart/nopmix_launcher/cart_nopmix_launcher_one_node.py:66:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/util/cart_utils.py:56:
(pylint-no-else-break) Unnecessary "else" after "break"
src/tests/ftest/cart/util/cart_utils.py:98:
(pylint-no-else-break) Unnecessary "else" after "break"
src/tests/ftest/cart/util/cart_utils.py:223:
(pylint-no-else-return) Unnecessary "else" after "return"
Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4445/28/execution/node/72/log |
Looks like the YAML params.get function changed between avocado 52 and 69? It used to return a 2-value tuple. So instead of just accesing the key-value pair via array indexing -- we use next(iter()) instead. https://avocado-framework.readthedocs.io/en/52.0/release_notes/52_0.html https://avocado-framework.readthedocs.io/en/53.0/release_notes/53_0.html https://avocado-framework.readthedocs.io/en/54.0/release_notes/54_0.html .... https://avocado-framework.readthedocs.io/en/69.0/release_notes/69_0.html Skip-build-leap15-rpm: true Skip-func-test-leap15: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-unit-tests: true Skip-nlt: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
We process env var's in YAML file from, so always call TestWithoutServers.setUp() in cart Python test scripts. Skip-build-leap15-rpm: true Skip-func-test-leap15: true Skip-build-ubuntu-clang: true Skip-build-leap15-icc: true Skip-coverity-test: true Quick-build: true Test-tag: cart Skip-unit-tests: true Skip-nlt: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4445/29/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules
FYI: Errors found in lines not modified in the patch:
src/tests/ftest/cart/corpc/cart_corpc_five_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/corpc/cart_corpc_one_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/corpc/cart_corpc_two_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/ctl/cart_ctl_one_node.py:68:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/ghost_rank_rpc/cart_ghost_rank_prc_one_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/group_test/group_test.py:53:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/iv/cart_iv_one_node.py:540:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/iv/cart_iv_two_node.py:301:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/nopmix_launcher/cart_nopmix_launcher_one_node.py:67:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/rpc/cart_rpc_one_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/rpc/cart_rpc_two_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/selftest/cart_selftest_three_node.py:72:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/util/cart_utils.py:56:
(pylint-no-else-break) Unnecessary "else" after "break"
src/tests/ftest/cart/util/cart_utils.py:98:
(pylint-no-else-break) Unnecessary "else" after "break"
src/tests/ftest/cart/util/cart_utils.py:223:
(pylint-no-else-return) Unnecessary "else" after "return"
Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4445/29/execution/node/72/log |
Expected 12 spaces, but only found 10. Skip-build: true Skip-build-centos7-gcc: true Skip-build-centos7-gcc-debug: true Skip-build-centos7-gcc-release: true Skip-build-centos8-gcc-dev: true Skip-build-leap15-gcc: true Skip-build-leap15-icc: true Skip-build-ubuntu-clang: true Skip-bullseye: true Skip-coverity-test: true Skip-func-hw-test: true Skip-func-test: true Skip-func-test-vm: true Skip-nlt: true Skip-run_test: true Skip-scan-centos-rpms: true Skip-test: true Skip-test-centos-rpms: true Skip-unit-test: true Skip-unit-test-memcheck: true Skip-unit-tests: true Signed-off-by: Ethan Mallove <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. No errors found by checkpatch.
FYI: Errors found in lines not modified in the patch:
src/tests/ftest/cart/corpc/cart_corpc_five_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/corpc/cart_corpc_one_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/corpc/cart_corpc_two_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/ctl/cart_ctl_one_node.py:68:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/ghost_rank_rpc/cart_ghost_rank_prc_one_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/group_test/group_test.py:53:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/iv/cart_iv_one_node.py:540:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/iv/cart_iv_two_node.py:301:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/nopmix_launcher/cart_nopmix_launcher_one_node.py:67:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/rpc/cart_rpc_one_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/rpc/cart_rpc_two_node.py:52:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/selftest/cart_selftest_three_node.py:72:
(pylint-undefined-variable) Undefined variable 'main'
src/tests/ftest/cart/util/cart_utils.py:56:
(pylint-no-else-break) Unnecessary "else" after "break"
src/tests/ftest/cart/util/cart_utils.py:98:
(pylint-no-else-break) Unnecessary "else" after "break"
src/tests/ftest/cart/util/cart_utils.py:223:
(pylint-no-else-return) Unnecessary "else" after "return"
Tests pass, and logs show env vars are read in from YAML file. (Note: Last commit resolves pylint indentation issues.) |
PR-4445 breaks LEAP 15 tests: https://build.hpdd.intel.com/job/daos-stack/job/daos/job/master/4073/ Resurrrecting this branch: #4445 Test-tag: cart Skip-coverity-test: true Skip-func-hw-test-small: true Skip-func-hw-test-medium: true Skip-func-hw-test-large: true Signed-off-by: Ethan Mallove <[email protected]>
No description provided.