Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch pytest-xdist algorithm to worksteal #5792

Merged
merged 7 commits into from
Mar 6, 2024
Merged

Switch pytest-xdist algorithm to worksteal #5792

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
9835fb7
Switch `pytest-xdist` algorithm to `worksteal`
bdice Mar 5, 2024
93b4cd7
Refactor to use run scripts.
bdice Mar 5, 2024
3f8dba5
Update ci/test_python_singlegpu.sh
bdice Mar 5, 2024
d32f802
Set up exit codes.
bdice Mar 5, 2024
cf92c59
Merge branch 'pytest-xdist-worksteal' of github.com:bdice/cuml into p…
bdice Mar 5, 2024
8dd5a7e
Merge branch 'branch-24.04' into pytest-xdist-worksteal
bdice Mar 6, 2024
bba216a
Use python -m pytest.
bdice Mar 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions ci/test_python_singlegpu.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,15 @@ EXITCODE=0
trap "EXITCODE=1" ERR
set +e

rapids-logger "pytest cuml single GPU..."
rapids-logger "pytest cuml single GPU"
./ci/run_cuml_singlegpu_pytests.sh \
--numprocesses=8 \
--dist=worksteal \
--junitxml="${RAPIDS_TESTS_DIR}/junit-cuml.xml" \
--cov-config=../../.coveragerc \
--cov=cuml \
--cov-report=xml:"${RAPIDS_COVERAGE_DIR}/cuml-coverage.xml" \
--cov-report=term \
--cov-report=term

rapids-logger "memory leak pytests..."
bdice marked this conversation as resolved.
Show resolved Hide resolved

Expand All @@ -29,7 +30,7 @@ rapids-logger "memory leak pytests..."
--cov=cuml \
--cov-report=xml:"${RAPIDS_COVERAGE_DIR}/cuml-memleak-coverage.xml" \
--cov-report=term \
-m "memleak" \
-m "memleak"

rapids-logger "Test script exiting with value: $EXITCODE"
exit ${EXITCODE}
16 changes: 15 additions & 1 deletion ci/test_wheel.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,19 @@ python -m pip install $(echo ./dist/cuml*.whl)[test]
if [[ "$(arch)" == "aarch64" && "${RAPIDS_BUILD_TYPE}" == "pull-request" ]]; then
python ci/wheel_smoke_test_cuml.py
else
python -m pytest ./python/cuml/tests -k 'not test_sparse_pca_inputs' -n 4 --ignore=python/cuml/tests/dask && python -m pytest ./python/cuml/tests -k 'test_sparse_pca_inputs' && python -m pytest ./python/cuml/tests/dask
rapids-logger "pytest cuml single GPU"
./ci/run_cuml_singlegpu_pytests.sh \
--numprocesses=8 \
--dist=worksteal \
-k 'not test_sparse_pca_inputs' \
--junitxml="${RAPIDS_TESTS_DIR}/junit-cuml.xml"

# Run test_sparse_pca_inputs separately
./ci/run_cuml_singlegpu_pytests.sh \
-k 'test_sparse_pca_inputs' \
--junitxml="${RAPIDS_TESTS_DIR}/junit-cuml-sparse-pca.xml"

rapids-logger "pytest cuml-dask"
./ci/run_cuml_dask_pytests.sh \
--junitxml="${RAPIDS_TESTS_DIR}/junit-cuml-dask.xml"
fi
Loading