Skip to content

Commit

Permalink
Reduce error handling verbosity in CI tests scripts (#1113)
Browse files Browse the repository at this point in the history
This PR adds a less verbose [trap method](https://github.com/rapidsai/cugraph/blob/f2b081075704aabc789603e14ce552eac3fbe692/ci/test.sh#L19), for error handling to help ensure that we capture all potential error codes in our test scripts, and works as follows:

- setting an environment variable, EXITCODE, with a default value of 0
- setting a trap statement triggered by ERR signals which will set EXITCODE=1 when any commands return a non-zero exit code
cc @ajschmidt8

Authors:
  - Ajay Thorve (https://github.com/AjayThorve)

Approvers:
  - AJ Schmidt (https://github.com/ajschmidt8)

URL: #1113
  • Loading branch information
AjayThorve authored Feb 10, 2023
1 parent b0e4f2d commit f44ba33
Showing 1 changed file with 6 additions and 22 deletions.
28 changes: 6 additions & 22 deletions ci/test_python.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# Copyright (c) 2022, NVIDIA CORPORATION.
# Copyright (c) 2022-2023, NVIDIA CORPORATION.

set -euo pipefail

Expand All @@ -24,17 +24,18 @@ PYTHON_CHANNEL=$(rapids-download-conda-from-s3 python)
RAPIDS_TESTS_DIR=${RAPIDS_TESTS_DIR:-"${PWD}/test-results"}
RAPIDS_COVERAGE_DIR=${RAPIDS_COVERAGE_DIR:-"${PWD}/coverage-results"}
mkdir -p "${RAPIDS_TESTS_DIR}" "${RAPIDS_COVERAGE_DIR}"
SUITEERROR=0

rapids-print-env

rapids-mamba-retry install \
-c "${PYTHON_CHANNEL}" \
--channel "${PYTHON_CHANNEL}" \
dask-cuda

rapids-logger "Check GPU usage"
nvidia-smi

EXITCODE=0
trap "EXITCODE=1" ERR
set +e

rapids-logger "pytest dask-cuda"
Expand All @@ -53,12 +54,6 @@ timeout 30m pytest \
--cov-report=xml:"${RAPIDS_COVERAGE_DIR}/dask-cuda-coverage.xml" \
--cov-report=term \
tests
exitcode=$?

if (( ${exitcode} != 0 )); then
SUITEERROR=${exitcode}
echo "FAILED: 1 or more tests in dask-cuda"
fi
popd

rapids-logger "Run local benchmark"
Expand All @@ -67,23 +62,12 @@ python dask_cuda/benchmarks/local_cudf_shuffle.py \
-d 0 \
--runs 1 \
--backend dask
exitcode=$?

if (( ${exitcode} != 0 )); then
SUITEERROR=${exitcode}
echo "FAILED: Local benchmark with dask comms"
fi

python dask_cuda/benchmarks/local_cudf_shuffle.py \
--partition-size="1 KiB" \
-d 0 \
--runs 1 \
--backend explicit-comms
exitcode=$?

if (( ${exitcode} != 0 )); then
SUITEERROR=${exitcode}
echo "FAILED: Local benchmark with explicit comms"
fi

exit ${SUITEERROR}
rapids-logger "Test script exiting with value: $EXITCODE"
exit ${EXITCODE}

0 comments on commit f44ba33

Please sign in to comment.