Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable all disabled/skipped pandas tests for cudf.pandas #15445

Closed
wants to merge 3 commits into from
Closed

Re-enable all disabled/skipped pandas tests for cudf.pandas #15445

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d848b66
Re-enable all disabled/skipped pandas tests for cudf.pandas
mroeschke Apr 3, 2024
a58dc94
Merge remote-tracking branch 'upstream/branch-24.06' into pandas/test…
mroeschke Apr 4, 2024
9e716fb
Ignore tslibs/test_parsing
mroeschke Apr 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 3 additions & 111 deletions python/cudf/cudf/pandas/scripts/run-pandas-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,19 +22,6 @@ set -euo pipefail
# of Pandas installed.
PANDAS_VERSION=$(python -c "import pandas; print(pandas.__version__)")

PYTEST_IGNORES="--ignore=tests/io/test_user_agent.py \
Copy link
Contributor

@galipremsagar galipremsagar Apr 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Starting pandas-2, I for sure know we have hangs in this group of tests, we will need to investigate those hangs hence opened separate issues for each one so that it's easier to enable and check each file/directory separately

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah although locally I am not seeing the hangs, I can see it in this PR now. I may have it narrowed it down to tests/tslibs/test_parsing.py

--ignore=tests/interchange/test_impl.py \
--ignore=tests/window/test_dtypes.py \
--ignore=tests/strings/test_api.py \
--ignore=tests/window/test_numba.py \
--ignore=tests/window \
--ignore=tests/io/pytables \
--ignore=tests/plotting \
--ignore=tests/scalar \
--ignore=tests/series/test_arithmetic.py \
--ignore=tests/tslibs/test_parsing.py \
--ignore=tests/io/parser/common/test_read_errors.py"

mkdir -p pandas-testing
cd pandas-testing

Expand All @@ -60,8 +47,6 @@ if [ ! -d "pandas-tests" ]; then
[tool.pytest.ini_options]
xfail_strict = true
filterwarnings = [
"error:Sparse:FutureWarning",
"error:The SparseArray:FutureWarning",
# Will be fixed in numba 0.56: https://github.com/numba/numba/issues/7758
"ignore:`np.MachAr` is deprecated:DeprecationWarning:numba",
]
Expand All @@ -72,7 +57,7 @@ markers = [
"db: tests requiring a database (mysql or postgres)",
"clipboard: mark a pd.read_clipboard test",
"arm_slow: mark a test as slow for arm64 architecture",
"arraymanager: mark a test to run with ArrayManager enabled",
"skip_ubsan: Tests known to fail UBSAN check"
]
EOF
# append the contents of patch-confest.py to conftest.py
Expand Down Expand Up @@ -100,105 +85,12 @@ cat ../python/cudf/cudf/pandas/scripts/conftest-patch.py >> pandas-tests/conftes
# Run the tests
cd pandas-tests/

# TODO: Get a postgres & mysql container set up on the CI
# test_overwrite_warns unsafely patchs over Series.mean affecting other tests when run in parallel
# test_complex_series_frame_alignment randomly selects a DataFrames and axis to test but particular random selection(s) always fails
# test_numpy_ufuncs_basic compares floating point values to unbounded precision, sometimes leading to failures
TEST_NUMPY_UFUNCS_BASIC_FLAKY="not test_numpy_ufuncs_basic[float-exp] \
and not test_numpy_ufuncs_basic[float-exp2] \
and not test_numpy_ufuncs_basic[float-expm1] \
and not test_numpy_ufuncs_basic[float-log] \
and not test_numpy_ufuncs_basic[float-log2] \
and not test_numpy_ufuncs_basic[float-log10] \
and not test_numpy_ufuncs_basic[float-log1p] \
and not test_numpy_ufuncs_basic[float-sqrt] \
and not test_numpy_ufuncs_basic[float-sin] \
and not test_numpy_ufuncs_basic[float-cos] \
and not test_numpy_ufuncs_basic[float-tan] \
and not test_numpy_ufuncs_basic[float-arcsin] \
and not test_numpy_ufuncs_basic[float-arccos] \
and not test_numpy_ufuncs_basic[float-arctan] \
and not test_numpy_ufuncs_basic[float-sinh] \
and not test_numpy_ufuncs_basic[float-cosh] \
and not test_numpy_ufuncs_basic[float-tanh] \
and not test_numpy_ufuncs_basic[float-arcsinh] \
and not test_numpy_ufuncs_basic[float-arccosh] \
and not test_numpy_ufuncs_basic[float-arctanh] \
and not test_numpy_ufuncs_basic[float-deg2rad] \
and not test_numpy_ufuncs_basic[float-rad2deg] \
and not test_numpy_ufuncs_basic[num_float64-exp] \
and not test_numpy_ufuncs_basic[num_float64-exp2] \
and not test_numpy_ufuncs_basic[num_float64-expm1] \
and not test_numpy_ufuncs_basic[num_float64-log] \
and not test_numpy_ufuncs_basic[num_float64-log2] \
and not test_numpy_ufuncs_basic[num_float64-log10] \
and not test_numpy_ufuncs_basic[num_float64-log1p] \
and not test_numpy_ufuncs_basic[num_float64-sqrt] \
and not test_numpy_ufuncs_basic[num_float64-sin] \
and not test_numpy_ufuncs_basic[num_float64-cos] \
and not test_numpy_ufuncs_basic[num_float64-tan] \
and not test_numpy_ufuncs_basic[num_float64-arcsin] \
and not test_numpy_ufuncs_basic[num_float64-arccos] \
and not test_numpy_ufuncs_basic[num_float64-arctan] \
and not test_numpy_ufuncs_basic[num_float64-sinh] \
and not test_numpy_ufuncs_basic[num_float64-cosh] \
and not test_numpy_ufuncs_basic[num_float64-tanh] \
and not test_numpy_ufuncs_basic[num_float64-arcsinh] \
and not test_numpy_ufuncs_basic[num_float64-arccosh] \
and not test_numpy_ufuncs_basic[num_float64-arctanh] \
and not test_numpy_ufuncs_basic[num_float64-deg2rad] \
and not test_numpy_ufuncs_basic[num_float64-rad2deg] \
and not test_numpy_ufuncs_basic[num_float32-exp] \
and not test_numpy_ufuncs_basic[num_float32-exp2] \
and not test_numpy_ufuncs_basic[num_float32-expm1] \
and not test_numpy_ufuncs_basic[num_float32-log] \
and not test_numpy_ufuncs_basic[num_float32-log2] \
and not test_numpy_ufuncs_basic[num_float32-log10] \
and not test_numpy_ufuncs_basic[num_float32-log1p] \
and not test_numpy_ufuncs_basic[num_float32-sqrt] \
and not test_numpy_ufuncs_basic[num_float32-sin] \
and not test_numpy_ufuncs_basic[num_float32-cos] \
and not test_numpy_ufuncs_basic[num_float32-tan] \
and not test_numpy_ufuncs_basic[num_float32-arcsin] \
and not test_numpy_ufuncs_basic[num_float32-arccos] \
and not test_numpy_ufuncs_basic[num_float32-arctan] \
and not test_numpy_ufuncs_basic[num_float32-sinh] \
and not test_numpy_ufuncs_basic[num_float32-cosh] \
and not test_numpy_ufuncs_basic[num_float32-tanh] \
and not test_numpy_ufuncs_basic[num_float32-arcsinh] \
and not test_numpy_ufuncs_basic[num_float32-arccosh] \
and not test_numpy_ufuncs_basic[num_float32-arctanh] \
and not test_numpy_ufuncs_basic[num_float32-deg2rad] \
and not test_numpy_ufuncs_basic[num_float32-rad2deg] \
and not test_numpy_ufuncs_basic[nullable_float-exp] \
and not test_numpy_ufuncs_basic[nullable_float-exp2] \
and not test_numpy_ufuncs_basic[nullable_float-expm1] \
and not test_numpy_ufuncs_basic[nullable_float-log] \
and not test_numpy_ufuncs_basic[nullable_float-log2] \
and not test_numpy_ufuncs_basic[nullable_float-log10] \
and not test_numpy_ufuncs_basic[nullable_float-log1p] \
and not test_numpy_ufuncs_basic[nullable_float-sqrt] \
and not test_numpy_ufuncs_basic[nullable_float-sin] \
and not test_numpy_ufuncs_basic[nullable_float-cos] \
and not test_numpy_ufuncs_basic[nullable_float-tan] \
and not test_numpy_ufuncs_basic[nullable_float-arcsin] \
and not test_numpy_ufuncs_basic[nullable_float-arccos] \
and not test_numpy_ufuncs_basic[nullable_float-arctan] \
and not test_numpy_ufuncs_basic[nullable_float-sinh] \
and not test_numpy_ufuncs_basic[nullable_float-cosh] \
and not test_numpy_ufuncs_basic[nullable_float-tanh] \
and not test_numpy_ufuncs_basic[nullable_float-arcsinh] \
and not test_numpy_ufuncs_basic[nullable_float-arccosh] \
and not test_numpy_ufuncs_basic[nullable_float-arctanh] \
and not test_numpy_ufuncs_basic[nullable_float-deg2rad] \
and not test_numpy_ufuncs_basic[nullable_float-rad2deg]"

# TODO: Remove "not db" once a postgres & mysql container set up on the CI
PANDAS_CI="1" timeout 30m python -m pytest -p cudf.pandas \
-v -m "not single_cpu and not db" \
-k "not test_overwrite_warns and not test_complex_series_frame_alignment and not test_to_parquet_gcs_new_file and not test_qcut_nat and not test_add and not test_ismethods and $TEST_NUMPY_UFUNCS_BASIC_FLAKY" \
--import-mode=importlib \
--ignore=tests/tslibs/test_parsing.py \
-o xfail_strict=True \
${PYTEST_IGNORES} \
"$@" || [ $? = 1 ] # Exit success if exit code was 1 (permit test failures but not other errors)

mv *.json ..
Expand Down
Loading