Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First pass at adding testing for pylibcudf #15300

Merged
merged 49 commits into from
Apr 1, 2024

Conversation

vyasr
Copy link
Contributor

@vyasr vyasr commented Mar 14, 2024

Description

This PR adds tests of the pylibcudf.copying module along with establishing the infrastructure and best practices for writing pylibcudf tests going forward (and adding associated documentation).

Resolves #15133

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@vyasr vyasr added tests Unit testing for project non-breaking Non-breaking change labels Mar 14, 2024
@vyasr vyasr self-assigned this Mar 14, 2024
@github-actions github-actions bot added the Python Affects Python cuDF API. label Mar 14, 2024
@vyasr vyasr added improvement Improvement / enhancement to an existing function and removed Python Affects Python cuDF API. labels Mar 14, 2024
@github-actions github-actions bot added libcudf Affects libcudf (C++/CUDA) code. Python Affects Python cuDF API. ci labels Mar 15, 2024
docs/cudf/source/developer_guide/pylibcudf.md Outdated Show resolved Hide resolved
python/cudf/cudf/tests/pylibcudf/common/utils.py Outdated Show resolved Hide resolved
python/cudf/cudf/tests/pylibcudf/common/utils.py Outdated Show resolved Hide resolved
python/cudf/cudf/tests/pylibcudf/common/utils.py Outdated Show resolved Hide resolved
ci/test_python_cudf.sh Outdated Show resolved Hide resolved
CUDF_EXPECTS(num_indices % 2 == 0, "Array of indices needs to have an even number of elements.");
CUDF_EXPECTS(num_indices % 2 == 0,
"Array of indices needs to have an even number of elements.",
std::invalid_argument);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: docstring now wants updated to match the actual return value.

Comment on lines 91 to 95
CUDF_EXPECTS(begin >= 0, "Starting index cannot be negative.", std::out_of_range);
CUDF_EXPECTS(
end >= begin, "End index cannot be smaller than the starting index.", std::invalid_argument);
CUDF_EXPECTS(end <= input.size(), "Slice range out of bounds.", std::out_of_range);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: although probably not as part of this PR. We should encode the validation mechanism for slice indices in one function, then all of the error messages would be the same, and if we decide the change our position on end <= input.size() (say) we could do it in one place.

@github-actions github-actions bot removed CMake CMake build issue conda Java Affects Java cuDF API. labels Mar 25, 2024
@vyasr vyasr marked this pull request as ready for review March 25, 2024 22:41
@vyasr vyasr requested review from a team as code owners March 25, 2024 22:41
@vyasr vyasr requested review from shwina, isVoid, bdice and mhaseeb123 March 25, 2024 22:41
@vyasr
Copy link
Contributor Author

vyasr commented Mar 25, 2024

This PR is already getting quite large, so I decided to skip addressing nullable data here. We definitely need to do that, but we can do it in a follow-up, perhaps in @brandon-b-miller's PR adding tests for binops #15279

# A simple wrapper around pytest.raises that defaults to looking for cudf exceptions
match = kwargs.get("match", None)
if match is None:
kwargs["match"] = "CUDF failure at"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note. CUDF being all caps here is kinda weird; should it be either libcudf or cuDF?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a fair question, we could suggest that change to the libcudf team if we wanted. The text comes from https://github.com/rapidsai/cudf/blob/branch-24.06/cpp/include/cudf/utilities/error.hpp#L190 (and the corresponding CUDF_FAIL definition.

return pytest.raises(expected_exception, *args, **kwargs)


# TODO: Consider moving these type utilities into pylibcudf.types itself.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also consider making them methods of plc.DataType

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Do you have a preference? And should we do that in a follow-up PR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do it in a follow up. I'd prefer making them methods as pyarrow (and incidentally Polars) does that. But as Nick would say, it's a weakly held opinion

Copy link
Contributor

@shwina shwina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests themseleves look great! Just a few small questions around tooling. Great work, @vyasr!

@brandon-b-miller
Copy link
Contributor

/merge

@rapids-bot rapids-bot bot merged commit aab6137 into rapidsai:branch-24.06 Apr 1, 2024
68 checks passed
@vyasr vyasr added the pylibcudf Issues specific to the pylibcudf package label May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change pylibcudf Issues specific to the pylibcudf package Python Affects Python cuDF API. tests Unit testing for project
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[FEA] Add tests of pylibcudf
6 participants