Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix cudf::hash_partition kernel launch error with decimal128 types #12863

Merged

Conversation

davidwendt
Copy link
Contributor

Description

Fixes cudf::hash_partition error when using decimal128 column types. The internal optimized path, copy_block_partitions, uses shared-memory for copying fixed-width type column elements. For int128_t type, the shared-memory needed (~64KB) is larger than the maximum size (~48KB) allowed causing a kernel launch failure.

The optimized path is now restricted to only fixed-width types int64_t and below. The int128_t column types will fall through to the gather-map pattern instead. Accommodating this type in the existing copy-block implementation would likely penalize the performance of the other fixed-width types. If the new implementation becomes insufficient, we could explore a special optimized path in the future for the single type int128_t.

An existing gtest for fixed-point types was updated to include a decimal128 column to catch this kind of error in the future.

Closes #12852

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@davidwendt davidwendt added bug Something isn't working 2 - In Progress Currently a work in progress libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change labels Feb 28, 2023
@davidwendt davidwendt self-assigned this Feb 28, 2023
@davidwendt
Copy link
Contributor Author

@revans2 @firestarman Could you test out this fix with the Spark testcase from #12852 ?

@firestarman
Copy link
Contributor

firestarman commented Mar 1, 2023

@davidwendt This PR can fix #12852. Thanks a lot.

@davidwendt davidwendt added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Mar 1, 2023
@davidwendt davidwendt marked this pull request as ready for review March 1, 2023 23:37
@davidwendt davidwendt requested a review from a team as a code owner March 1, 2023 23:37
cpp/src/partitioning/partitioning.cu Outdated Show resolved Hide resolved
@davidwendt
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit a78fdd5 into rapidsai:branch-23.04 Mar 7, 2023
@davidwendt davidwendt deleted the bug-hashpartition-decimal128 branch March 7, 2023 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team bug Something isn't working libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] The hashPartition API may return corrupted data when there are columns with type of DECIMAL128.
6 participants