Add compound aggregations to cudf::segmented_reduce #12573
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
davidwendt
added
feature request
|
This will target 23.04 once it is ready. |
davidwendt
commented
Jan 18, 2023
| @@ -229,92 +228,6 @@ std::unique_ptr<scalar> reduce(InputIterator d_in, | |||
| return std::unique_ptr<scalar>(result); | |||
| } | |||
|
|
|||
| /** | |||
There was a problem hiding this comment.
These functions were moved to segmented_reduction.cuh
davidwendt
commented
Jan 18, 2023
| @@ -338,171 +338,5 @@ std::unique_ptr<scalar> merge_sets( | |||
| rmm::cuda_stream_view stream, | |||
| rmm::mr::device_memory_resource* mr = rmm::mr::get_current_device_resource()); | |||
|
|
|||
| /** | |||
There was a problem hiding this comment.
These functions were moved to segmented_reduction_functions.hpp
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## branch-23.04 #12573 +/- ##
===============================================
Coverage ? 85.81%
===============================================
Files ? 158
Lines ? 25153
Branches ? 0
===============================================
Hits ? 21586
Misses ? 3567
Partials ? 0 Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
bdice
reviewed
Jan 20, 2023
…cudf into reduction-segmented-compound
robertmaynard
approved these changes
Jan 30, 2023
ajschmidt8
approved these changes
Jan 30, 2023
hyperbolic2346
requested changes
Jan 30, 2023
hyperbolic2346
approved these changes
Jan 30, 2023
…cudf into reduction-segmented-compound
3 tasks
…cudf into reduction-segmented-compound
…cudf into reduction-segmented-compound
3 tasks
davidwendt
commented
Feb 2, 2023
bdice
reviewed
Feb 2, 2023
There was a problem hiding this comment.
Reviewed with @davidwendt. A few comments attached. Great work!
bdice
requested changes
Feb 2, 2023
bdice
approved these changes
Feb 3, 2023
|
/merge |
rapids-bot bot
pushed a commit
that referenced
this pull request
Feb 7, 2023
Reworks some internal source specific to fixed-point types using `cudf::reduce` by removing the duplicated code logic. This was found while working on #12573 and #10432. Since the fix is requires no dependencies, this separate PR is used to minimize code review churn. This should help with code consistency with the fixed-point-specific logic when added to segmented-reduction. No function has changed so all existing gtests are adequate. Authors: - David Wendt (https://github.com/davidwendt) Approvers: - Yunsong Wang (https://github.com/PointKernel) - Bradley Dice (https://github.com/bdice) URL: #12652
rapids-bot bot
pushed a commit
that referenced
this pull request
Feb 23, 2023
Depends on #12573 Adds additional support for fixed-point types in `cudf::segmented_reduce` for simple aggregations: sum, product, and sum-of-squares. Reference: #10432 Authors: - David Wendt (https://github.com/davidwendt) Approvers: - Mike Wilson (https://github.com/hyperbolic2346) - Nghia Truong (https://github.com/ttnghia) - Bradley Dice (https://github.com/bdice) URL: #12680
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adds mean, variance, and standard deviation aggregation support to
cudf::segmented_reduce. These are compound (multi-step) aggregations and are modeled after the same aggregations supported butcudf::reduce. Once this approved and merged, the visitor pattern for this approach will be reworked for bothcudf::reduceandcudf::segmented_reduceas per #10432.The source tree for
src/reductionshas been adjusted to put all segmented-reduce source files intosrc/reductions/segmentedand removing thesegmented_prefix from those file names.Also, the segmented-reduce functions have been moved from
cudf/detail/reduction_functions.hppinto their owncudf/detail/segmented_reduction_functions.hpp. Likewise, the segmented-reduce CUB calls have been moved fromcudf/detail/reduction.cuhto the newcudf/detail/segmented_reduction.cuhto help minimize including CUB headers.Additionally, the sum-of-squares aggregation is also included since it was a simple reduction only requiring the appropriate aggregation class registration and source file.
Finally, gtests are added for these new types. The compound types only support floating-point outputs.
Follow on PRs will address the visitor pattern already mentioned above as well as additional data types. Discussion on additional aggregations will occur in the reference issue #10432.
Checklist