Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve the performance of low cardinality groupby #16619

Merged
merged 175 commits into from
Nov 8, 2024
Merged

Improve the performance of low cardinality groupby #16619

merged 175 commits into from
Nov 8, 2024

Conversation

PointKernel
Copy link
Member

@PointKernel PointKernel commented Aug 20, 2024 •
edited
Loading

Description

This PR enhances groupby performance for low-cardinality input cases. When applicable, it leverages shared memory for initial aggregation, followed by global memory aggregation to reduce atomic contention and improve performance.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@PointKernel PointKernel added 2 - In Progress Currently a work in progress libcudf Affects libcudf (C++/CUDA) code. CMake CMake build issue Performance Performance related issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Aug 20, 2024
@github-actions github-actions bot removed the CMake CMake build issue label Aug 20, 2024
@PointKernel PointKernel mentioned this pull request Aug 21, 2024
Merged
3 tasks
@PointKernel PointKernel self-assigned this Aug 21, 2024
rapids-bot bot pushed a commit that referenced this pull request Aug 27, 2024
@PointKernel
Add num_multiprocessors utility (#16628)
2d494ed 
This PR introduces a new `num_multiprocessors` utility and moves the existing `elements_per_thread` host utility to the new `cuda.hpp` header. 

Needed by #16619.

Authors:
  - Yunsong Wang (https://github.com/PointKernel)

Approvers:
  - David Wendt (https://github.com/davidwendt)
  - Mark Harris (https://github.com/harrism)

URL: #16628
@PointKernel PointKernel added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Oct 31, 2024
@PointKernel
Copy link
Member Author

This PR is ready for review.

davidwendt
davidwendt approved these changes Nov 6, 2024
View reviewed changes
Copy link
Contributor

@davidwendt davidwendt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Just a couple suggestions.

cpp/src/groupby/hash/compute_shared_memory_aggs.cu Outdated Show resolved Hide resolved
cpp/src/groupby/hash/single_pass_functors.cuh Outdated Show resolved Hide resolved
hyperbolic2346
hyperbolic2346 requested changes Nov 8, 2024
View reviewed changes
Copy link
Contributor

@hyperbolic2346 hyperbolic2346 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure about the copyright data, but otherwise looks good.

cpp/src/groupby/hash/single_pass_functors.cuh Outdated Show resolved Hide resolved
vyasr
vyasr approved these changes Nov 8, 2024
View reviewed changes
Copy link
Contributor

@vyasr vyasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cmake approval.

@PointKernel
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit 2e0d2d6 into rapidsai:branch-24.12 Nov 8, 2024
105 checks passed
@PointKernel PointKernel deleted the shm-groupby branch November 8, 2024 21:15
@GregoryKimball GregoryKimball mentioned this pull request Nov 18, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davidwendt davidwendt davidwendt approved these changes

@karthikeyann karthikeyann karthikeyann approved these changes

@hyperbolic2346 hyperbolic2346 hyperbolic2346 approved these changes

@vyasr vyasr vyasr approved these changes

@harrism harrism Awaiting requested review from harrism harrism is a code owner automatically assigned from rapidsai/cudf-cpp-codeowners

@mhaseeb123 mhaseeb123 Awaiting requested review from mhaseeb123

Assignees

@hyperbolic2346 hyperbolic2346

@PointKernel PointKernel

Labels
3 - Ready for Review Ready for review by team CMake CMake build issue improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change Performance Performance related issue
Projects
cuDF Python
Status: Done
libcudf
Status: Landed
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@PointKernel @harrism @GregoryKimball @mhaseeb123 @vyasr @hyperbolic2346 @karthikeyann @davidwendt