Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deallocate encoded data in ORC writer immediately after compression #12770

Merged
merged 7 commits into from
Mar 4, 2023
Merged

Deallocate encoded data in ORC writer immediately after compression #12770

merged 7 commits into from
Mar 4, 2023

Conversation

vuule
Copy link
Contributor

@vuule vuule commented Feb 14, 2023 •
edited
Loading

Description

Reduces total memory footprint over the execution time (but not peak memory usage, which happens during compression). This change helps overall memory usage when writing multiple files in parallel using the same GPU.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@github-actions github-actions bot added the libcudf Affects libcudf (C++/CUDA) code. label Feb 14, 2023
@vuule vuule self-assigned this Feb 14, 2023
@vuule vuule added non-breaking Non-breaking change improvement Improvement / enhancement to an existing function labels Feb 14, 2023
@vuule vuule marked this pull request as ready for review February 16, 2023 01:09
@vuule vuule requested a review from a team as a code owner February 16, 2023 01:09
@vuule vuule requested review from bdice and divyegala February 16, 2023 01:09
@vuule vuule requested review from davidwendt and ttnghia February 28, 2023 23:38
@vuule
Copy link
Contributor Author

vuule commented Mar 4, 2023

/merge

@rapids-bot rapids-bot bot merged commit 00c887b into rapidsai:branch-23.04 Mar 4, 2023
@vuule vuule deleted the orc-write-early-enc-dealloc branch March 4, 2023 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davidwendt davidwendt davidwendt approved these changes

@ttnghia ttnghia ttnghia approved these changes

@bdice bdice Awaiting requested review from bdice bdice is a code owner automatically assigned from rapidsai/cudf-cpp-codeowners

@divyegala divyegala Awaiting requested review from divyegala divyegala is a code owner automatically assigned from rapidsai/cudf-cpp-codeowners

Assignees

@vuule vuule

Labels
improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vuule @ttnghia @davidwendt