Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Centralize chunked reading code in the parquet reader to reader_impl_chunking.cu #14262

Merged

Conversation

nvdbaranec
Copy link
Contributor

As a precursor to further chunked reader work, this PR centralizes chunk-related code (mostly from the reader::impl class) into reader_impl_chunking.cu and reader_impl_chunking.hpp. Also cleans up some variable naming and locations in reader::impl and the file_intermediate_data and pass_intermediate_data classes.

… been changed to ::parquet::detail, ::parquet::gpu has been renamed to ::parquet::detail, and several detail-style files which

were just using ::parquet have been moved into parquet::detail.
@nvdbaranec nvdbaranec added libcudf Affects libcudf (C++/CUDA) code. cuIO cuIO issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Oct 9, 2023
@nvdbaranec nvdbaranec requested review from a team as code owners October 9, 2023 15:58
@nvdbaranec nvdbaranec requested review from vyasr and vuule October 9, 2023 15:58
@github-actions github-actions bot added the CMake CMake build issue label Oct 9, 2023
Copy link
Contributor

@vuule vuule left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, assuming this is mostly code shuffle. Hard to compare when the code is moved between files.

@nvdbaranec
Copy link
Contributor Author

Looks good, assuming this is mostly code shuffle. Hard to compare when the code is moved between files.

Yes. Most of it is just the big blob from reader_impl_preprocess.cu being moved into read_impl_chunking.cu. No functionality changes.

Copy link
Contributor

@robertmaynard robertmaynard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving CMake changes

@nvdbaranec nvdbaranec requested a review from ttnghia October 10, 2023 15:31
@vuule
Copy link
Contributor

vuule commented Oct 10, 2023

/merge

@rapids-bot rapids-bot bot merged commit b4fd77b into rapidsai:branch-23.12 Oct 10, 2023
57 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CMake CMake build issue cuIO cuIO issue improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants