Skip to content

Commit

Permalink
[FEA] Add DELTA_BINARY_PACKED decoding support to Parquet reader (#13637
Browse files Browse the repository at this point in the history
)

Part of #13501. This adds support for decoding Parquet pages that are DELTA_BINARY_PACKED.

In addition to adding delta support, this PR incorporates changes introduced in #13622, such as using a mask to determine which decoding kernels to run, and adding parameters to  the `page_state_buffers_s` struct to reduce the amount of shared memory used.

Authors:
  - Ed Seidl (https://github.com/etseidl)
  - Vukasin Milovanovic (https://github.com/vuule)

Approvers:
  - Vukasin Milovanovic (https://github.com/vuule)
  - https://github.com/nvdbaranec
  - Bradley Dice (https://github.com/bdice)
  - GALI PREM SAGAR (https://github.com/galipremsagar)

URL: #13637
  • Loading branch information
etseidl authored Aug 23, 2023
1 parent abac227 commit c39c04d
Show file tree
Hide file tree
Showing 15 changed files with 1,376 additions and 627 deletions.
2 changes: 2 additions & 0 deletions cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -390,10 +390,12 @@ add_library(
src/io/orc/writer_impl.cu
src/io/parquet/compact_protocol_reader.cpp
src/io/parquet/compact_protocol_writer.cpp
src/io/parquet/decode_preprocess.cu
src/io/parquet/page_data.cu
src/io/parquet/chunk_dict.cu
src/io/parquet/page_enc.cu
src/io/parquet/page_hdr.cu
src/io/parquet/page_delta_decode.cu
src/io/parquet/page_string_decode.cu
src/io/parquet/predicate_pushdown.cpp
src/io/parquet/reader.cpp
Expand Down
Loading

0 comments on commit c39c04d

Please sign in to comment.