Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] reading a parquet file with no rows and nested types can throw an error #8323

Closed
revans2 opened this issue May 21, 2021 · 2 comments · Fixed by #8350
Closed

[BUG] reading a parquet file with no rows and nested types can throw an error #8323

revans2 opened this issue May 21, 2021 · 2 comments · Fixed by #8350
Assignees
@nvdbaranec
Labels
bug Something isn't working cuIO cuIO issue libcudf Affects libcudf (C++/CUDA) code.

Comments

@revans2
Copy link
Contributor

revans2 commented May 21, 2021 •
edited by harrism
Loading

Describe the bug
Because of the bug fixed by #8314 I decided to take a look at everywhere that make_empty_column is used without an explicit type id.

cudf/cpp/src/io/parquet/reader_impl.cu

Line 1575 in de579a5

out_columns.emplace_back(make_empty_column(_output_columns[i].type));

is one of those places, but it looks like all of the readers have similar problems, just not all of them support nested types yet.

Steps/Code to reproduce bug
Just read the attached file empty_nested.snappy.parquet.zip and you get an error like.

cuDF failure at: ../src/column/column_factories.cpp:66: make_empty_column is invalid to call on nested types

Expected behavior
You get the correct empty table back.
@revans2 revans2 added bug Something isn't working Needs Triage Need team to review and classify labels May 21, 2021
@vuule vuule added cuIO cuIO issue libcudf Affects libcudf (C++/CUDA) code. and removed Needs Triage Need team to review and classify labels May 24, 2021
@harrism
Copy link
Member

harrism commented May 25, 2021

@nvdbaranec do you want to tackle #8322 together with this one?

@nvdbaranec
Copy link
Contributor

@nvdbaranec do you want to tackle #8322 together with this one?

Yep, I'll handle it. (Separate PR though)

@nvdbaranec nvdbaranec mentioned this issue May 25, 2021
Merged
rapids-bot bot pushed a commit that referenced this issue May 27, 2021
@nvdbaranec
Handle nested column types properly for empty parquet files. (#8350)
bcb1237 
Fixes:  #8323

Also fixes a recently introduced bug in the test column equality checker.  The code was previously relying on accesses to device memory being transparently handled by `thrust::device_vector`

Authors:
  - https://github.com/nvdbaranec

Approvers:
  - Mike Wilson (https://github.com/hyperbolic2346)
  - Devavret Makkar (https://github.com/devavret)
  - Nghia Truong (https://github.com/ttnghia)

URL: #8350
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nvdbaranec nvdbaranec

Labels
bug Something isn't working cuIO cuIO issue libcudf Affects libcudf (C++/CUDA) code.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Handle nested column types properly for empty parquet files. nvdbaranec/cudf
4 participants
@harrism @revans2 @vuule @nvdbaranec