Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error Infering Schema for LogicalType::UNKNOWN #1557

Closed
tustvold opened this issue Apr 13, 2022 · 0 comments · Fixed by #1558
Closed

Error Infering Schema for LogicalType::UNKNOWN #1557

tustvold opened this issue Apr 13, 2022 · 0 comments · Fixed by #1558
Labels
bug parquet Changes to the parquet crate

Comments

@tustvold
Copy link
Contributor

Describe the bug

Currently in the abscence of an arrow schema, LogicalType::UNKNOWN results in the following error message

Unable to convert parquet INT32 logical type Some(UNKNOWN(NullType)) or converted type NONE

To Reproduce

Create a parquet file with a null array and strip out the arrow schema.

Expected behavior

As per the docs this should be inferred as the null type
@tustvold tustvold added the bug label Apr 13, 2022
tustvold added a commit to tustvold/arrow-rs that referenced this issue Apr 13, 2022
@tustvold
Add option to skip decoding arrow metadata from parquet (apache#1459)
019cd07 
Fix inference from null logical type (apache#1557)

Replace some `&Option<T>` with `Option<&T>` (apache#1556)
@tustvold tustvold mentioned this issue Apr 13, 2022
Merged
tustvold added a commit that referenced this issue Apr 14, 2022
@tustvold @alamb
Add option to skip decoding arrow metadata from parquet (#1459) (#1558)
d701939 
* Add option to skip decoding arrow metadata from parquet (#1459)

Fix inference from null logical type (#1557)

Replace some `&Option<T>` with `Option<&T>` (#1556)

* Update parquet/src/arrow/arrow_reader.rs

Co-authored-by: Andrew Lamb <[email protected]>

* Fmt

Co-authored-by: Andrew Lamb <[email protected]>
@alamb alamb added the parquet Changes to the parquet crate label Apr 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug parquet Changes to the parquet crate
Projects
None yet
Milestone
No milestone
2 participants
@alamb @tustvold