-
Notifications
You must be signed in to change notification settings - Fork 915
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix ORC reading of files with struct columns that have null values #9005
Fix ORC reading of files with struct columns that have null values #9005
Conversation
Codecov Report
@@ Coverage Diff @@
## branch-21.10 #9005 +/- ##
===============================================
Coverage ? 10.65%
===============================================
Files ? 114
Lines ? 19077
Branches ? 0
===============================================
Hits ? 2033
Misses ? 17044
Partials ? 0 Continue to review full report at Codecov.
|
@gpucibot merge |
Fixes #8910
Number of values in the null stream of a child column depends on the number of valid elements in the parent column.
This PR changes the reading logic to account for the number of parent null values when parsing child null streams.
Namely, the output row is offset by the number of null values in the parent column, in all previous stripes. To allow efficient parsing, null counts are inclusive_scan'd before the columns in the level are parsed.