Superimpose null masks for STRUCT columns. #9144
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Per rapidsai#5700, when a STRUCT column is constructed, the null mask of the parent column is bitwise-ANDed with that of all its children, such that a null row in the parent column corresponds to nulls in all its children. This is done recursively, allowing grand-child columns to also have nulls at the same row positions. `superimpose_parent_nulls()` makes this functionality available for columns that might not have been constructed through `make_struct_column()`, e.g. with columns received directly from Arrow. It does not require that the `column_view` is modifiable. For a STRUCT `column_view` argument, a new equivalent instance is created, with all its children's null masks modified to account for the parent nulls. `superimpose_parent_nulls()` can be used for all code that assumes that the child null masks account for the nulls in the parents (and grandparents, ad infinitum).
mythrocks
added
feature request
Codecov Report
@@ Coverage Diff @@
## branch-21.10 #9144 +/- ##
===============================================
Coverage ? 10.82%
===============================================
Files ? 115
Lines ? 19122
Branches ? 0
===============================================
Hits ? 2070
Misses ? 17052
Partials ? 0 Continue to review full report at Codecov.
|
ttnghia
reviewed
Aug 30, 2021
ttnghia
reviewed
Aug 30, 2021
jrhemstad
reviewed
Aug 30, 2021
ttnghia
reviewed
Sep 1, 2021
ttnghia
reviewed
Sep 1, 2021
ttnghia
approved these changes
Sep 1, 2021
|
@gpucibot merge |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Per #5700, when a STRUCT column is constructed, the null mask of the parent
column is bitwise-ANDed with that of all its children, such that a null row
in the parent column corresponds to nulls in all its children. This is done
recursively, allowing grand-child columns to also have nulls at the same
row positions.
superimpose_parent_nulls()makes this functionality available for columnsthat might not have been constructed through
make_struct_column(), e.g.with columns received directly from Arrow. It does not require that the
column_viewis modifiable. For a STRUCTcolumn_viewargument, a newequivalent instance is created, with all its children's null masks modified
to account for the parent nulls.
superimpose_parent_nulls()can be used for all code that assumes that thechild null masks account for the nulls in the parents (and grandparents,
ad infinitum).