-
Notifications
You must be signed in to change notification settings - Fork 915
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Explicitly compute null count in concatenate APIs #13104
Explicitly compute null count in concatenate APIs #13104
Conversation
@vuule locally this change passes all tests for concatenation itself but fails some tests in the Parquet reader. Would you be able to help me look into those? I am unsure if the reader is catching some edge cases that the test is missing, or if the Parquet code has some null sanitization bug that is causing the failures. I mention sanitization because the failure specifically occurs for list and struct types. |
Co-authored-by: David Wendt <[email protected]> Co-authored-by: Nghia Truong <[email protected]>
Co-authored-by: Nghia Truong <[email protected]>
|
Tests fail because chunked Parquet reader returns columns with incorrect null count, and the new concatenate relies on this info. |
Depends on #13111 |
/merge |
Description
The total number of nulls in the output can be computed by summing the nulls in the input columns.
Contributes to #11968
Checklist