-
Notifications
You must be signed in to change notification settings - Fork 915
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplify null count checking in column equality comparator #13312
Simplify null count checking in column equality comparator #13312
Conversation
This looks to be safe. As discussed offline with @nvdbaranec, this special function for null counting is to handle non-empty nulls, which we no longer support. The remaining todo here is to add a test with an EDIT: This testing is now explicitly handled by those assertion functions as of #14559 |
e5265d8
to
3375c37
Compare
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
3375c37
to
0e1a778
Compare
0e1a778
to
ad5265d
Compare
/merge |
@vyasr do you expect a performance impact from this simplification? |
No, this is only a test change |
Confirming the last comment, which I discussed with Greg offline. |
Description
This PR removes an extra code path used for checking the equality of the null count when verifying if columns are equivalent (not equal). The purpose of this code path was to verify a specific definition of equivalence for columns containing unsanitized nulls, i.e. by ignoring the stored null count and directly verifying the validity of the underlying null mask. This is no longer necessary because we required sanitized null masks to be output from all libcudf APIs now (see the "libcudf expects nested types to have sanitized null masks" section in the developer guide), and this requirement will be enforced with the merge of #14559.
Checklist