Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Add null_count_if_available or lazy_null_count to column and column_view #12256

Closed
ttnghia opened this issue Nov 29, 2022 · 2 comments
Closed
Labels
feature request New feature or request improvement Improvement / enhancement to an existing function Performance Performance related issue

Comments

@ttnghia
Copy link
Contributor

ttnghia commented Nov 29, 2022

The classes column and column_view have null_count functions that return the number of nulls in the given column. The column holds an internal null count value which is defaulted to UNKNOW_NULL_COUNT.

During various transformations, we need to copy that null count value around but may not really care about what that value is. However, if the function null_count() is called but the column's internal number of nulls has not been computed, a kernel will always be launched to compute the actual null count. That is a waste of time because we can just return UNKNOW_NULL_COUNT without compute anything.

I suggest adding a new function that allows returning the column's internal null count directly without computation. That function can be named null_count_if_availabe or lazy_null_count or anything else that is relevant.

@ttnghia ttnghia added feature request New feature or request Needs Triage Need team to review and classify Performance Performance related issue improvement Improvement / enhancement to an existing function labels Nov 29, 2022
@davidwendt
Copy link
Contributor

The UNKNOWN_NULL_COUNT feature will be removed with #11968
So we will not be needing these extra functions.

@GregoryKimball GregoryKimball changed the title [FEA] Add null_count_if_availabe or lazy_null_count to column and column_view [FEA] Add null_count_if_available or lazy_null_count to column and column_view Dec 22, 2022
@GregoryKimball
Copy link
Contributor

I'll close this in favor of #11968 for now

@bdice bdice removed the Needs Triage Need team to review and classify label Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request improvement Improvement / enhancement to an existing function Performance Performance related issue
Projects
None yet
Development

No branches or pull requests

4 participants