You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently pyarrow.parquet.core.filters_to_expression handles equality/inequality operators and in and not in operators.
I propose adding is_nan, is_null, is_valid operators where value passed would be ignored but they would
return field.is_nan() field.is_null() field.is_valid() expressions.
…pache#38774)
Adding is_nan, is_null and is_valid operators to DNF filter expression.
JacekPliszka
changed the title
Add is_nan, is_null, is_valid as operators to DNF filters
[Python] Add is_nan, is_null, is_valid as operators to DNF filters
Nov 18, 2023
Describe the enhancement requested
Currently pyarrow.parquet.core.filters_to_expression handles equality/inequality operators and in and not in operators.
I propose adding is_nan, is_null, is_valid operators where value passed would be ignored but they would
return field.is_nan() field.is_null() field.is_valid() expressions.
This is a very easy change but it would allow null/nan filtering in DNF form.
These functions are already implemented for pyarrow.dataset.Expression https://arrow.apache.org/docs/python/generated/pyarrow.dataset.Expression.html
Component(s)
Python
The text was updated successfully, but these errors were encountered: