You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a csv dataset in which nulls are encoded differently in different columns. It looks like when reading csv data with pyarrow, the same list of null_values must be used for all columns (see ConvertOptions). This concerns me because a value used as a null code in one column ("9999" for example) may be a valid non-null value in another column. In pandas's read_csv, it is possible to pass a dictionary specifying different null codes for different columns. Could this functionality be added to pyarrow?
Component(s)
Other
The text was updated successfully, but these errors were encountered:
Describe the enhancement requested
I have a csv dataset in which nulls are encoded differently in different columns. It looks like when reading csv data with pyarrow, the same list of null_values must be used for all columns (see ConvertOptions). This concerns me because a value used as a null code in one column ("9999" for example) may be a valid non-null value in another column. In pandas's read_csv, it is possible to pass a dictionary specifying different null codes for different columns. Could this functionality be added to pyarrow?
Component(s)
Other
The text was updated successfully, but these errors were encountered: