You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Cannot convert from arrow table to cudf dataframe when the string in the column is long.
The total number of characters in a cudf column is limited.
RuntimeError: cuDF failure at: /workspace/.conda-bld/work/cpp/src/copying/concatenate.cu:391: Total number of concatenated chars exceeds size_type range
Describe the solution you'd like
Resolve the limit on the total number of characters in a column.
Similar to the concept of batch in the arrow, a dataframe consists of multiple subframes and can be used for groupby calculation.
The text was updated successfully, but these errors were encountered:
Thanks for filing an issue. For now, we recommend using Dask or Spark to solve this limitation by splitting the data into partitions when processing larger dataframes.
#3958 (comment) has more context on the current status. We'll make sure to take your request and proposed solution into consideration in future discussions of this string character maximum.
Is your feature request related to a problem? Please describe.
Cannot convert from arrow table to cudf dataframe when the string in the column is long.
The total number of characters in a cudf column is limited.
#8748 (comment)
Describe the solution you'd like
The text was updated successfully, but these errors were encountered: