[FEA] Add in std::overflow handling in CUDF and plugin #8451
Labels
feature request
New feature or request
reliability
Features to improve reliability or bugs that severly impact the reliability of the plugin
Is your feature request related to a problem? Please describe.
rapidsai/cudf#13323 went into 23.06, which added in support for throwing a std::overflow_error when a column would violate the limits on a CUDF column.
We should add in translations in the CUDF JNI layer so that when we see a std::overflow_error we throw a java exception that corresponds to it.
Then in the plugin when we see this type of an exception we can treat this as something very similar to a split and retry exception. We need to be a bit careful because they are not 100% the same. Some "split" operations may just change config settings instead of changing the actual input data. Also most split operations with a split is no longer possible will throw a SplitAndRetryOOM exception to indicate this. We don't want to do that if it was not actually caused by a SplitAndRetryOOM.
The main places to think about this are probably with a project operation or with aggregations (collect_set/collect_list) like operations. The later is more difficult to make it work though.
The text was updated successfully, but these errors were encountered: