You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
To be able to support sorting and comparisons on nested types we needed to get rid of all non-empty nulls. To help with this we added in some asserts when we see them. It looks like the GpuMd5 implementation is using mergeAndSetValidity which is producing an exception in this case.
Steps/Code to reproduce bug
Run the unit tests, but revert #8183 first
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 845.0 failed 1 times, most recent failure: Lost task 0.0 in stage 845.0 (TID 2577) (10.28.9.123 executor driver): java.lang.AssertionError: Column has non-empty nulls
at ai.rapids.cudf.AssertEmptyNulls.assertNullsAreEmpty(AssertEmptyNulls.java:33)
at ai.rapids.cudf.ColumnView.<init>(ColumnView.java:71)
at ai.rapids.cudf.ColumnVector.<init>(ColumnVector.java:58)
at ai.rapids.cudf.ColumnView.mergeAndSetValidity(ColumnView.java:835)
at org.apache.spark.sql.rapids.GpuMd5.$anonfun$doColumnar$2(HashFunctions.scala:38)
at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29)
at org.apache.spark.sql.rapids.GpuMd5.$anonfun$doColumnar$1(HashFunctions.scala:37)
at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29)
at org.apache.spark.sql.rapids.GpuMd5.doColumnar(HashFunctions.scala:36)
Expected behavior
This should not happen, we either need to fix mergeAndSetValidity, which I thought we had done, but it might have been bitwiseMergeAndSetValidity. Not sure if we have two or not. Or we need to stop using it + #7698
The text was updated successfully, but these errors were encountered:
Describe the bug
To be able to support sorting and comparisons on nested types we needed to get rid of all non-empty nulls. To help with this we added in some asserts when we see them. It looks like the GpuMd5 implementation is using
mergeAndSetValidity
which is producing an exception in this case.Steps/Code to reproduce bug
Run the unit tests, but revert #8183 first
Expected behavior
This should not happen, we either need to fix
mergeAndSetValidity
, which I thought we had done, but it might have been bitwiseMergeAndSetValidity. Not sure if we have two or not. Or we need to stop using it + #7698The text was updated successfully, but these errors were encountered: