[BUG] Test failure hash_aggregate_test.py::test_hash_reduction_collect_set_on_nested_array_type DATAGEN_SEED=1705515231 #10209
Assignees
Labels
bug
Something isn't working
Comments
sameerz
added
bug
|
Duplicate of #10133? |
|
test_hash_reduction_collect_set_on_nested_array_type also failed by DATAGEN_SEED=1707498511. |
|
Cannot reproduce using these seed values anymore. Assuming that this is the same bug as #10133, I close this while #10133 will be closed by a fix in rapidsai/cudf#15243. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
The following tests failed in the nightly build
Detailed output
[2024-01-18T00:34:38.133Z] _ test_hash_reduction_collect_set_on_nested_array_type[[('a', RepeatSeq(Long)), ('b', RepeatSeq(Array(Array(Long))))]] _ [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] data_gen = [('a', RepeatSeq(Long)), ('b', RepeatSeq(Array(Array(Long))))] [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] @ignore_order(local=True, arrays=["collect_set"]) [2024-01-18T00:34:38.133Z] @allow_non_gpu("ProjectExec", *non_utc_allow) [2024-01-18T00:34:38.133Z] @pytest.mark.parametrize('data_gen', _gen_data_for_collect_set_op_nested, ids=idfn) [2024-01-18T00:34:38.133Z] def test_hash_reduction_collect_set_on_nested_array_type(data_gen): [2024-01-18T00:34:38.133Z] conf = copy_and_update(_float_conf, { [2024-01-18T00:34:38.133Z] "spark.rapids.sql.castFloatToString.enabled": "true", [2024-01-18T00:34:38.133Z] }) [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] def do_it(spark): [2024-01-18T00:34:38.133Z] return gen_df(spark, data_gen, length=100)\ [2024-01-18T00:34:38.133Z] .agg(f.collect_set('b').alias("collect_set")) [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] > assert_gpu_and_cpu_are_equal_collect(do_it, conf=conf) [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] ../../src/main/python/hash_aggregate_test.py:734: [2024-01-18T00:34:38.133Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [2024-01-18T00:34:38.133Z] ../../src/main/python/asserts.py:595: in assert_gpu_and_cpu_are_equal_collect [2024-01-18T00:34:38.133Z] _assert_gpu_and_cpu_are_equal(func, 'COLLECT', conf=conf, is_cpu_first=is_cpu_first, result_canonicalize_func_before_compare=result_canonicalize_func_before_compare) [2024-01-18T00:34:38.133Z] ../../src/main/python/asserts.py:503: in _assert_gpu_and_cpu_are_equal [2024-01-18T00:34:38.133Z] from_gpu = run_on_gpu() [2024-01-18T00:34:38.133Z] ../../src/main/python/asserts.py:496: in run_on_gpu [2024-01-18T00:34:38.133Z] from_gpu = with_gpu_session(bring_back, conf=conf) [2024-01-18T00:34:38.133Z] ../../src/main/python/spark_session.py:164: in with_gpu_session [2024-01-18T00:34:38.133Z] return with_spark_session(func, conf=copy) [2024-01-18T00:34:38.133Z] /opt/miniconda3/lib/python3.8/contextlib.py:75: in inner [2024-01-18T00:34:38.133Z] return func(*args, **kwds) [2024-01-18T00:34:38.133Z] ../../src/main/python/spark_session.py:131: in with_spark_session [2024-01-18T00:34:38.133Z] ret = func(_spark) [2024-01-18T00:34:38.133Z] ../../src/main/python/asserts.py:205: in [2024-01-18T00:34:38.133Z] bring_back = lambda spark: limit_func(spark).collect() [2024-01-18T00:34:38.133Z] /var/lib/jenkins/spark/spark-3.3.0-bin-hadoop3.2/python/lib/pyspark.zip/pyspark/sql/dataframe.py:817: in collect [2024-01-18T00:34:38.133Z] sock_info = self._jdf.collectToPython() [2024-01-18T00:34:38.133Z] /var/lib/jenkins/spark/spark-3.3.0-bin-hadoop3.2/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py:1321: in __call__ [2024-01-18T00:34:38.133Z] return_value = get_return_value( [2024-01-18T00:34:38.133Z] /var/lib/jenkins/spark/spark-3.3.0-bin-hadoop3.2/python/lib/pyspark.zip/pyspark/sql/utils.py:190: in deco [2024-01-18T00:34:38.133Z] return f(*a, **kw) [2024-01-18T00:34:38.133Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] answer = 'xro1810286' [2024-01-18T00:34:38.133Z] gateway_client = [2024-01-18T00:34:38.133Z] target_id = 'o1810285', name = 'collectToPython' [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] def get_return_value(answer, gateway_client, target_id=None, name=None): [2024-01-18T00:34:38.133Z] """Converts an answer received from the Java gateway into a Python object. [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] For example, string representation of integers are converted to Python [2024-01-18T00:34:38.133Z] integer, string representation of objects are converted to JavaObject [2024-01-18T00:34:38.133Z] instances, etc. [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] :param answer: the string returned by the Java gateway [2024-01-18T00:34:38.133Z] :param gateway_client: the gateway client used to communicate with the Java [2024-01-18T00:34:38.133Z] Gateway. Only necessary if the answer is a reference (e.g., object, [2024-01-18T00:34:38.133Z] list, map) [2024-01-18T00:34:38.133Z] :param target_id: the name of the object from which the answer comes from [2024-01-18T00:34:38.133Z] (e.g., *object1* in `object1.hello()`). Optional. [2024-01-18T00:34:38.133Z] :param name: the name of the member from which the answer comes from [2024-01-18T00:34:38.133Z] (e.g., *hello* in `object1.hello()`). Optional. [2024-01-18T00:34:38.133Z] """ [2024-01-18T00:34:38.133Z] if is_error(answer)[0]: [2024-01-18T00:34:38.133Z] if len(answer) > 1: [2024-01-18T00:34:38.133Z] type = answer[1] [2024-01-18T00:34:38.133Z] value = OUTPUT_CONVERTER[type](answer[2:], gateway_client) [2024-01-18T00:34:38.133Z] if answer[1] == REFERENCE_TYPE: [2024-01-18T00:34:38.133Z] > raise Py4JJavaError( [2024-01-18T00:34:38.133Z] "An error occurred while calling {0}{1}{2}.\n". [2024-01-18T00:34:38.133Z] format(target_id, ".", name), value) [2024-01-18T00:34:38.133Z] E py4j.protocol.Py4JJavaError: An error occurred while calling o1810285.collectToPython. [2024-01-18T00:34:38.133Z] E : org.apache.spark.SparkException: Job aborted due to stage failure: Task 7 in stage 25144.0 failed 1 times, most recent failure: Lost task 7.0 in stage 25144.0 (TID 788109) (10.136.6.4 executor 0): java.lang.IllegalArgumentException: ArrayType(LongType,true) is not supported for GPU processing yet. [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuColumnVector.getNonNestedRapidsType(GpuColumnVector.java:423) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:564) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:593) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:593) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuColumnVector.from(GpuColumnVector.java:709) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$3(GpuAggregateExec.scala:363) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$2(GpuAggregateExec.scala:361) [2024-01-18T00:34:38.133Z] E at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) [2024-01-18T00:34:38.133Z] E at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) [2024-01-18T00:34:38.133Z] E at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$1(GpuAggregateExec.scala:357) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.performReduction(GpuAggregateExec.scala:355) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.aggregate(GpuAggregateExec.scala:294) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$4(GpuAggregateExec.scala:311) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$3(GpuAggregateExec.scala:309) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$2(GpuAggregateExec.scala:308) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$AutoCloseableAttemptSpliterator.next(RmmRapidsRetryIterator.scala:477) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$RmmRapidsRetryIterator.next(RmmRapidsRetryIterator.scala:613) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$RmmRapidsRetryAutoCloseableIterator.next(RmmRapidsRetryIterator.scala:517) [2024-01-18T00:34:38.133Z] E at scala.collection.Iterator$$anon$11.next(Iterator.scala:496) [2024-01-18T00:34:38.133Z] E at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.aggregateInputBatches(GpuAggregateExec.scala:795) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.$anonfun$next$2(GpuAggregateExec.scala:752) [2024-01-18T00:34:38.133Z] E at scala.Option.getOrElse(Option.scala:189) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.next(GpuAggregateExec.scala:749) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.next(GpuAggregateExec.scala:711) [2024-01-18T00:34:38.133Z] E at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.$anonfun$next$6(GpuAggregateExec.scala:2042) [2024-01-18T00:34:38.133Z] E at scala.Option.map(Option.scala:230) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.next(GpuAggregateExec.scala:2042) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.next(GpuAggregateExec.scala:1906) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.execution.GpuShuffleExchangeExecBase$$anon$1.partNextBatch(GpuShuffleExchangeExecBase.scala:333) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.execution.GpuShuffleExchangeExecBase$$anon$1.hasNext(GpuShuffleExchangeExecBase.scala:355) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$2(RapidsShuffleInternalManagerBase.scala:285) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$2$adapted(RapidsShuffleInternalManagerBase.scala:278) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$1(RapidsShuffleInternalManagerBase.scala:278) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$1$adapted(RapidsShuffleInternalManagerBase.scala:277) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.write(RapidsShuffleInternalManagerBase.scala:277) [2024-01-18T00:34:38.133Z] E at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.Task.run(Task.scala:136) [2024-01-18T00:34:38.133Z] E at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) [2024-01-18T00:34:38.133Z] E at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) [2024-01-18T00:34:38.133Z] E at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) [2024-01-18T00:34:38.133Z] E at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [2024-01-18T00:34:38.133Z] E at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [2024-01-18T00:34:38.133Z] E at java.base/java.lang.Thread.run(Thread.java:833) [2024-01-18T00:34:38.133Z] E [2024-01-18T00:34:38.133Z] E Driver stacktrace: [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2672) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2608) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2607) [2024-01-18T00:34:38.133Z] E at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) [2024-01-18T00:34:38.133Z] E at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) [2024-01-18T00:34:38.133Z] E at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2607) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1182) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1182) [2024-01-18T00:34:38.133Z] E at scala.Option.foreach(Option.scala:407) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1182) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2860) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2802) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2791) [2024-01-18T00:34:38.133Z] E at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:952) [2024-01-18T00:34:38.133Z] E at org.apache.spark.SparkContext.runJob(SparkContext.scala:2228) [2024-01-18T00:34:38.133Z] E at org.apache.spark.SparkContext.runJob(SparkContext.scala:2249) [2024-01-18T00:34:38.133Z] E at org.apache.spark.SparkContext.runJob(SparkContext.scala:2268) [2024-01-18T00:34:38.133Z] E at org.apache.spark.SparkContext.runJob(SparkContext.scala:2293) [2024-01-18T00:34:38.133Z] E at org.apache.spark.rdd.RDD.$anonfun$collect$1(RDD.scala:1021) [2024-01-18T00:34:38.133Z] E at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) [2024-01-18T00:34:38.133Z] E at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) [2024-01-18T00:34:38.133Z] E at org.apache.spark.rdd.RDD.withScope(RDD.scala:406) [2024-01-18T00:34:38.133Z] E at org.apache.spark.rdd.RDD.collect(RDD.scala:1020) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:424) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.Dataset.$anonfun$collectToPython$1(Dataset.scala:3688) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.Dataset.$anonfun$withAction$2(Dataset.scala:3858) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3856) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3856) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.Dataset.collectToPython(Dataset.scala:3685) [2024-01-18T00:34:38.133Z] E at jdk.internal.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) [2024-01-18T00:34:38.133Z] E at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [2024-01-18T00:34:38.133Z] E at java.base/java.lang.reflect.Method.invoke(Method.java:568) [2024-01-18T00:34:38.133Z] E at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) [2024-01-18T00:34:38.133Z] E at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) [2024-01-18T00:34:38.133Z] E at py4j.Gateway.invoke(Gateway.java:282) [2024-01-18T00:34:38.133Z] E at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) [2024-01-18T00:34:38.133Z] E at py4j.commands.CallCommand.execute(CallCommand.java:79) [2024-01-18T00:34:38.133Z] E at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) [2024-01-18T00:34:38.133Z] E at py4j.ClientServerConnection.run(ClientServerConnection.java:106) [2024-01-18T00:34:38.133Z] E at java.base/java.lang.Thread.run(Thread.java:833) [2024-01-18T00:34:38.133Z] E Caused by: java.lang.IllegalArgumentException: ArrayType(LongType,true) is not supported for GPU processing yet. [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuColumnVector.getNonNestedRapidsType(GpuColumnVector.java:423) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:564) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:593) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:593) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuColumnVector.from(GpuColumnVector.java:709) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$3(GpuAggregateExec.scala:363) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$2(GpuAggregateExec.scala:361) [2024-01-18T00:34:38.133Z] E at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) [2024-01-18T00:34:38.133Z] E at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) [2024-01-18T00:34:38.133Z] E at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$1(GpuAggregateExec.scala:357) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.performReduction(GpuAggregateExec.scala:355) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.aggregate(GpuAggregateExec.scala:294) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$4(GpuAggregateExec.scala:311) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$3(GpuAggregateExec.scala:309) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$2(GpuAggregateExec.scala:308) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$AutoCloseableAttemptSpliterator.next(RmmRapidsRetryIterator.scala:477) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$RmmRapidsRetryIterator.next(RmmRapidsRetryIterator.scala:613) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$RmmRapidsRetryAutoCloseableIterator.next(RmmRapidsRetryIterator.scala:517) [2024-01-18T00:34:38.133Z] E at scala.collection.Iterator$$anon$11.next(Iterator.scala:496) [2024-01-18T00:34:38.133Z] E at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.aggregateInputBatches(GpuAggregateExec.scala:795) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.$anonfun$next$2(GpuAggregateExec.scala:752) [2024-01-18T00:34:38.133Z] E at scala.Option.getOrElse(Option.scala:189) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.next(GpuAggregateExec.scala:749) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.next(GpuAggregateExec.scala:711) [2024-01-18T00:34:38.133Z] E at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.$anonfun$next$6(GpuAggregateExec.scala:2042) [2024-01-18T00:34:38.133Z] E at scala.Option.map(Option.scala:230) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.next(GpuAggregateExec.scala:2042) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.next(GpuAggregateExec.scala:1906) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.execution.GpuShuffleExchangeExecBase$$anon$1.partNextBatch(GpuShuffleExchangeExecBase.scala:333) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.execution.GpuShuffleExchangeExecBase$$anon$1.hasNext(GpuShuffleExchangeExecBase.scala:355) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$2(RapidsShuffleInternalManagerBase.scala:285) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$2$adapted(RapidsShuffleInternalManagerBase.scala:278) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$1(RapidsShuffleInternalManagerBase.scala:278) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$1$adapted(RapidsShuffleInternalManagerBase.scala:277) [2024-01-18T00:34:38.133Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.133Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.write(RapidsShuffleInternalManagerBase.scala:277) [2024-01-18T00:34:38.133Z] E at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52) [2024-01-18T00:34:38.133Z] E at org.apache.spark.scheduler.Task.run(Task.scala:136) [2024-01-18T00:34:38.133Z] E at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) [2024-01-18T00:34:38.133Z] E at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) [2024-01-18T00:34:38.133Z] E at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) [2024-01-18T00:34:38.133Z] E at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [2024-01-18T00:34:38.133Z] E at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [2024-01-18T00:34:38.133Z] E ... 1 more [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] /var/lib/jenkins/spark/spark-3.3.0-bin-hadoop3.2/python/lib/py4j-0.10.9.5-src.zip/py4j/protocol.py:326: Py4JJavaError [2024-01-18T00:34:38.133Z] ----------------------------- Captured stdout call ----------------------------- [2024-01-18T00:34:38.133Z] ### CPU RUN ### [2024-01-18T00:34:38.133Z] ### GPU RUN ### [2024-01-18T00:34:38.133Z] _ test_hash_reduction_collect_set_on_nested_array_type[[('a', RepeatSeq(Long)), ('b', RepeatSeq(Array(Array(Boolean))))]] _ [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] data_gen = [('a', RepeatSeq(Long)), ('b', RepeatSeq(Array(Array(Boolean))))] [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] @ignore_order(local=True, arrays=["collect_set"]) [2024-01-18T00:34:38.133Z] @allow_non_gpu("ProjectExec", *non_utc_allow) [2024-01-18T00:34:38.133Z] @pytest.mark.parametrize('data_gen', _gen_data_for_collect_set_op_nested, ids=idfn) [2024-01-18T00:34:38.133Z] def test_hash_reduction_collect_set_on_nested_array_type(data_gen): [2024-01-18T00:34:38.133Z] conf = copy_and_update(_float_conf, { [2024-01-18T00:34:38.133Z] "spark.rapids.sql.castFloatToString.enabled": "true", [2024-01-18T00:34:38.133Z] }) [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] def do_it(spark): [2024-01-18T00:34:38.133Z] return gen_df(spark, data_gen, length=100)\ [2024-01-18T00:34:38.133Z] .agg(f.collect_set('b').alias("collect_set")) [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] > assert_gpu_and_cpu_are_equal_collect(do_it, conf=conf) [2024-01-18T00:34:38.133Z] [2024-01-18T00:34:38.133Z] ../../src/main/python/hash_aggregate_test.py:734: [2024-01-18T00:34:38.133Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [2024-01-18T00:34:38.133Z] ../../src/main/python/asserts.py:595: in assert_gpu_and_cpu_are_equal_collect [2024-01-18T00:34:38.133Z] _assert_gpu_and_cpu_are_equal(func, 'COLLECT', conf=conf, is_cpu_first=is_cpu_first, result_canonicalize_func_before_compare=result_canonicalize_func_before_compare) [2024-01-18T00:34:38.133Z] ../../src/main/python/asserts.py:503: in _assert_gpu_and_cpu_are_equal [2024-01-18T00:34:38.133Z] from_gpu = run_on_gpu() [2024-01-18T00:34:38.133Z] ../../src/main/python/asserts.py:496: in run_on_gpu [2024-01-18T00:34:38.133Z] from_gpu = with_gpu_session(bring_back, conf=conf) [2024-01-18T00:34:38.133Z] ../../src/main/python/spark_session.py:164: in with_gpu_session [2024-01-18T00:34:38.133Z] return with_spark_session(func, conf=copy) [2024-01-18T00:34:38.133Z] /opt/miniconda3/lib/python3.8/contextlib.py:75: in inner [2024-01-18T00:34:38.134Z] return func(*args, **kwds) [2024-01-18T00:34:38.134Z] ../../src/main/python/spark_session.py:131: in with_spark_session [2024-01-18T00:34:38.134Z] ret = func(_spark) [2024-01-18T00:34:38.134Z] ../../src/main/python/asserts.py:205: in [2024-01-18T00:34:38.134Z] bring_back = lambda spark: limit_func(spark).collect() [2024-01-18T00:34:38.134Z] /var/lib/jenkins/spark/spark-3.3.0-bin-hadoop3.2/python/lib/pyspark.zip/pyspark/sql/dataframe.py:817: in collect [2024-01-18T00:34:38.134Z] sock_info = self._jdf.collectToPython() [2024-01-18T00:34:38.134Z] /var/lib/jenkins/spark/spark-3.3.0-bin-hadoop3.2/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py:1321: in __call__ [2024-01-18T00:34:38.134Z] return_value = get_return_value( [2024-01-18T00:34:38.134Z] /var/lib/jenkins/spark/spark-3.3.0-bin-hadoop3.2/python/lib/pyspark.zip/pyspark/sql/utils.py:190: in deco [2024-01-18T00:34:38.134Z] return f(*a, **kw) [2024-01-18T00:34:38.134Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [2024-01-18T00:34:38.134Z] [2024-01-18T00:34:38.134Z] answer = 'xro1811206' [2024-01-18T00:34:38.134Z] gateway_client = [2024-01-18T00:34:38.134Z] target_id = 'o1811205', name = 'collectToPython' [2024-01-18T00:34:38.134Z] [2024-01-18T00:34:38.134Z] def get_return_value(answer, gateway_client, target_id=None, name=None): [2024-01-18T00:34:38.134Z] """Converts an answer received from the Java gateway into a Python object. [2024-01-18T00:34:38.134Z] [2024-01-18T00:34:38.134Z] For example, string representation of integers are converted to Python [2024-01-18T00:34:38.134Z] integer, string representation of objects are converted to JavaObject [2024-01-18T00:34:38.134Z] instances, etc. [2024-01-18T00:34:38.134Z] [2024-01-18T00:34:38.134Z] :param answer: the string returned by the Java gateway [2024-01-18T00:34:38.134Z] :param gateway_client: the gateway client used to communicate with the Java [2024-01-18T00:34:38.134Z] Gateway. Only necessary if the answer is a reference (e.g., object, [2024-01-18T00:34:38.134Z] list, map) [2024-01-18T00:34:38.134Z] :param target_id: the name of the object from which the answer comes from [2024-01-18T00:34:38.134Z] (e.g., *object1* in `object1.hello()`). Optional. [2024-01-18T00:34:38.134Z] :param name: the name of the member from which the answer comes from [2024-01-18T00:34:38.134Z] (e.g., *hello* in `object1.hello()`). Optional. [2024-01-18T00:34:38.134Z] """ [2024-01-18T00:34:38.134Z] if is_error(answer)[0]: [2024-01-18T00:34:38.134Z] if len(answer) > 1: [2024-01-18T00:34:38.134Z] type = answer[1] [2024-01-18T00:34:38.134Z] value = OUTPUT_CONVERTER[type](answer[2:], gateway_client) [2024-01-18T00:34:38.134Z] if answer[1] == REFERENCE_TYPE: [2024-01-18T00:34:38.134Z] > raise Py4JJavaError( [2024-01-18T00:34:38.134Z] "An error occurred while calling {0}{1}{2}.\n". [2024-01-18T00:34:38.134Z] format(target_id, ".", name), value) [2024-01-18T00:34:38.134Z] E py4j.protocol.Py4JJavaError: An error occurred while calling o1811205.collectToPython. [2024-01-18T00:34:38.134Z] E : org.apache.spark.SparkException: Job aborted due to stage failure: Task 13 in stage 25160.0 failed 1 times, most recent failure: Lost task 13.0 in stage 25160.0 (TID 788506) (10.136.6.4 executor 3): java.lang.IllegalArgumentException: ArrayType(BooleanType,true) is not supported for GPU processing yet. [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuColumnVector.getNonNestedRapidsType(GpuColumnVector.java:423) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:564) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:593) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:593) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuColumnVector.from(GpuColumnVector.java:709) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$3(GpuAggregateExec.scala:363) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$2(GpuAggregateExec.scala:361) [2024-01-18T00:34:38.134Z] E at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) [2024-01-18T00:34:38.134Z] E at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) [2024-01-18T00:34:38.134Z] E at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$1(GpuAggregateExec.scala:357) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.performReduction(GpuAggregateExec.scala:355) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.aggregate(GpuAggregateExec.scala:294) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$4(GpuAggregateExec.scala:311) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$3(GpuAggregateExec.scala:309) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$2(GpuAggregateExec.scala:308) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$AutoCloseableAttemptSpliterator.next(RmmRapidsRetryIterator.scala:477) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$RmmRapidsRetryIterator.next(RmmRapidsRetryIterator.scala:613) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$RmmRapidsRetryAutoCloseableIterator.next(RmmRapidsRetryIterator.scala:517) [2024-01-18T00:34:38.134Z] E at scala.collection.Iterator$$anon$11.next(Iterator.scala:496) [2024-01-18T00:34:38.134Z] E at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.aggregateInputBatches(GpuAggregateExec.scala:795) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.$anonfun$next$2(GpuAggregateExec.scala:752) [2024-01-18T00:34:38.134Z] E at scala.Option.getOrElse(Option.scala:189) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.next(GpuAggregateExec.scala:749) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.next(GpuAggregateExec.scala:711) [2024-01-18T00:34:38.134Z] E at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.$anonfun$next$6(GpuAggregateExec.scala:2042) [2024-01-18T00:34:38.134Z] E at scala.Option.map(Option.scala:230) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.next(GpuAggregateExec.scala:2042) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.next(GpuAggregateExec.scala:1906) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.execution.GpuShuffleExchangeExecBase$$anon$1.partNextBatch(GpuShuffleExchangeExecBase.scala:333) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.execution.GpuShuffleExchangeExecBase$$anon$1.hasNext(GpuShuffleExchangeExecBase.scala:355) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$2(RapidsShuffleInternalManagerBase.scala:285) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$2$adapted(RapidsShuffleInternalManagerBase.scala:278) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$1(RapidsShuffleInternalManagerBase.scala:278) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$1$adapted(RapidsShuffleInternalManagerBase.scala:277) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.write(RapidsShuffleInternalManagerBase.scala:277) [2024-01-18T00:34:38.134Z] E at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.Task.run(Task.scala:136) [2024-01-18T00:34:38.134Z] E at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) [2024-01-18T00:34:38.134Z] E at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) [2024-01-18T00:34:38.134Z] E at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) [2024-01-18T00:34:38.134Z] E at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [2024-01-18T00:34:38.134Z] E at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [2024-01-18T00:34:38.134Z] E at java.base/java.lang.Thread.run(Thread.java:833) [2024-01-18T00:34:38.134Z] E [2024-01-18T00:34:38.134Z] E Driver stacktrace: [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2672) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2608) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2607) [2024-01-18T00:34:38.134Z] E at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) [2024-01-18T00:34:38.134Z] E at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) [2024-01-18T00:34:38.134Z] E at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2607) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1182) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1182) [2024-01-18T00:34:38.134Z] E at scala.Option.foreach(Option.scala:407) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1182) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2860) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2802) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2791) [2024-01-18T00:34:38.134Z] E at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:952) [2024-01-18T00:34:38.134Z] E at org.apache.spark.SparkContext.runJob(SparkContext.scala:2228) [2024-01-18T00:34:38.134Z] E at org.apache.spark.SparkContext.runJob(SparkContext.scala:2249) [2024-01-18T00:34:38.134Z] E at org.apache.spark.SparkContext.runJob(SparkContext.scala:2268) [2024-01-18T00:34:38.134Z] E at org.apache.spark.SparkContext.runJob(SparkContext.scala:2293) [2024-01-18T00:34:38.134Z] E at org.apache.spark.rdd.RDD.$anonfun$collect$1(RDD.scala:1021) [2024-01-18T00:34:38.134Z] E at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) [2024-01-18T00:34:38.134Z] E at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) [2024-01-18T00:34:38.134Z] E at org.apache.spark.rdd.RDD.withScope(RDD.scala:406) [2024-01-18T00:34:38.134Z] E at org.apache.spark.rdd.RDD.collect(RDD.scala:1020) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:424) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.Dataset.$anonfun$collectToPython$1(Dataset.scala:3688) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.Dataset.$anonfun$withAction$2(Dataset.scala:3858) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3856) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3856) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.Dataset.collectToPython(Dataset.scala:3685) [2024-01-18T00:34:38.134Z] E at jdk.internal.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) [2024-01-18T00:34:38.134Z] E at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [2024-01-18T00:34:38.134Z] E at java.base/java.lang.reflect.Method.invoke(Method.java:568) [2024-01-18T00:34:38.134Z] E at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) [2024-01-18T00:34:38.134Z] E at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) [2024-01-18T00:34:38.134Z] E at py4j.Gateway.invoke(Gateway.java:282) [2024-01-18T00:34:38.134Z] E at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) [2024-01-18T00:34:38.134Z] E at py4j.commands.CallCommand.execute(CallCommand.java:79) [2024-01-18T00:34:38.134Z] E at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) [2024-01-18T00:34:38.134Z] E at py4j.ClientServerConnection.run(ClientServerConnection.java:106) [2024-01-18T00:34:38.134Z] E at java.base/java.lang.Thread.run(Thread.java:833) [2024-01-18T00:34:38.134Z] E Caused by: java.lang.IllegalArgumentException: ArrayType(BooleanType,true) is not supported for GPU processing yet. [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuColumnVector.getNonNestedRapidsType(GpuColumnVector.java:423) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:564) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:593) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuColumnVector.typeConversionAllowed(GpuColumnVector.java:593) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuColumnVector.from(GpuColumnVector.java:709) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$3(GpuAggregateExec.scala:363) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$2(GpuAggregateExec.scala:361) [2024-01-18T00:34:38.134Z] E at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) [2024-01-18T00:34:38.134Z] E at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) [2024-01-18T00:34:38.134Z] E at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$performReduction$1(GpuAggregateExec.scala:357) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.performReduction(GpuAggregateExec.scala:355) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.aggregate(GpuAggregateExec.scala:294) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$4(GpuAggregateExec.scala:311) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$3(GpuAggregateExec.scala:309) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.AggHelper.$anonfun$aggregateWithoutCombine$2(GpuAggregateExec.scala:308) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$AutoCloseableAttemptSpliterator.next(RmmRapidsRetryIterator.scala:477) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$RmmRapidsRetryIterator.next(RmmRapidsRetryIterator.scala:613) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.RmmRapidsRetryIterator$RmmRapidsRetryAutoCloseableIterator.next(RmmRapidsRetryIterator.scala:517) [2024-01-18T00:34:38.134Z] E at scala.collection.Iterator$$anon$11.next(Iterator.scala:496) [2024-01-18T00:34:38.134Z] E at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.aggregateInputBatches(GpuAggregateExec.scala:795) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.$anonfun$next$2(GpuAggregateExec.scala:752) [2024-01-18T00:34:38.134Z] E at scala.Option.getOrElse(Option.scala:189) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.next(GpuAggregateExec.scala:749) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.GpuMergeAggregateIterator.next(GpuAggregateExec.scala:711) [2024-01-18T00:34:38.134Z] E at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.$anonfun$next$6(GpuAggregateExec.scala:2042) [2024-01-18T00:34:38.134Z] E at scala.Option.map(Option.scala:230) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.next(GpuAggregateExec.scala:2042) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.DynamicGpuPartialSortAggregateIterator.next(GpuAggregateExec.scala:1906) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.execution.GpuShuffleExchangeExecBase$$anon$1.partNextBatch(GpuShuffleExchangeExecBase.scala:333) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.execution.GpuShuffleExchangeExecBase$$anon$1.hasNext(GpuShuffleExchangeExecBase.scala:355) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$2(RapidsShuffleInternalManagerBase.scala:285) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$2$adapted(RapidsShuffleInternalManagerBase.scala:278) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$1(RapidsShuffleInternalManagerBase.scala:278) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.$anonfun$write$1$adapted(RapidsShuffleInternalManagerBase.scala:277) [2024-01-18T00:34:38.134Z] E at com.nvidia.spark.rapids.Arm$.withResource(Arm.scala:29) [2024-01-18T00:34:38.134Z] E at org.apache.spark.sql.rapids.RapidsShuffleThreadedWriterBase.write(RapidsShuffleInternalManagerBase.scala:277) [2024-01-18T00:34:38.134Z] E at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52) [2024-01-18T00:34:38.134Z] E at org.apache.spark.scheduler.Task.run(Task.scala:136) [2024-01-18T00:34:38.134Z] E at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) [2024-01-18T00:34:38.134Z] E at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) [2024-01-18T00:34:38.134Z] E at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) [2024-01-18T00:34:38.134Z] E at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [2024-01-18T00:34:38.134Z] E at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [2024-01-18T00:34:38.134Z] E ... 1 more [2024-01-18T00:34:38.134Z] [2024-01-18T00:34:38.134Z] /var/lib/jenkins/spark/spark-3.3.0-bin-hadoop3.2/python/lib/py4j-0.10.9.5-src.zip/py4j/protocol.py:326: Py4JJavaErrorSteps/Code to reproduce bug
Nightly build, DATAGEN_SEED=1705515231
Expected behavior
Tests pass
Environment details (please complete the following information)
Additional context
The text was updated successfully, but these errors were encountered: