[BUG] Spark3.1 tpch failures #586
Assignees
Labels
bug
Something isn't working
P0
Must have for release
Spark 3.1+
Bugs only related to Spark 3.1 or higher
Milestone
Comments
tgravescs
added
bug
|
Note you have to run this with a non-local mode cluster so that it actually does a broadcast. |
|
Note that I have xfailed the tests for now under #588 |
|
I was unable to repro this failure on 3.1 with latest plugin. Will update here after further triaging. |
|
This appears to be caused by a bug in Spark 3.1.0 that we didn't have time to track down. The latest 3.1.0-SNAPSHOT no longer fails. |
tgravescs
pushed a commit
to tgravescs/spark-rapids
that referenced
this issue
Nov 30, 2023
…IDIA#586) Signed-off-by: spark-rapids automation <[email protected]> Signed-off-by: spark-rapids automation <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Spark 3.1 integration test failures:
08:43:58 FAILED src/main/python/tpch_test.py::test_tpch_q16[conf0][ALLOW_NON_GPU(BroadcastNestedLoopJoinExec,Or,IsNull,EqualTo,AttributeReference,BroadcastExchangeExec)]
08:43:58 FAILED src/main/python/tpch_test.py::test_tpch_q16[conf1][ALLOW_NON_GPU(BroadcastNestedLoopJoinExec,Or,IsNull,EqualTo,AttributeReference,BroadcastExchangeExec)]
08:43:58 FAILED src/main/python/tpch_test.py::test_tpch_q17[conf0][APPROXIMATE_FLOAT]
08:43:58 FAILED src/main/python/tpch_test.py::test_tpch_q17[conf1][APPROXIMATE_FLOAT]
08:43:58 FAILED src/main/python/tpch_test.py::test_tpch_q18[conf0][INCOMPAT, APPROXIMATE_FLOAT, ALLOW_NON_GPU(TakeOrderedAndProjectExec,SortOrder,AttributeReference)]
08:43:58 FAILED src/main/python/tpch_test.py::test_tpch_q18[conf1][INCOMPAT, APPROXIMATE_FLOAT, ALLOW_NON_GPU(TakeOrderedAndProjectExec,SortOrder,AttributeReference)]
08:43:58 FAILED src/main/python/tpch_test.py::test_tpch_q20[conf0] - py4j.protocol.Py4...
08:43:58 FAILED src/main/python/tpch_test.py::test_tpch_q20[conf1] - py4j.protocol.Py4...
The issue is:
Caused by: java.io.InvalidClassException: org.apache.spark.sql.execution.joins.EmptyHashedRelation$; no valid constructor
08:43:58 E at java.io.ObjectStreamClass$ExceptionInfo.newInvalidClassException(ObjectStreamClass.java:169)
changed: apache/spark@c37357a
The text was updated successfully, but these errors were encountered: