You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When working on enbling Parquet Unit tests on ParquetFieldIdIOSuite(#11366), it shows that all 5 tests have passed, but in the ternimal you can see that, the tests of "read parquet file ids" and "global read/write flag should work correctly" throws java.lang.RuntimeExceptions, without stopping the UT.
24/09/06 10:39:29.802 Executor task launch worker for task 1.0 in stage 42.0 (TID 79) ERROR Executor: Exception in task 1.0 in stage 42.0 (TID 79)
java.lang.RuntimeException: Found duplicate field(s) "1": [a, rand2] in case-insensitive mode
at com.nvidia.spark.rapids.ParquetSchemaUtils$.$anonfun$clipParquetGroupFields$8(ParquetSchemaUtils.scala:310) ~[rapids-4-spark-aggregator_2.12-24.10.0-SNAPSHOT-spark330.jar:?]
at scala.Option.map(Option.scala:230) ~[scala-library-2.12.15.jar:?]
at com.nvidia.spark.rapids.ParquetSchemaUtils$.matchIdField$1(ParquetSchemaUtils.scala:302) ~[rapids-4-spark-aggregator_2.12-24.10.0-SNAPSHOT-spark330.jar:?]
at com.nvidia.spark.rapids.ParquetSchemaUtils$.$anonfun$clipParquetGroupFields$10(ParquetSchemaUtils.scala:320) ~[rapids-4-spark-aggregator_2.12-24.10.0-SNAPSHOT-spark330.jar:?]
at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293) ~[scala-library-2.12.15.jar:?]
at scala.collection.Iterator.foreach(Iterator.scala:943) ~[scala-library-2.12.15.jar:?]
at scala.collection.Iterator.foreach$(Iterator.scala:943) ~[scala-library-2.12.15.jar:?]
Steps/Code to reproduce bug
Checkout PR (#11366),
comment RapidsTestSettings.scala in tests/src/test/spark330
.exclude("read parquet file without ids", ADJUST_UT("Showing that the test has passed, but there is RuntimeException thrown"))
.exclude("global read/write flag should work correctly", ADJUST_UT("Showing that the test has passed, but there is RuntimeException thrown"))
compile and run the test with mvn -nsu -Dbuildver=330 -pl tests -Dsuites="org.apache.spark.sql.rapids.suites.RapidsParquetFieldIdSchemaSuite" test
Expected behavior
If Exception thrown, UT should fail.
The text was updated successfully, but these errors were encountered:
Feng-Jiang28
changed the title
[BUG] SPARK UT Framework showing test passed, but Exception printed without failing UT.
[BUG] SPARK UT Framework showing tests passed, but Exception printed without failing UT.
Sep 6, 2024
Describe the bug
When working on enbling Parquet Unit tests on ParquetFieldIdIOSuite(#11366), it shows that all 5 tests have passed, but in the ternimal you can see that, the tests of "read parquet file ids" and "global read/write flag should work correctly" throws java.lang.RuntimeExceptions, without stopping the UT.
Steps/Code to reproduce bug
Checkout PR (#11366),
comment RapidsTestSettings.scala in tests/src/test/spark330
compile and run the test with
mvn -nsu -Dbuildver=330 -pl tests -Dsuites="org.apache.spark.sql.rapids.suites.RapidsParquetFieldIdSchemaSuite" test
Expected behavior
If Exception thrown, UT should fail.
The text was updated successfully, but these errors were encountered: