Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Failing Test test_hash_groupby_approx_percentile_partial_fallback_to_cpu for Databricks 13.3 #9490

Closed
razajafri opened this issue Oct 19, 2023 · 0 comments · Fixed by #9690
Labels
task Work required that improves the product but is not user facing

Comments

@razajafri
Copy link
Collaborator

razajafri commented Oct 19, 2023

Unable to find ApproximatePercentile in the GPU plan

E                   py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.sql.rapids.ExecutionPlanCaptureCallback.assertDidFallBack.
E                   : java.lang.AssertionError: assertion failed: Could not find ApproximatePercentile in the GPU plans:
E                   ResultQueryStage 1, Statistics(sizeInBytes=3.5 KiB, rowCount=100, ColumnStat: N/A, isRuntime=true)
E                   +- ObjectHashAggregate(keys=[k#138], functions=[finalmerge_approx_percentile(merge buf#147) AS approx_percentile(v#139, [0.1,0.2], 10000, 0, 0)#142], output=[k#138, approx_percentile(v, array(0.1, 0.2), 10000)#143])
E                      +- AQEShuffleRead coalesced
E                         +- ShuffleQueryStage 0, Statistics(sizeInBytes=11.6 KiB, rowCount=100, ColumnStat: N/A, isRuntime=true)
E                            +- Exchange hashpartitioning(k#138, 4), ENSURE_REQUIREMENTS, [plan_id=281]
E                               +- ObjectHashAggregate(keys=[k#138], functions=[partial_approx_percentile(v#139, [0.1,0.2], 10000, 0, 0) AS buf#147], output=[k#138, buf#147])
E                                  +- *(1) Scan ExistingRDD[k#138,v#139]
E                   
E                       at scala.Predef$.assert(Predef.scala:223)
E                       at org.apache.spark.sql.rapids.ShimmedExecutionPlanCaptureCallbackImpl.assertDidFallBack(ShimmedExecutionPlanCaptureCallbackImpl.scala:144)
E                       at org.apache.spark.sql.rapids.ShimmedExecutionPlanCaptureCallbackImpl.assertDidFallBack(ShimmedExecutionPlanCaptureCallbackImpl.scala:156)
E                       at org.apache.spark.sql.rapids.ExecutionPlanCaptureCallback$.assertDidFallBack(ExecutionPlanCaptureCallback.scala:85)
E                       at org.apache.spark.sql.rapids.ExecutionPlanCaptureCallback.assertDidFallBack(ExecutionPlanCaptureCallback.scala)
E                       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
E                       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
E                       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
E                       at java.lang.reflect.Method.invoke(Method.java:498)
E                       at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
E                       at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:397)
E                       at py4j.Gateway.invoke(Gateway.java:306)
E                       at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
E                       at py4j.commands.CallCommand.execute(CallCommand.java:79)
E                       at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:195)
E                       at py4j.ClientServerConnection.run(ClientServerConnection.java:115)
E                       at java.lang.Thread.run(Thread.java:750)

@razajafri razajafri changed the title Fix Integation Test Failures related to hash_aggregate_test.py Fix Integation Test Failures related to hash_aggregate_test.py for Databricks 13.3 Oct 19, 2023
@razajafri razajafri changed the title Fix Integation Test Failures related to hash_aggregate_test.py for Databricks 13.3 Fix Failing Test test_hash_groupby_approx_percentile_partial_fallback_to_cpu for Databricks 13.3 Oct 19, 2023
@sameerz sameerz added the task Work required that improves the product but is not user facing label Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
task Work required that improves the product but is not user facing
Projects
None yet
2 participants