You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Running with the RapidsShuffleManager while in local mode causes the following exceptions to be printed to the console every few seconds:
21/06/16 18:12:28 ERROR Inbox: Ignoring error
java.lang.IllegalStateException: Heartbeat from unknown executor BlockManagerId(driver, 10.28.9.126, 32931, None)
at com.nvidia.spark.rapids.RapidsShuffleHeartbeatManager.executorHeartbeat(RapidsShuffleHeartbeatManager.scala:84)
at com.nvidia.spark.rapids.RapidsDriverPlugin.receive(Plugin.scala:149)
at org.apache.spark.internal.plugin.PluginEndpoint$$anonfun$receiveAndReply$1.applyOrElse(PluginEndpoint.scala:57)
at org.apache.spark.rpc.netty.Inbox.$anonfun$process$1(Inbox.scala:103)
at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:203)
at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)
at org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:75)
at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:41)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
21/06/16 18:12:28 ERROR RapidsShuffleHeartbeatEndpoint: Error during heartbeat
java.lang.IllegalStateException: Heartbeat from unknown executor BlockManagerId(driver, 10.28.9.126, 32931, None)
at com.nvidia.spark.rapids.RapidsShuffleHeartbeatManager.executorHeartbeat(RapidsShuffleHeartbeatManager.scala:84)
at com.nvidia.spark.rapids.RapidsDriverPlugin.receive(Plugin.scala:149)
at org.apache.spark.internal.plugin.PluginEndpoint$$anonfun$receiveAndReply$1.applyOrElse(PluginEndpoint.scala:57)
at org.apache.spark.rpc.netty.Inbox.$anonfun$process$1(Inbox.scala:103)
at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:203)
at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)
at org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:75)
at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:41)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Steps/Code to reproduce bug
Run spark-shell in local mode with UCX (e.g.: --master local[*] --conf spark.shuffle.manager=com.nvidia.spark.rapids.spark301.RapidsShuffleManager
Expected behavior
No exception errors on the console
Environment details (please complete the following information)
Spark 3.0.1
RAPIDS Accelerator 21.08.0-SNAPSHOT
The text was updated successfully, but these errors were encountered:
Describe the bug
Running with the RapidsShuffleManager while in local mode causes the following exceptions to be printed to the console every few seconds:
Steps/Code to reproduce bug
Run spark-shell in local mode with UCX (e.g.:
--master local[*] --conf spark.shuffle.manager=com.nvidia.spark.rapids.spark301.RapidsShuffleManager
Expected behavior
No exception errors on the console
Environment details (please complete the following information)
Spark 3.0.1
RAPIDS Accelerator 21.08.0-SNAPSHOT
The text was updated successfully, but these errors were encountered: