You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Feb 3, 2021. It is now read-only.
"In standalone mode, simply start your workers with spark.shuffle.service.enabled set to true."
It seems the external shuffle service is built in to standalone mode. So only the spark.shuffle.service.enabled spark-defaults.conf is required. The script does not need to be run and seems to occasionally cause worker registration to fail as the workers and the external shuffle service both try to listen to port 7337.
The text was updated successfully, but these errors were encountered:
https://spark.apache.org/docs/latest/job-scheduling.html#configuration-and-setup
"In standalone mode, simply start your workers with spark.shuffle.service.enabled set to true."
It seems the external shuffle service is built in to standalone mode. So only the
spark.shuffle.service.enabled
spark-defaults.conf is required. The script does not need to be run and seems to occasionally cause worker registration to fail as the workers and the external shuffle service both try to listen to port 7337.The text was updated successfully, but these errors were encountered: