You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Our current Databricks documentation shows how to launch a Databricks cluster (single and multi-node) with the RAPIDS environment. However it doesn't actually discuss how to leverage the multi-node cluster with Dask.
We need to document this.
I expect this will involve using dask-yarn to start the scheduler and worker processes and will need to use dask-cuda-worker as their startup script. Databricks doesn't use Yarn, we would need to explore whatever it does use.
Related issue #231 covers documenting the single-node setup.
The text was updated successfully, but these errors were encountered:
Our current Databricks documentation shows how to launch a Databricks cluster (single and multi-node) with the RAPIDS environment. However it doesn't actually discuss how to leverage the multi-node cluster with Dask.
We need to document this.
I expect this will involve using dask-yarn to start the scheduler and worker processes and will need to useDatabricks doesn't use Yarn, we would need to explore whatever it does use.dask-cuda-worker
as their startup script.Related issue #231 covers documenting the single-node setup.
The text was updated successfully, but these errors were encountered: