Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix broadcast root during the replication call #3655

Merged
merged 7 commits into from
Jun 13, 2023
Merged

Fix broadcast root during the replication call #3655

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
4a43a82
fix the root of the broadcast
Jun 10, 2023
7e3b6f8
remove outdated comments
Jun 10, 2023
3323da9
fix style
Jun 10, 2023
1bd2d82
Merge remote-tracking branch 'upstream/branch-23.06' into branch-23.0…
Jun 10, 2023
929e491
fix typo
Jun 10, 2023
9428f27
get consistent gpu IDs from raft
Jun 12, 2023
67d94ae
remove unused import and functions
Jun 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion python/cugraph/cugraph/dask/common/input_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,20 @@ def create(cls, data, client=None, batch_enabled=False):
else:
raise TypeError("Graph data must be dask-cudf dataframe")

broadcast_worker = None
if batch_enabled:
worker_ranks = client.run(Comms.get_worker_id, Comms.get_session_id())
# The worker with 'rank = 0' must be the root of the broadcast.
broadcast_worker = list(worker_ranks.keys())[
list(worker_ranks.values()).index(0)
]

gpu_futures = client.sync(
_extract_partitions, data, client, batch_enabled=batch_enabled
_extract_partitions,
data,
client,
batch_enabled=batch_enabled,
broadcast_worker=broadcast_worker,
)
workers = tuple(OrderedDict.fromkeys(map(lambda x: x[0], gpu_futures)))
return DistributedDataHandler(
Expand Down
8 changes: 4 additions & 4 deletions python/cugraph/cugraph/dask/common/part_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,16 +114,16 @@ def persist_dask_df_equal_parts_per_worker(dask_df, client):
return dask_df


async def _extract_partitions(dask_obj, client=None, batch_enabled=False):
async def _extract_partitions(
dask_obj, client=None, batch_enabled=False, broadcast_worker=None
):
client = default_client() if client is None else client
worker_list = Comms.get_workers()

# dask.dataframe or dask.array
if isinstance(dask_obj, (daskDataFrame, daskArray, daskSeries)):
# parts = persist_distributed_data(dask_obj, client)
# FIXME: persist data to the same worker when batch_enabled=True
if batch_enabled:
persisted = client.persist(dask_obj, workers=worker_list[0])
persisted = client.persist(dask_obj, workers=broadcast_worker)
else:
# repartition the 'dask_obj' to get as many partitions as there
# are workers
Expand Down