Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lazy "cudf" registration for p2p-related dispatch functions #11040

Merged
merged 1 commit into from
Apr 5, 2024

Conversation

rjzamora
Copy link
Member

@rjzamora rjzamora commented Apr 5, 2024

While debugging some p2p failures in RAPIDS, I discovered that some workers occasionally hit the from_pyarrow_table_dispatch without having dask_cudf imported. This scenario is a bit hard to test, but the best way to avoid it is to have these p2p-related dispatch functions registered for a "lazy" dask_cudf import in dask.

@rjzamora rjzamora added the bug Something is broken label Apr 5, 2024
@rjzamora rjzamora self-assigned this Apr 5, 2024
Copy link
Contributor

github-actions bot commented Apr 5, 2024

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

     15 files  ±0       15 suites  ±0   3h 19m 50s ⏱️ + 2m 11s
 13 107 tests ±0   12 173 ✅ ±0     930 💤 ±0  4 ❌ ±0 
162 263 runs  ±0  142 174 ✅ +4  20 085 💤  - 4  4 ❌ ±0 

For more details on these failures, see this check.

Results for commit 2b381ea. ± Comparison against base commit ba24c0a.

@quasiben
Copy link
Member

quasiben commented Apr 5, 2024

Looks good. Is there a corresponding test we should add here or in the cuDF repo ?

@rjzamora
Copy link
Member Author

rjzamora commented Apr 5, 2024

Is there a corresponding test we should add here or in the cuDF repo

Test is a bit tricky and will be a modification of rapidsai/cudf#15469 - I'd like to avoid adding something fragile to gpuCI until the current suite is back in the green.

@quasiben
Copy link
Member

quasiben commented Apr 5, 2024

Maybe we can come back to it after all PRs are merged in. Thanks @rjzamora

@quasiben quasiben merged commit c860eb3 into dask:main Apr 5, 2024
27 of 29 checks passed
@rjzamora rjzamora deleted the lazy-register-p2p branch April 5, 2024 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something is broken
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants