Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HOTFIX] Changed code to disable k_truss on CUDA 11.4 differently #1813

Merged

Conversation

rlratzel
Copy link
Contributor

@rlratzel rlratzel commented Sep 9, 2021

Changed code to disable k_truss on CUDA 11.4 to not use numba.cuda.runtime.get_version() at import time since this creates a CUDA context which breaks dask LocalCUDACluster init (causes a nccl init invalid usage exception).

This code change results in almost the same UX as before, except explicit imports of the k_truss module no longer raise the exception immediately, and instead raise only when a k_truss algo is called.

Tested by ensuring unit tests for the conditional enabling of k_truss still work and also checked that a LocalCUDACluster could properly be created.

See #1811 for reference

@rlratzel rlratzel added bug Something isn't working ! - Hotfix non-breaking Non-breaking change labels Sep 9, 2021
@rlratzel rlratzel self-assigned this Sep 9, 2021
@rlratzel rlratzel requested a review from a team as a code owner September 9, 2021 20:54
@BradReesWork BradReesWork added this to the 21.10 milestone Sep 14, 2021
@BradReesWork
Copy link
Member

@raydouglass you need to merge this since I do not have permissions

@ajschmidt8 ajschmidt8 merged commit fa6f0f1 into rapidsai:branch-21.08 Sep 16, 2021
@rlratzel rlratzel deleted the branch-21.08-cuda114ktrusshotfix branch June 17, 2022 00:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants