Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent CUDA context errors when testing on single-GPU #737

Merged

Conversation

pentschev
Copy link
Member

Add a DASK_CUDA_TEST_SINGLE_GPU environment variable that allows
informing a single-GPU system is used for testing (such as gpuCI). This
then prevents throwing errors when attempting to mock create CUDA
context on devices that are specified via CUDA_VISIBLE_DEVICES but are
unavailable in the system.

Add a `DASK_CUDA_TEST_SINGLE_GPU` environment variable that allows
informing a single-GPU system is used for testing (such as gpuCI). This
then prevents throwing errors when attempting to mock create CUDA
context on devices that are specified via `CUDA_VISIBLE_DEVICES` but are
unavailable in the system.
@pentschev pentschev added bug Something isn't working 3 - Ready for Review Ready for review by team non-breaking Non-breaking change labels Sep 24, 2021
@pentschev pentschev requested review from a team as code owners September 24, 2021 13:09
@github-actions github-actions bot added gpuCI gpuCI issue python python code needed labels Sep 24, 2021
Copy link
Member

@madsbk madsbk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pentschev
Copy link
Member Author

Thanks @madsbk for the lightning fast review! 😄

@codecov-commenter
Copy link

codecov-commenter commented Sep 24, 2021

Codecov Report

Merging #737 (bb7c9d4) into branch-21.10 (8e6ab70) will increase coverage by 1.53%.
The diff coverage is 86.39%.

Impacted file tree graph

@@               Coverage Diff                @@
##           branch-21.10     #737      +/-   ##
================================================
+ Coverage         87.63%   89.16%   +1.53%     
================================================
  Files                15       15              
  Lines              1658     1911     +253     
================================================
+ Hits               1453     1704     +251     
- Misses              205      207       +2     
Impacted Files Coverage Δ
dask_cuda/cuda_worker.py 77.64% <ø> (ø)
dask_cuda/get_device_memory_objects.py 90.00% <0.00%> (+21.94%) ⬆️
dask_cuda/utils.py 81.49% <60.52%> (-5.78%) ⬇️
dask_cuda/proxify_device_objects.py 95.45% <80.00%> (+6.56%) ⬆️
dask_cuda/initialize.py 89.79% <84.37%> (+0.90%) ⬆️
dask_cuda/proxify_host_file.py 93.51% <90.66%> (-5.88%) ⬇️
dask_cuda/proxy_object.py 91.34% <94.44%> (+1.70%) ⬆️
dask_cuda/device_host_file.py 71.66% <100.00%> (+1.50%) ⬆️
dask_cuda/explicit_comms/dataframe/shuffle.py 98.69% <100.00%> (+0.65%) ⬆️
dask_cuda/is_device_object.py 92.59% <100.00%> (+29.62%) ⬆️
... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8355187...bb7c9d4. Read the comment docs.

@pentschev
Copy link
Member Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 19097b1 into rapidsai:branch-21.10 Sep 24, 2021
@pentschev pentschev deleted the prevent-cuda-context-test-errors branch October 22, 2021 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team bug Something isn't working gpuCI gpuCI issue non-breaking Non-breaking change python python code needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants