ProxifyHostFile: tracking of external objects

[madsbk]

This PR implements tracking of external objects. The idea is that tasks can register objects that should be exposed to spilling when ProxifyHostFile is running out of memory.

Also, this PR makes the long running explicit-comms tasks local_shuffle() use this new feature, which means it shouldn't require more GPU memory than the regular shuffle implementation in Dask.

[codecov-io]

Codecov Report

Merging #527 (2eb4acb) into branch-0.19 (a574ccb) will increase coverage by 30.50%.
The diff coverage is 92.30%.

@@               Coverage Diff                @@
##           branch-0.19     #527       +/-   ##
================================================
+ Coverage        61.90%   92.41%   +30.50%     
================================================
  Files               22       16        -6     
  Lines             2452     1595      -857     
================================================
- Hits              1518     1474       -44     
+ Misses             934      121      -813     

Impacted Files	Coverage Δ
dask_cuda/proxify_device_objects.py	88.52% <69.23%> (+6.89%)	⬆️
dask_cuda/explicit_comms/dataframe/shuffle.py	98.61% <100.00%> (+0.83%)	⬆️
dask_cuda/proxify_host_file.py	100.00% <100.00%> (ø)
dask_cuda/proxy_object.py	91.26% <100.00%> (+1.18%)	⬆️
dask_cuda/_version.py
dask_cuda/benchmarks/local_cudf_shuffle.py
dask_cuda/benchmarks/local_cupy_map_overlap.py
dask_cuda/benchmarks/utils.py
dask_cuda/benchmarks/local_cupy.py
dask_cuda/benchmarks/local_cudf_merge.py
... and 9 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a574ccb...bc67216. Read the comment docs.

[madsbk]

@beckernick @VibhuJawa, it would be great if one of you have time try this PR. You need to set DASK_JIT_UNSPILL=True and DASK_EXPLICIT_COMMS=True.

[pentschev]

I have some questions here, but overall LGTM.

[madsbk]

Thanks for the review @pentschev, I think I have addressed all of your concerns. Also added some overall description: c6321bd

[pentschev]

LGTM, thanks @madsbk !

[pentschev]

If there's no rush, I would wait another day to see if @beckernick or @VibhuJawa have a chance to try out the PR, otherwise it can be merged any time IMO.

[madsbk]

Since explicit-comms's shuffle is disabled by default, I think it is safe to merge.

@gpucibot merge

[madsbk]

@gpucibot merge

[pentschev]

Thanks @madsbk !