[REVIEW] Pin dask and distributed for release

[galipremsagar]

This PR pins dask and distributed to 2022.9.2 for 22.10 release.

xref: rapidsai/cudf#11822

[galipremsagar]

@charlesbluca looks like the mix of dask-core(nightly) & dask(conda-forge) packages is biting us here again:

(cudfdev) pgali@dt07:/nvme/0/pgali/dask-cuda$ python
Python 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:56:21) 
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dask_cuda
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nvme/0/pgali/dask-cuda/dask_cuda/__init__.py", line 12, in <module>
    from .cuda_worker import CUDAWorker
  File "/nvme/0/pgali/dask-cuda/dask_cuda/cuda_worker.py", line 12, in <module>
    from distributed import Nanny
  File "/nvme/0/pgali/envs/cudfdev/lib/python3.9/site-packages/distributed/__init__.py", line 30, in <module>
    from distributed.deploy import Adaptive, LocalCluster, SpecCluster, SSHCluster
  File "/nvme/0/pgali/envs/cudfdev/lib/python3.9/site-packages/distributed/deploy/__init__.py", line 7, in <module>
    from distributed.deploy.local import LocalCluster
  File "/nvme/0/pgali/envs/cudfdev/lib/python3.9/site-packages/distributed/deploy/local.py", line 13, in <module>
    from distributed.deploy.utils import nprocesses_nthreads
  File "/nvme/0/pgali/envs/cudfdev/lib/python3.9/site-packages/distributed/deploy/utils.py", line 6, in <module>
    from dask.utils import factors
ImportError: cannot import name 'factors' from 'dask.utils' (/nvme/0/pgali/envs/cudfdev/lib/python3.9/site-packages/dask/utils.py)

So I went ahead and added a flag-based channel(dask/label/dev) removal in our conda recipe build.

cc: @jakirkham @pentschev

[pentschev]

Regarding the pinned version, it seems like there will be a release tomorrow after all: dask/community#278 . If possible, let's pin to that, we want to get dask/distributed#6996 in which will fix all the uncatched exceptions when shutting down a Dask cluster with UCX.

[galipremsagar]

rerun tests

[codecov-commenter]

Codecov Report

Base: 0.00% // Head: 0.00% // No change to project coverage 👍

Coverage data is based on head (ac1b938) compared to base (c0ae66c).
Patch has no changes to coverable lines.

Additional details and impacted files

@@             Coverage Diff              @@
##           branch-22.10   #1003   +/-   ##
============================================
  Coverage          0.00%   0.00%           
============================================
  Files                23      23           
  Lines              3102    3102           
============================================
  Misses             3102    3102           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[galipremsagar]

rerun tests

[pentschev]

LGTM, thanks @galipremsagar !

[pentschev]

Maybe in a follow-up PR, but would it make more sense to use the same logic as we do in ci/gpu/build.sh, or is there a reason I'm not aware for keeping pip install?

[galipremsagar]

I think at this point these are redundant, I'm planning on getting these removed in 22.12 unpinning PRs across some repos which still do pip installs.

[pentschev]

Ah ok, that would be even better. Thanks Prem!

[galipremsagar]

@gpucibot merge
@gpucibot merge

[galipremsagar]

@gpucibot merge