Remove Distributed tests from CI

[pentschev]

Now that Distributed runs GPU tests in CI, running them here is redundant and can be removed.

[pentschev]

@quasiben @charlesbluca could you take a look at this and let me know if there's any reason why we shouldn't do it now? I think all of the tests are now being properly executed in Distributed's repo, so they're probably not required here anymore. Note that the same tests are running in the UCX-Py repo, I'm leaving them there for now because we test both the UCX 1.9 conda package and the 1.11 build from source.

[pentschev]

rerun tests

[charlesbluca]

I think these should be good to remove - would it be worth it to add something like UCX_VER to the Dask/Distributed test/image matrix moving forward, or is that less of a priority since UCX-Py is getting upstreamed?

[pentschev]

I think these should be good to remove - would it be worth it to add something like UCX_VER to the Dask/Distributed test/image matrix moving forward, or is that less of a priority since UCX-Py is getting upstreamed?

The problem is that today we need to build UCX 1.11 from source, which is done during the CI build script, and this is something I would not like to do in Distributed CI to prevent unexpected failures from blocking CI there. Once we officially upgrade to UCX 1.11 and build conda packages for it we don't need to keep testing both versions, then we can just use the 1.11 package in Distributed and remove Distributed tests from UCX-Py as well.

[pentschev]

Thanks @charlesbluca for comment and @ajschmidt8 for approval. Merging.

[pentschev]

@gpucibot merge