[BUG] dask_sql does not import on CPU CI when RAPIDS is installed

[lmeyerov]

What happened:

import dask_sql throws RuntimeError

This is due to not handling that import dask_cudf in RAPIDS 22.04 may now throw RuntimeError (vs just ImportError)

The env scenario where this would happen is a CPU box that has (inactive) rapids libs. E.g., CPU CI, or a universal distro, where rapids python is installed in the env, but there is no actual libcuda, and the user is planning to use just cpu paths with dask_sql

This is a continuation of upstream issue rapidsai/rmm#1022

What you expected to happen:

import dask_sql

to import without exn and not use a GPU

Minimal Complete Verifiable Example:

See rapidsai/rmm#1022

Anything else we need to know?:

A hotfix for 22.04 may be to catch RuntimeError during the import dask_cudf catch block, and re-release to conda. TBD if RAPIDS will re-release 22.04 (e.g., containers) after this.

The long-term fix is unclear b/c RAPIDS is trying to decide how this scenario should behave..

Environment:

• dask-sql version: 22.04
• Python version: 3.8
• Operating System: ubuntu
• Install method (conda, pip, source):

• cpu docker host
• rapids base container (22.04)

[charlesbluca]

Thanks for raising this issue @lmeyerov! As mentioned in rapidsai/rmm#1022, this should be resolved on cuDF's end with rapidsai/cudf#10653.

However, I do think that this issue serves as good motivation to move the cuDF / dask-cuDF imports in dask-sql so that they only occur if a user attempts to do GPU operations - this ensures that an issue like this shouldn't cause issues for us in the future. I will follow up with a PR closing this issue 🙂

[charlesbluca]

Opened #480 which should resolve this issue

[charlesbluca]

Hey @lmeyerov this change is now incorporated into the latest dask-sql nightlies - not sure if you are pulling those for Graphistry CI but the issue should now be resolved on both ends 😄 feel free to reopen if this problem seems to be persisting

[lmeyerov]

Thanks! RAPIDS fixed on their side as well, and good to know no repeats going forward in case they regress again