[FEA] Latest cudf breaks latest pandas

[mirekphd]

Describe the bug
Latest binary Anaconda release of cudf is incompatible with the latest PyPi release of pandas, causing reproducible import failure in the latter in various systems. This bug has been reproduced and analyzed here: pandas-dev/pandas#42506

Steps/Code to reproduce bug
Run this docker container with the clashing versions of both packages installed and try to import pandas:

$ docker run mirekphd/ml-gpu-py38-cuda112-cust:20210806 python -c "import pandas as pd; print(pd.__version__)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/conda/lib/python3.8/site-packages/pandas/__init__.py", line 144, in <module>
    from pandas.io.api import (
  File "/opt/conda/lib/python3.8/site-packages/pandas/io/api.py", line 8, in <module>
    from pandas.io.excel import ExcelFile, ExcelWriter, read_excel
  File "/opt/conda/lib/python3.8/site-packages/pandas/io/excel/__init__.py", line 1, in <module>
    from pandas.io.excel._base import ExcelFile, ExcelWriter, read_excel
  File "/opt/conda/lib/python3.8/site-packages/pandas/io/excel/_base.py", line 33, in <module>
    from pandas.io.parsers import TextParser
  File "/opt/conda/lib/python3.8/site-packages/pandas/io/parsers/__init__.py", line 1, in <module>
    from pandas.io.parsers.readers import (
  File "/opt/conda/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 17, in <module>
    from pandas._typing import (
ImportError: cannot import name 'DtypeArg' from 'pandas._typing' (/opt/conda/lib/python3.8/site-packages/pandas/_typing.py)

Expected behavior
Pandas import should succeed, pandas version 1.3.1 should be printed out.

Environment overview (please complete the following information)

• Environment location: [Docker]
• Method of cuDF install: [conda]:

$pip install cython cupy-cuda112 
$conda install -c rapidsai -c nvidia -c conda-forge blazingsql cudf python=$PYTHON_VERSION cudatoolkit

Environment details
Please run and paste the output of the cudf/print_env.sh script here, to gather any other relevant environment details
Warning: the script was not bundled with this version of cudf.

Additional context
The current workarounds are uninstalling cudf or downgrading pandas to the latest version compatible with cudf requirements (pandas==1.2.5), both of which restore pandas ability to be imported,
More info at:
pandas-dev/pandas#42506

[beckernick]

Hi @mirekphd . When multiple libraries have a high pace of development, the combined set of changes and dependencies can lead to unexpected breakages, exactly as you've noticed. As a result, cuDF currently pins pandas versions to pandas >=1.0,<1.3.0dev0.

We'll evaluate updating our pandas version requirements. In the meantime, you should be fine if you install pandas from conda-forge, as conda's SAT solver will identify the requirement from cuDF and install an appropriate version.

I'm going to close this issue as answered. Please let us know if the suggestion above doesn't work for you, and feel free to reopen the issue.

EDIT: I'm going to leave open this issue and tag it as a feature request to update our pandas requirement.

[beckernick]

Now covered by #9125