azureml-sdk downgrades pyarrow to 3.0.0 which breaks cudf

[mmccarty]

Steps to reproduce

1. Create a fresh RAPIDS conda environment
   conda create -n rapids-22.06 -c rapidsai -c nvidia -c conda-forge rapids=22.06 python=3.8 cudatoolkit=11.5
2. conda activate rapids-22.06
3. conda list | grep pyarrow shows 7.0.0 installed
4. Launch python/ipython and import cudf should work
5. pip install azureml-sdk
6. Launch python/ipython and import cudf fails
7. conda list | grep pyarrow shows 3.0.0 installed

Error:

$ python -m cudf
Traceback (most recent call last):
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/runpy.py", line 144, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/site-packages/cudf/__init__.py", line 13, in <module>
    from cudf import api, core, datasets, testing
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/site-packages/cudf/datasets.py", line 7, in <module>
    from cudf._lib.transform import bools_to_mask
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/site-packages/cudf/_lib/__init__.py", line 4, in <module>
    from . import (
  File "cudf/_lib/avro.pyx", line 1, in init cudf._lib.avro
  File "cudf/_lib/column.pyx", line 1, in init cudf._lib.column
  File "cudf/_lib/scalar.pyx", line 37, in init cudf._lib.scalar
  File "cudf/_lib/interop.pyx", line 1, in init cudf._lib.interop
AttributeError: module 'pyarrow.lib' has no attribute 'MonthDayNanoIntervalArray'

[mmccarty]

Related to Azure/azure-sdk-for-python#25124 and Azure/azure-sdk-for-python#24644

[jacobtomlinson]

It looks like azureml-sdk has been superseded by azure-ai-ml so this ticket should be closed by rapidsai/deployment#181.

cc @skirui-source