Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] incompatible protobuf dep version when installing cudf #10990

Closed
pxLi opened this issue May 27, 2022 · 3 comments · Fixed by #10996
Closed

[BUG] incompatible protobuf dep version when installing cudf #10990

pxLi opened this issue May 27, 2022 · 3 comments · Fixed by #10996
Assignees
Labels
bug Something isn't working

Comments

@pxLi
Copy link
Member

pxLi commented May 27, 2022

Describe the bug
To use conda install cudf=22.08/22.06 will introduce incompatible protobuf version as dependency,

conda install -c conda-forge mamba
mamba install -y -c rapidsai -c rapidsai-nightly -c nvidia -c conda-forge -c defaults cudf=22.08 python=3.8 cudatoolkit=11.2

which gave us protobuf==3.19.4

when we try import cudf, we will get,

>>> import cudf
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/lib/python3.8/site-packages/cudf/__init__.py", line 71, in <module>
    from cudf.io import (
  File "/opt/conda/lib/python3.8/site-packages/cudf/io/__init__.py", line 8, in <module>
    from cudf.io.orc import read_orc, read_orc_metadata, to_orc
  File "/opt/conda/lib/python3.8/site-packages/cudf/io/orc.py", line 14, in <module>
    from cudf.utils.metadata import (  # type: ignore
  File "/opt/conda/lib/python3.8/site-packages/cudf/utils/metadata/orc_column_statistics_pb2.py", line 7, in <module>
    from google.protobuf.internal import builder as _builder

image

and this also happens to 22.06 intermittently, if we are lucky enough we could get protobuf>=3.20+, but not always

manual run pip install --upgrade protobuf to upgrade protobuf version (3.20+) could resolve this issue.
Seems this is like one issue before #8570

Steps/Code to reproduce bug

conda install -c conda-forge mamba
mamba install -y -c rapidsai -c rapidsai-nightly -c nvidia -c conda-forge -c defaults cudf=22.08 python=3.8 cudatoolkit=11.2

python
import cudf

Expected behavior
import cudf correctly

Environment overview (please complete the following information)
nvidia/cuda:11.2.2-runtime-centos7

@shwina
Copy link
Contributor

shwina commented May 27, 2022

Hi @pxLi -- is this installing into a fresh environment or an existing one? Perhaps you could try mamba create rather than mamba install?

@galipremsagar galipremsagar self-assigned this May 27, 2022
@galipremsagar galipremsagar removed the Needs Triage Need team to review and classify label May 27, 2022
@pxLi
Copy link
Member Author

pxLi commented May 30, 2022

Hi @pxLi -- is this installing into a fresh environment or an existing one? Perhaps you could try mamba create rather than mamba install?

hi, I guess this does not help consistently. ref: #8570 (comment)
Pin the correct version in cudf conda env should help in this case #10996 thx @galipremsagar

ajschmidt8 pushed a commit that referenced this issue May 31, 2022
Fixes: #10990 

This PR pins the` protobuf` version to the latest and chooses narrower pinning because of the lower likelihood of breaking API changes between minor patch versions as opposed to major version changes.

Authors:
   - GALI PREM SAGAR (https://github.com/galipremsagar)

Approvers:
   - AJ Schmidt (https://github.com/ajschmidt8)
@galipremsagar
Copy link
Contributor

resolved in #10996

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants