Relax protobuf lower bound to 3.20. #15506
Conversation
|
The test in 6dda780 went about like I expected. Wheels succeeded and conda ran into conflicts (specifically with pytorch, which is used for some cudf tests). Protobuf is used and pinned by more dependencies on the conda side, so I think it would be okay to relax the pinnings in dependencies.yaml (which would affect wheels) and leave the previous pinnings for conda recipes (meta.yaml). @beckernick Would that be a sufficient solution for Colab integration? |
This reverts commit 6dda780.
bdice
added
improvement
Nice! Thanks for doing this test. Perhaps a naive question, but are there any IO implications if the ORC file statistics are generated from a cuDF built against a protobuf major version that differs from the cuDF used to read the ORC file later? |
|
Good question - I am pretty sure the answer is no, it doesn’t matter. This is only used for reading statistics, the writer is unaffected. If an old protobuf caused problems at runtime, I would expect it to show up somewhere in the cudf test suite. Everything passed so I think we’re okay. |
|
See #15511 for my longer-term proposal (I have no objections to getting this PR in sooner). |
|
I also tested the artifacts from this PR in Colab and saw no conflicts. 
|
|
/merge |
|
I confirmed that nightlies with this PR have no conflicts with Colab's pre-installed package versions.
|
This PR drops the lower bound of protobuf to 3.20, to make cuDF compatible with the versions used in Google Colab. I tested this manually in Google Colab, which uses protobuf 3.20, and cuDF 24.02 seemed to work fine when reading ORC statistics (the only runtime feature in cuDF that needs protobuf). Note: cuDF 24.02 was built was a newer protobuf/protoc, version 4.x. I will test this by forcing protobuf 3.20 in CI, and then revert those changes if tests pass. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Nick Becker (https://github.com/beckernick) - Ray Douglass (https://github.com/raydouglass) - Matthew Roeschke (https://github.com/mroeschke) URL: rapidsai#15506
Backport of #15506 to cuDF 24.04. Also backports #15574 to ignore a warning from newer cupy releases. --------- Co-authored-by: Vyas Ramasubramani <[email protected]>
Description
This PR drops the lower bound of protobuf to 3.20, to make cuDF compatible with the versions used in Google Colab.
I tested this manually in Google Colab, which uses protobuf 3.20, and cuDF 24.02 seemed to work fine when reading ORC statistics (the only runtime feature in cuDF that needs protobuf). Note: cuDF 24.02 was built was a newer protobuf/protoc, version 4.x.
I will test this by forcing protobuf 3.20 in CI, and then revert those changes if tests pass.
Checklist