[REVIEW] Upgrade arrow to 4.0.1 #7495
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
CMake
galipremsagar
added
2 - In Progress
galipremsagar
added
0 - Blocked
kkraus14
reviewed
Mar 4, 2021
kkraus14
reviewed
Mar 4, 2021
kkraus14
reviewed
Mar 4, 2021
kkraus14
previously requested changes
Mar 4, 2021
kkraus14
added
breaking
galipremsagar
commented
Mar 5, 2021
galipremsagar
added
5 - Ready to Merge
|
rerun tests |
2 similar comments
|
rerun tests |
|
rerun tests |
|
@gpucibot merge |
1 similar comment
|
@gpucibot merge |
isVoid
approved these changes
Jun 29, 2021
dantegd
approved these changes
Jun 29, 2021
This was referenced Jun 30, 2021
This was referenced Jun 30, 2021
rapids-bot bot
pushed a commit
that referenced
this pull request
Jul 2, 2021
With #7495 merged, it seems like the dev environment files create an environment with the CPU packages for `pyarrow` and `arrow-cpp`; this results in failure when trying to compile libcudf or `import cudf`: ```python ModuleNotFoundError: No module named 'pyarrow._cuda' from cudf import rmm File "/opt/conda/lib/python3.8/site-packages/cudf/__init__.py", line 11, in <module> from cudf import core, datasets, testing File "/opt/conda/lib/python3.8/site-packages/cudf/core/__init__.py", line 3, in <module> from cudf.core import _internals, buffer, column, column_accessor, common File "/opt/conda/lib/python3.8/site-packages/cudf/core/_internals/__init__.py", line 3, in <module> from cudf.core._internals.where import where File "/opt/conda/lib/python3.8/site-packages/cudf/core/_internals/where.py", line 11, in <module> from cudf.core.column import ColumnBase File "/opt/conda/lib/python3.8/site-packages/cudf/core/column/__init__.py", line 3, in <module> from cudf.core.column.categorical import CategoricalColumn File "/opt/conda/lib/python3.8/site-packages/cudf/core/column/categorical.py", line 25, in <module> from cudf import _lib as libcudf File "/opt/conda/lib/python3.8/site-packages/cudf/_lib/__init__.py", line 4, in <module> from . import ( ImportError: libarrow_cuda.so.400: cannot open shared object file: No such file or directory ``` This updates the dev environments and recipe to ensure that the GPU package of `pyarrow` (and `arrow-cpp` accordingly) are used. Authors: - Charles Blackmon-Luca (https://github.com/charlesbluca) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) URL: #8637
This was referenced Jul 2, 2021
Merged
rapids-bot bot
pushed a commit
to rapidsai/cuspatial
that referenced
this pull request
Jul 2, 2021
…ing (#430) This PR contains three distinct changes required to get cuspatial builds working and tests passing again: 1. RMM switched to rapids-cmake (rapidsai/rmm#800), which requires CMake 3.20.1, so this PR includes the required updates for that. 2. The Arrow upgrade in cudf also moved the location of testing utilities (rapidsai/cudf#7495). Long term cuspatial needs to move away from use of the testing utilities, which are not part of cudf's public API, but we are currently blocked by rapidsai/cudf#8646, so this PR just imports the internal `assert_eq` method as a stopgap to get tests passing. 3. The changes in rapidsai/cudf#8373 altered the way that metadata was propagated to libcudf outputs from previously existing cuDF Python objects. The new code paths require cuspatial to override metadata copying at the GeoDataFrame rather than the GeoColumn level in order to ensure that information about column types is lost in the libcudf round trip and the metadata copying functions are now called on the output DataFrame rather than the input one. This PR supersedes #427, #428, and #429, all of which can now be closed. Authors: - Vyas Ramasubramani (https://github.com/vyasr) - GALI PREM SAGAR (https://github.com/galipremsagar) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Christopher Harris (https://github.com/cwharris) URL: #430
rapids-bot bot
pushed a commit
to rapidsai/cuspatial
that referenced
this pull request
Jul 21, 2021
As of rapidsai/cudf#7495 the `cudf.tests.utils` module (and in particular the `assert_eq` function) are no longer part of the public API. This PR switches tests to use the public testing functions in the `cudf.testing` subpackage. This PR is currently blocked by #430 and rapidsai/cudf#8646. Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Christopher Harris (https://github.com/cwharris) - Paul Taylor (https://github.com/trxcllnt) URL: #431
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: #7224
This PR:
cudf.testingmodule.