prefer wheel-provided libcuspatial.so in load_library(), use RTLD_LOCAL #1483
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jameslamb
added
5 - DO NOT MERGE
vyasr
reviewed
Nov 13, 2024
Co-authored-by: Vyas Ramasubramani <[email protected]>
vyasr
reviewed
Nov 13, 2024
3 tasks
This was referenced Nov 14, 2024
vyasr
approved these changes
Nov 14, 2024
jameslamb
changed the title
jameslamb
changed the title
bdice
approved these changes
Nov 14, 2024
|
/merge |
rapids-bot bot
pushed a commit
to rapidsai/kvikio
that referenced
this pull request
Nov 14, 2024
Contributes to rapidsai/build-planning#118 Modifies `libkvikio.load_library()` in the following ways: * prefer wheel-provided `libkvikio.so` to system installation * expose environment variable `RAPIDS_LIBKVIKIO_PREFER_SYSTEM_LIBRARY` for switching that preference * load `libkvikio.so` with `RTLD_LOCAL`, to prevent adding symbols to the global namespace ([dlopen docs](https://linux.die.net/man/3/dlopen)) ## Notes for Reviewers ### How I tested this Tested the general approach on rapidsai/cudf#17316 and rapidsai/cuspatial#1483. Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) URL: #551
3 tasks
rapids-bot bot
pushed a commit
that referenced
this pull request
Nov 15, 2024
Contributes to rapidsai/build-planning#118 The pattern introduced in #1483 breaks editable installs in devcontainers. In that type of build, `libcuspatial.so` is built outside of the wheel but **not installed**, so it can't be found by `ld`. Extension modules in `cuspatial` are able to find it via RPATHs instead. This proposes: * try-catching the entire library-loading attempt, to silently do nothing in cases like that * ~adding an import of the `cuspatial` Python library in the `devcontainers` CI job, as a smoke test to catch issues like this in the future~ *(edit: removed those, [`devcontainer` builds run on CPU nodes](https://github.com/rapidsai/shared-workflows/blob/4e84062f333ce5649bc65029d3979569e2d0a045/.github/workflows/build-in-devcontainer.yaml#L19))* ## Notes for Reviewers ### How I tested this Tested this approach on rapidsai/kvikio#553 # Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Bradley Dice (https://github.com/bdice) URL: #1484
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Contributes to rapidsai/build-planning#118
Modifies
libcuspatial.load_library()in the following ways:libcuspatial.soto system installationRAPIDS_LIBCUSPATIAL_PREFER_SYSTEM_LIBRARYfor switching that preferencelibcuspatial.sowithRTLD_LOCAL, to prevent adding symbols to the global namespace (dlopen docs)Notes for Reviewers
How I tested this
See "How I tested this" in rapidsai/cudf#17316
Also opened this PR originally pulling in built packages from rapidsai/cudf#17316
Checklist