From 75a69cc477a4197107d418c4d560839df3616a94 Mon Sep 17 00:00:00 2001 From: James Lamb Date: Fri, 15 Nov 2024 11:44:18 -0600 Subject: [PATCH] fix library-loading issues in editable installs (#553) Contributes to https://github.com/rapidsai/build-planning/issues/118 The pattern introduced in #551 breaks editable installs in devcontainers. In that type of build, `libkvikio.so` is built outside of the wheel but **not installed**, so it can't be found by `ld`. Extension modules in `kvikio` are able to find it via RPATHs instead. This proposes: * try-catching the entire library-loading attempt, to silently do nothing in cases like that * adding an import of the `kvikio` Python library in the `devcontainers` CI job, as a smoke test to catch issues like this in the future ## Notes for Reviewers ### How I tested this Reproduced that with the CUDA 12.5 pip devcontainers today: ```shell build-all python -c "import kvikio" # OSError: libkvikio.so: cannot open shared object file: No such file or directory ``` Confirmed that the changes in this PR fix that. Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) - Lawrence Mitchell (https://github.com/wence-) URL: https://github.com/rapidsai/kvikio/pull/553 --- .github/workflows/pr.yaml | 1 + python/libkvikio/libkvikio/load.py | 12 +++++++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pr.yaml b/.github/workflows/pr.yaml index fb439d8751..613b135035 100644 --- a/.github/workflows/pr.yaml +++ b/.github/workflows/pr.yaml @@ -104,6 +104,7 @@ jobs: build_command: | sccache -z; build-all --verbose; + python -c "import kvikio; print(kvikio.__version__)"; sccache -s; wheel-cpp-build: secrets: inherit diff --git a/python/libkvikio/libkvikio/load.py b/python/libkvikio/libkvikio/load.py index 2cca4b5fdc..8856923eaf 100644 --- a/python/libkvikio/libkvikio/load.py +++ b/python/libkvikio/libkvikio/load.py @@ -63,9 +63,15 @@ def load_library(): # Prefer the libraries bundled in this package. If they aren't found # (which might be the case in builds where the library was prebuilt # before packaging the wheel), look for a system installation. - libkvikio_lib = _load_wheel_installation(soname) - if libkvikio_lib is None: - libkvikio_lib = _load_system_installation(soname) + try: + libkvikio_lib = _load_wheel_installation(soname) + if libkvikio_lib is None: + libkvikio_lib = _load_system_installation(soname) + except OSError: + # If none of the searches above succeed, just silently return None + # and rely on other mechanisms (like RPATHs on other DSOs) to + # help the loader find the library. + pass # The caller almost never needs to do anything with this library, but no # harm in offering the option since this object at least provides a handle