Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] libcudf install does not install nvcomp dependency #16986

Closed
jlowe opened this issue Oct 3, 2024 · 4 comments · Fixed by #16988
Closed

[BUG] libcudf install does not install nvcomp dependency #16986

jlowe opened this issue Oct 3, 2024 · 4 comments · Fixed by #16988
Labels
bug Something isn't working CMake CMake build issue libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS

Comments

@jlowe
Copy link
Member

jlowe commented Oct 3, 2024

Describe the bug
After #16946 performing a cmake install of libcudf no longer installs the files for the nvcomp dependency (i.e.: cmake config, shared libraries, etc.). This results in downstream projects depending on the libcudf build result to fail to configure due to the missing nvcomp dependency of libcudf, e.g.:

09:25:30  [INFO]      [exec] -- Configuring incomplete, errors occurred!
09:25:30  [INFO]      [exec] CMake Error at /usr/local/cmake-3.26.4-linux-x86_64/share/cmake-3.26/Modules/CMakeFindDependencyMacro.cmake:76 (find_package):
09:25:30  [INFO]      [exec]   By not providing "Findnvcomp.cmake" in CMAKE_MODULE_PATH this project has
09:25:30  [INFO]      [exec]   asked CMake to find a package configuration file provided by "nvcomp", but
09:25:30  [INFO]      [exec]   CMake did not find one.
09:25:30  [INFO]      [exec] 
09:25:30  [INFO]      [exec]   Could not find a package configuration file provided by "nvcomp" with any
09:25:30  [INFO]      [exec]   of the following names:
09:25:30  [INFO]      [exec] 
09:25:30  [INFO]      [exec]     nvcompConfig.cmake
09:25:30  [INFO]      [exec]     nvcomp-config.cmake
09:25:30  [INFO]      [exec] 
09:25:30  [INFO]      [exec]   Add the installation prefix of "nvcomp" to CMAKE_PREFIX_PATH or set
09:25:30  [INFO]      [exec]   "nvcomp_DIR" to a directory containing one of the above files.  If "nvcomp"
09:25:30  [INFO]      [exec]   provides a separate development package or SDK, be sure it has been
09:25:30  [INFO]      [exec]   installed.
09:25:30  [INFO]      [exec] Call Stack (most recent call first):
09:25:30  [INFO]      [exec]   /target/libcudf-install/lib64/cmake/cudf/cudf-dependencies.cmake:53 (find_dependency)
09:25:30  [INFO]      [exec]   /target/libcudf-install/lib64/cmake/cudf/cudf-config.cmake:90 (include)
09:25:30  [INFO]      [exec]   /target/jni/cmake-build/_deps/rapids-cmake-src/rapids-cmake/find/package.cmake:125 (find_package)
09:25:30  [INFO]      [exec]   CMakeLists.txt:146 (rapids_find_package)

Steps/Code to reproduce bug
Perform a cmake install of libcudf and note the lack of nvcomp artifacts in the install area.

To reproduce the downstream project error noted above:

git clone --depth 1 --branch branch-24.12 https://github.com/NVIDIA/spark-rapids-jni.git
cd spark-rapids-jni/thirdparty/cudf
git checkout branch-24.12
git pull
cd ../..
build/build-in-docker clean package -Dsubmodule.check.skip=true -Dlibcudf.dependency.mode=latest -DskipTests

Reverting #16946 in thirdparty/cudf before the build/build-in-docker command allows the build to succeed.

Expected behavior
nvcomp build artifacts are installed alongside libcudf artifacts for a libcudf install since nvcomp is a required dependency.

@jlowe jlowe added bug Something isn't working CMake CMake build issue libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS labels Oct 3, 2024
@bdice
Copy link
Contributor

bdice commented Oct 3, 2024

@KyleFromNVIDIA Could you take a look?

@KyleFromNVIDIA
Copy link
Contributor

Well, that is unfortunate. Luckily, this is only an issue for static builds, since nvcomp is a PRIVATE dependency of cudf, and even in static builds the only thing we need is the nvcomp shared library.

We may be able to add special logic to the cudf export file that, instead of calling find_dependency(nvcomp), simply adds the nvcomp::nvcomp imported target based on the location of the shared library in the nvcomp wheel.

@bdice
Copy link
Contributor

bdice commented Oct 3, 2024

I was with you up to the last part “based on the location of the shared library in the nvcomp wheel.” This should not affect wheel builds? This should only affect static libcudf builds, like those used by Spark — which have nothing to do with wheels. I think the issue is with the CPM-fetch code path, because we find nvcomp via conda packages or wheels for the other paths.

@KyleFromNVIDIA
Copy link
Contributor

Ah, you're right. In that case the solution is much simpler: provide an option in cpp/CMakeLists.txt to export or not export nvcomp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working CMake CMake build issue libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants