Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Statically link the CUDA runtime into a single shared library #164

Merged
merged 3 commits into from
Apr 11, 2022

Conversation

jlowe
Copy link
Member

@jlowe jlowe commented Apr 7, 2022

Closes #146.

This is a major update to the way this project builds and is designed to build all of the libcudf, libcudfjni, and libspark_rapids_jni code into a single shared library that is statically linked with the CUDA runtime. Because we are no longer leveraging the contents of a cudf-like jar, the project has been collapsed into a single top-level Maven module that pulls in the cudf Java sources.

During the build, CMake is invoked to build libcudf and libcudfjni as static libraries. Those static libraries are then whole-archive linked into a libcudf.so library which is statically linked with the CUDA runtime. In addition a stub libcudfjni.so library is created as well that depends on libcudf.so. This somewhat contorted build is done for the following reasons:

  • only have one static link of the CUDA runtime to minimize the number of separate CUDA contexts
  • preserve backwards compatibility with the cudf Java source that expects to find a libcudf.so and libcudfjni.so
  • preserve backwards compatibility with RAPIDS accelerated UDFs which may directly depend on libcudf.so and/or libcudfjni.so

This depends on rapidsai/cudf#10619.

@jlowe jlowe added the build label Apr 7, 2022
@jlowe jlowe self-assigned this Apr 7, 2022
@jlowe jlowe marked this pull request as ready for review April 7, 2022 22:38
@jlowe
Copy link
Member Author

jlowe commented Apr 7, 2022

build

1 similar comment
@jlowe
Copy link
Member Author

jlowe commented Apr 8, 2022

build

@jlowe
Copy link
Member Author

jlowe commented Apr 8, 2022

build

Copy link
Collaborator

@pxLi pxLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I will monitoring other nightly pipeline's status to see if we need to do an update

@pxLi pxLi merged commit 8c91522 into NVIDIA:branch-22.06 Apr 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Statically link the CUDA runtime
2 participants