From 44e720d4c47caf5e2d3329182b5e4b8677dcb2a0 Mon Sep 17 00:00:00 2001 From: Tim Head Date: Thu, 6 Oct 2022 10:15:57 +0200 Subject: [PATCH 1/3] Update build instructions This explicilty mentions that nvcc has to be installed manually and updates the build and pytest commands. --- BUILD.md | 34 +++++++++++++--------------------- 1 file changed, 13 insertions(+), 21 deletions(-) diff --git a/BUILD.md b/BUILD.md index 31af4984d1..fa009bc104 100644 --- a/BUILD.md +++ b/BUILD.md @@ -29,7 +29,7 @@ In addition to the libraries included with cudatoolkit 11.0+, there are some oth #### Required - [RMM](https://github.com/rapidsai/rmm) corresponding to RAFT version. - + #### Optional - [Thrust](https://github.com/NVIDIA/thrust) v1.15 / [CUB](https://github.com/NVIDIA/cub) - On by default but can be disabled. - [cuCollections](https://github.com/NVIDIA/cuCollections) - Used in `raft::sparse::distance` API. @@ -139,7 +139,7 @@ RAFT's cmake has the following configurable flags available:. | RAFT_COMPILE_DIST_LIBRARY | ON, OFF | OFF | Compiles the `libraft-distance` shared library | | RAFT_ENABLE_NN_DEPENDENCIES | ON, OFF | OFF | Searches for dependencies of nearest neighbors API, such as FAISS, and compiles them if not found. Needed for `raft::spatial::knn` | | RAFT_ENABLE_thrust_DEPENDENCY | ON, OFF | ON | Enables the Thrust dependency. This can be disabled when using many simple utilities or to override with a different Thrust version. | -| RAFT_USE_FAISS_STATIC | ON, OFF | OFF | Statically link FAISS into `libraft-nn` | +| RAFT_USE_FAISS_STATIC | ON, OFF | OFF | Statically link FAISS into `libraft-nn` | | RAFT_STATIC_LINK_LIBRARIES | ON, OFF | ON | Build static link libraries instead of shared libraries | | DETECT_CONDA_ENV | ON, OFF | ON | Enable detection of conda environment for dependencies | | NVTX | ON, OFF | OFF | Enable NVTX Markers | @@ -151,7 +151,7 @@ Currently, shared libraries are provided for the `libraft-nn` and `libraft-dista ### Python -Conda environment scripts are provided for installing the necessary dependencies for building and using the Python APIs. It is preferred to use `mamba`, as it provides significant speedup over `conda`. The following example will install create and install dependencies for a CUDA 11.5 conda environment: +Conda environment scripts are provided for installing the necessary dependencies for building and using the Python APIs. It is preferred to use `mamba`, as it provides significant speedup over `conda`. In addition you will have to manually install `nvcc` as it will not be installed as part of the conda environment. The following example will install create and install dependencies for a CUDA 11.5 conda environment: ```bash mamba env create --name raft_env_name -f conda/environments/raft_dev_cuda11.5.yml @@ -161,32 +161,24 @@ mamba activate raft_env_name The Python APIs can be built using the `build.sh` script: ```bash -./build.sh raft-dask pylibraft -``` - -`setup.py` can also be used to build the Python APIs manually: -```bash -cd python/raft-dask -python setup.py build_ext --inplace -python setup.py install - -cd python/pylibraft -python setup.py build_ext --inplace -python setup.py install +# to build pylibraft +./build.sh libraft pylibraft --install --compile-libs +# to build raft-dask +./build.sh libraft raft-dask --install --compile-libs ``` To run the Python tests: ```bash cd python/raft-dask -py.test -s -v raft +py.test -s -v -cd python pylibraft -py.test -s -v pylibraft +cd python/pylibraft +py.test -s -v ``` ## Using RAFT in downstream projects -There are two different strategies for including RAFT in downstream projects, depending on whether or not the required dependencies are already installed and available on the `lib` and `include` paths. +There are two different strategies for including RAFT in downstream projects, depending on whether or not the required dependencies are already installed and available on the `lib` and `include` paths. ### C++ header-only integration using cmake @@ -219,7 +211,7 @@ The following example tells the compiler to ignore the pre-compiled templates fo ### Building RAFT C++ from source in cmake -RAFT uses the [RAPIDS-CMake](https://github.com/rapidsai/rapids-cmake) library so it can be more easily included into downstream projects. RAPIDS cmake provides a convenience layer around the [CMake Package Manager (CPM)](https://github.com/cpm-cmake/CPM.cmake). +RAFT uses the [RAPIDS-CMake](https://github.com/rapidsai/rapids-cmake) library so it can be more easily included into downstream projects. RAPIDS cmake provides a convenience layer around the [CMake Package Manager (CPM)](https://github.com/cpm-cmake/CPM.cmake). The following example is similar to invoking `find_package(raft)` but uses `rapids_cpm_find`, which provides a richer and more flexible configuration landscape by using CPM to fetch any dependencies not already available to the build. The `raft::raft` link target will be made available and it's recommended that it be used as a `PRIVATE` link dependency in downstream projects. The `COMPILE_LIBRARIES` option enables the building the shared libraries. @@ -234,7 +226,7 @@ set(RAFT_PINNED_TAG "branch-${RAFT_VERSION}") function(find_and_configure_raft) set(oneValueArgs VERSION FORK PINNED_TAG USE_FAISS_STATIC COMPILE_LIBRARIES ENABLE_NN_DEPENDENCIES CLONE_ON_PIN - USE_NN_LIBRARY USE_DISTANCE_LIBRARY + USE_NN_LIBRARY USE_DISTANCE_LIBRARY ENABLE_thrust_DEPENDENCY) cmake_parse_arguments(PKG "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN} ) From 4d49c6c60af09cbc402457e77ed393bc982c4785 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Thu, 6 Oct 2022 13:17:52 -0400 Subject: [PATCH 2/3] Adding manual install back in --- BUILD.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/BUILD.md b/BUILD.md index fa009bc104..735d3c6dbb 100644 --- a/BUILD.md +++ b/BUILD.md @@ -158,7 +158,7 @@ mamba env create --name raft_env_name -f conda/environments/raft_dev_cuda11.5.ym mamba activate raft_env_name ``` -The Python APIs can be built using the `build.sh` script: +The Python APIs can be built and installed using the `build.sh` script: ```bash # to build pylibraft @@ -167,6 +167,18 @@ The Python APIs can be built using the `build.sh` script: ./build.sh libraft raft-dask --install --compile-libs ``` +`setup.py` can also be used to build the Python APIs manually: + +``` +cd python/raft-dask +python setup.py build_ext --inplace +python setup.py install + +cd python/pylibraft +python setup.py build_ext --inplace +python setup.py install +``` + To run the Python tests: ```bash cd python/raft-dask From 3978928572ed1df6d9a30ab2c6f3943278fbdc1f Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Thu, 6 Oct 2022 13:20:33 -0400 Subject: [PATCH 3/3] Adding command to build docs to the BUILD readme --- BUILD.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/BUILD.md b/BUILD.md index 735d3c6dbb..c94bb24204 100644 --- a/BUILD.md +++ b/BUILD.md @@ -10,6 +10,7 @@ - [Googlebench](#gbench) - [C++ Using Cmake](#cpp_using_cmake) - [Python](#python) + - [Documentation](#docs) - [Using RAFT in downstream projects](#use_raft) - [Cmake Header-only Integration](#cxx_integration) - [Using Shared Libraries in Cmake](#use_shared_libs) @@ -188,6 +189,18 @@ cd python/pylibraft py.test -s -v ``` +### Documentation + +The documentation requires that the C++ headers and python packages have been built and installed. + +The following will build the docs along with the C++ and Python packages: + +``` +./build.sh libraft pylibraft raft-dask docs --compile-libs --install +``` + + + ## Using RAFT in downstream projects There are two different strategies for including RAFT in downstream projects, depending on whether or not the required dependencies are already installed and available on the `lib` and `include` paths.