From 5b83ddd9fe440762ff39279d72ef85c1d83f5438 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Thu, 27 Jan 2022 21:52:10 -0500 Subject: [PATCH 01/10] iMore readme updates --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 88612f13f3..0ad18f82d3 100755 --- a/README.md +++ b/README.md @@ -1,24 +1,24 @@ #
 RAFT: RAPIDS Analytics Framework Toolkit
-RAFT is a [Scipy-like](https://scipy.org/) library for scientific computing, containing CUDA-accelerated building-blocks for rapidly composing analytics in the [RAPIDS](https://rapids.ai) ecosystem. These building-blocks include infrastructure as well as mathematical computational primitives, which accelerate the development of algorithms for data science applications. +RAFT contains fundamental widely-used algorithms and primitives for data science and ML. The algorithms are CUDA-accelerated and form building-blocks for rapidly composing analytics in the [RAPIDS](https://rapids.ai) ecosystem. By taking a primitives-based approach to algorithm development, RAFT 1. accelerates algorithm construction time 2. reduces the maintenance burden by maximizing reuse across projects, and 3. centralizes the core computations, allowing future optimizations to benefit all algorithms that use them. -RAFT provides a header-only C++ API (with optional shared libraries to accelerate build time) that cover the following general categories: +RAFT provides a header-only C++ API with optional shared libraries that contain algorithms in the following general categories: ##### | Category | Description / Examples | | --- | --- | | **Data Formats** | sparse & dense, conversions, and data generations | | **Data Generation** | sparse, spatial, machine learning datasets | -| **Dense Linear Algebra** | matrix arithmetic, norms, factorization | +| **Dense Linear Algebra** | matrix arithmetic, norms, factorization, least squares & eigenvalue problems | | **Spatial** | pairwise distances, nearest neighbors, neighborhood graph construction | | **Sparse Operations** | linear algebra, slicing, symmetrization, norms, spectral embedding, msf | | **Basic Clustering** | spectral clustering, hierarchical clustering, k-means | -| **Optimizers** | eigenvalue decomposition, least squares, and lanczos | +| **Iterative Solvers** | lanczos algorithm | | **Statistics** | sampling, moments, metrics | | **Distributed Tools** | multi-node multi-gpu infrastructure | From 77052b05b9803cc2672ee45318d52a1883c0830e Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Fri, 28 Jan 2022 10:05:54 -0500 Subject: [PATCH 02/10] Further updates --- README.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 0ad18f82d3..17eaa1923a 100755 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ #
 RAFT: RAPIDS Analytics Framework Toolkit
-RAFT contains fundamental widely-used algorithms and primitives for data science and ML. The algorithms are CUDA-accelerated and form building-blocks for rapidly composing analytics in the [RAPIDS](https://rapids.ai) ecosystem. +RAFT contains fundamental widely-used algorithms and primitives for data science, graph and ML. The algorithms are CUDA-accelerated and form building-blocks for rapidly composing analytics in the [RAPIDS](https://rapids.ai) ecosystem. By taking a primitives-based approach to algorithm development, RAFT 1. accelerates algorithm construction time @@ -16,13 +16,12 @@ RAFT provides a header-only C++ API with optional shared libraries that contain | **Data Generation** | sparse, spatial, machine learning datasets | | **Dense Linear Algebra** | matrix arithmetic, norms, factorization, least squares & eigenvalue problems | | **Spatial** | pairwise distances, nearest neighbors, neighborhood graph construction | -| **Sparse Operations** | linear algebra, slicing, symmetrization, norms, spectral embedding, msf | +| **Sparse Operations** | linear algebra, eigenvalue problems, slicing, symmetrization, norms, spectral embedding, msf | | **Basic Clustering** | spectral clustering, hierarchical clustering, k-means | -| **Iterative Solvers** | lanczos algorithm | -| **Statistics** | sampling, moments, metrics | +| **Statistics** | sampling, moments and summary statistics, metrics | | **Distributed Tools** | multi-node multi-gpu infrastructure | -RAFT also provides a Python API that enables the building of multi-node multi-GPU algorithms in the [Dask](https://dask.org/) ecosystem. We are continuing to improve the coverage of the Python API to expose the building-blocks from the categories above. +RAFT also provides a Python API that currently includes infrastructure for building multi-node multi-GPU algorithms in the [Dask](https://dask.org/) ecosystem. We are continuing to improve the coverage of the Python API to expose the building-blocks from the categories above. ## Getting started From a970c0496583d15d545d69063c04eebc7511fe99 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Mon, 31 Jan 2022 14:40:29 -0500 Subject: [PATCH 03/10] iFurther updates --- README.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 17eaa1923a..a084a802db 100755 --- a/README.md +++ b/README.md @@ -1,13 +1,13 @@ #
 RAFT: RAPIDS Analytics Framework Toolkit
-RAFT contains fundamental widely-used algorithms and primitives for data science, graph and ML. The algorithms are CUDA-accelerated and form building-blocks for rapidly composing analytics in the [RAPIDS](https://rapids.ai) ecosystem. +RAFT contains fundamental widely-used algorithms and primitives for data science, graph and machine learning. The algorithms are CUDA-accelerated and form building-blocks for rapidly composing analytics in the [RAPIDS](https://rapids.ai) ecosystem. By taking a primitives-based approach to algorithm development, RAFT 1. accelerates algorithm construction time 2. reduces the maintenance burden by maximizing reuse across projects, and 3. centralizes the core computations, allowing future optimizations to benefit all algorithms that use them. -RAFT provides a header-only C++ API with optional shared libraries that contain algorithms in the following general categories: +At its core, RAFT is a header-only C++ library with optional shared libraries that span the following categories: ##### | Category | Description / Examples | @@ -21,7 +21,11 @@ RAFT provides a header-only C++ API with optional shared libraries that contain | **Statistics** | sampling, moments and summary statistics, metrics | | **Distributed Tools** | multi-node multi-gpu infrastructure | -RAFT also provides a Python API that currently includes infrastructure for building multi-node multi-GPU algorithms in the [Dask](https://dask.org/) ecosystem. We are continuing to improve the coverage of the Python API to expose the building-blocks from the categories above. +RAFT also provides a Python library that includes +1. a python wrapper around the `raft::handle_t` for managing cuda library resources +2. building multi-node multi-GPU algorithms that leverage [Dask](https://dask.org/) + +We are continuing to improve the Python API by exposing the core algorithms and primitives from the categories above. ## Getting started From b9063dc3f212e8122578aa8d9b4da8705a605753 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Wed, 2 Feb 2022 13:03:15 -0500 Subject: [PATCH 04/10] updates to categories --- README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index a084a802db..c8f2a542a2 100755 --- a/README.md +++ b/README.md @@ -12,12 +12,14 @@ At its core, RAFT is a header-only C++ library with optional shared libraries th ##### | Category | Description / Examples | | --- | --- | -| **Data Formats** | sparse & dense, conversions, and data generations | +| **Data Formats** | sparse & dense, conversions, data generation | | **Data Generation** | sparse, spatial, machine learning datasets | -| **Dense Linear Algebra** | matrix arithmetic, norms, factorization, least squares & eigenvalue problems | +| **Dense Linear Algebra** | matrix arithmetic, norms, factorization, least squares, svd & eigenvalue problems | | **Spatial** | pairwise distances, nearest neighbors, neighborhood graph construction | -| **Sparse Operations** | linear algebra, eigenvalue problems, slicing, symmetrization, norms, spectral embedding, msf | +| **Sparse Operations** | linear algebra, eigenvalue problems, slicing, symmetrization, norms, spectral embedding | | **Basic Clustering** | spectral clustering, hierarchical clustering, k-means | +| **Combinatorial Optimization** | linear assignment problem, minimum spanning forest | +| **Iterative Solvers** | lanczos | | **Statistics** | sampling, moments and summary statistics, metrics | | **Distributed Tools** | multi-node multi-gpu infrastructure | From 3a4e00d1aa0142068f88776dcadb847bcb2c180e Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Wed, 2 Feb 2022 17:28:08 -0500 Subject: [PATCH 05/10] build instructions update --- BUILD.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/BUILD.md b/BUILD.md index f2186b9740..9538f8b41a 100644 --- a/BUILD.md +++ b/BUILD.md @@ -13,7 +13,7 @@ - NVIDIA driver 450.80.02+ - Pascal architecture of better (Compute capability >= 6.0) -C++ RAFT is a header-only library but provides the option of building shared libraries with template instantiations for common types to speed up compile times for larger projects. The recommended way to build and install RAFT is to use the `build.sh` script in the root of the repository. This script can build both the C++ and Python code and provides options for building and installing the shared libraries. +C++ RAFT is a header-only library but provides the option of building shared libraries with template instantiations for common types to speed up compile times for larger projects. The recommended way to build and install RAFT is to use the `build.sh` script in the root of the repository. This script can build both the C++ and Python code and provides options for building and installing the individual shared libraries. To run C++ tests: @@ -30,14 +30,14 @@ python -m pytest raft To build manually, you can also use `CMake` and setup.py directly. -For C++, the `RAFT_COMPILE_LIBRARIES` option can be used to compile the shared libraries. Shared libraries are provided for the `nn` and `distance` packages currently. The `nn` package requires FAISS, which will be built from source if it is not already installed. [FAISS](https://github.com/facebookresearch/faiss) can optionally be statically compiled into the `nn` shared library with the `RAFT_USE_FAISS_STATIC` option. +For C++, the `RAFT_COMPILE_LIBRARIES` option can be used to compile the shared libraries. Shared libraries are provided for the `libraft-nn` and `libraft-distance` components currently. The `libraft-nn` component depends upon [FAISS](https://github.com/facebookresearch/faiss) and the `RAFT_ENABLE_NN_DEPENDENCIES` option will build it from source if it is not already installed. FAISS can optionally be statically compiled into the `libraft-nn` shared library with the `RAFT_USE_FAISS_STATIC` option. To install RAFT into a specific location, use `CMAKE_INSTALL_PREFIX`. The snippet below will install it into the current conda environment. ```bash cd cpp mkdir build cd build -cmake -DRAFT_COMPILE_LIBRARIES=ON -DRAFT_USE_FAISS_STATIC=OFF -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX ../ +cmake -DRAFT_COMPILE_LIBRARIES=ON -DRAFT_USE_FAISS_STATIC=OFF -DRAFT_ENABLE_NN_DEPENDENCIES=ON -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX ../ make install ``` @@ -51,12 +51,16 @@ python setup.py install ## Using RAFT in downstream projects -### C++ Integration +### C++ header-only integration using cmake -Use RAFT in cmake projects with `find_package(raft)` for header-only operation and the `raft::raft` target will be available for configuring linking and `RAFT_INCLUDE_DIR` will be available for includes. Note that if any packages are used which require downstream dependencies, such as the `nn` package requiring FAISS, these dependencies will have be installed and configured in cmake independently. +Use RAFT in cmake projects with `find_package(raft)` for header-only operation and the `raft::raft` target will be available for configuring linking and `RAFT_INCLUDE_DIR` will be available for includes. Note that if any packages are used which require downstream dependencies, such as the `libraft-nn` package requiring FAISS, these dependencies will have be installed and configured in cmake independently. + +### Using pre-compiled shared libraries Use `find_package(raft COMPONENTS nn, distance)` to enable the shared libraries and pass dependencies through separate targets for each component. In this example, `raft::distance` and `raft::nn` targets will be available for configuring linking paths. These targets will also pass through any transitive dependencies (such as FAISS in the case of the `nn` package). + + ### Building RAFT C++ from source RAFT uses the [RAPIDS cmake](https://github.com/rapidsai/rapids-cmake) library, so it can be easily included into downstream projects. RAPIDS cmake provides a convenience layer around the [Cmake Package Manager (CPM)](https://github.com/cpm-cmake/CPM.cmake). The following example is similar to building RAFT itself from source but allows it to be done in cmake, providing the `raft::raft` link target and `RAFT_INCLUDE_DIR` for includes. The `COMPILE_LIBRARIES` option enables the building of the shared libraries From b9bd6cf1eddbb5d2246b31152db23fd780c0bd84 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Fri, 4 Feb 2022 17:25:51 -0500 Subject: [PATCH 06/10] build updates for using specializations --- BUILD.md | 6 ++++++ README.md | 4 ++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/BUILD.md b/BUILD.md index 9538f8b41a..51bc344631 100644 --- a/BUILD.md +++ b/BUILD.md @@ -59,7 +59,13 @@ Use RAFT in cmake projects with `find_package(raft)` for header-only operation a Use `find_package(raft COMPONENTS nn, distance)` to enable the shared libraries and pass dependencies through separate targets for each component. In this example, `raft::distance` and `raft::nn` targets will be available for configuring linking paths. These targets will also pass through any transitive dependencies (such as FAISS in the case of the `nn` package). +The pre-compiled libraries contain template specializations for commonly used types and require the additional include of header files with `extern template` definitions that tell the compiler not to instantiate templates that are already contained in the shared libraries. By convention, these header files are named `spectializations.hpp` and located in the base directory for the packages that contain specializations. +The following example shows how to use the `libraft-distance` API with the pre-compiled specializations: +```c++ +#include +#include +``` ### Building RAFT C++ from source diff --git a/README.md b/README.md index c8f2a542a2..47c653c484 100755 --- a/README.md +++ b/README.md @@ -10,13 +10,13 @@ By taking a primitives-based approach to algorithm development, RAFT At its core, RAFT is a header-only C++ library with optional shared libraries that span the following categories: ##### -| Category | Description / Examples | +| Category | Examples | | --- | --- | | **Data Formats** | sparse & dense, conversions, data generation | | **Data Generation** | sparse, spatial, machine learning datasets | | **Dense Linear Algebra** | matrix arithmetic, norms, factorization, least squares, svd & eigenvalue problems | | **Spatial** | pairwise distances, nearest neighbors, neighborhood graph construction | -| **Sparse Operations** | linear algebra, eigenvalue problems, slicing, symmetrization, norms, spectral embedding | +| **Sparse Operations** | linear algebra, eigenvalue problems, slicing, symmetrization, connected component labeling | | **Basic Clustering** | spectral clustering, hierarchical clustering, k-means | | **Combinatorial Optimization** | linear assignment problem, minimum spanning forest | | **Iterative Solvers** | lanczos | From ef4fd6ba945b6893a8aa09465b46e44577e1537b Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Fri, 4 Feb 2022 18:17:28 -0500 Subject: [PATCH 07/10] Many important updates to build.md --- BUILD.md | 86 ++++++++++++++++++++++++++++++++++++++++++++++++------- README.md | 3 +- 2 files changed, 78 insertions(+), 11 deletions(-) diff --git a/BUILD.md b/BUILD.md index 51bc344631..ab77bb034f 100644 --- a/BUILD.md +++ b/BUILD.md @@ -13,42 +13,108 @@ - NVIDIA driver 450.80.02+ - Pascal architecture of better (Compute capability >= 6.0) -C++ RAFT is a header-only library but provides the option of building shared libraries with template instantiations for common types to speed up compile times for larger projects. The recommended way to build and install RAFT is to use the `build.sh` script in the root of the repository. This script can build both the C++ and Python code and provides options for building and installing the individual shared libraries. +C++ RAFT is a header-only library but provides the option of building shared libraries with template instantiations for common types to speed up compile times for larger projects. -To run C++ tests: +The recommended way to build and install RAFT is to use the `build.sh` script in the root of the repository. This script can build both the C++ and Python code and provides options for building and installing the headers, Googletests, and individual shared libraries. + +### Header-only C++ + +RAFT depends on many different core libraries such as `thrust`, `cub`, `cucollections`, and `rmm`, which will be downloaded automatically by `cmake` even when only installing the headers. It's important to note that while all the headers will be installed and available, some parts of the RAFT API depend on libraries like `FAISS`, which can also be downloaded in the RAFT build but will need to be told to do so. +The following example builds and installs raft in header-only mode: ```bash -./test_raft +./build.sh libraft --nogtest ``` -To run Python tests, if `install` setup.py target is not run: +### Shared C++ Libraries (optional) + +Shared libraries are provided to speed up compile times for larger libraries which may heavily utilize some of the APIs. These shared libraries can also significantly improve re-compile times while developing against the APIs. + +Build all the shared libraries by passing `--compile-libs` flag to `build.sh`: ```bash -cd python -python -m pytest raft +./build.sh libraft --compile-libs --nogtest +``` + +To remain flexible, the individual shared libraries have their own flags and multiple can be used (though currently only the `nn` and `distance` packages contain shared libraries): +```bash +./build.sh libraft --compile-nn --compile-dist --nogtest +``` + +### Googletests + +Compile the Googletests by removing the `--nogtest` flag from `build.sh`: +```bash +./build.sh libraft --compile-nn --compile-dist ``` -To build manually, you can also use `CMake` and setup.py directly. +To run C++ tests: + +```bash +./test_raft +``` -For C++, the `RAFT_COMPILE_LIBRARIES` option can be used to compile the shared libraries. Shared libraries are provided for the `libraft-nn` and `libraft-distance` components currently. The `libraft-nn` component depends upon [FAISS](https://github.com/facebookresearch/faiss) and the `RAFT_ENABLE_NN_DEPENDENCIES` option will build it from source if it is not already installed. FAISS can optionally be statically compiled into the `libraft-nn` shared library with the `RAFT_USE_FAISS_STATIC` option. +### Build C++ Using cmake To install RAFT into a specific location, use `CMAKE_INSTALL_PREFIX`. The snippet below will install it into the current conda environment. ```bash cd cpp mkdir build cd build -cmake -DRAFT_COMPILE_LIBRARIES=ON -DRAFT_USE_FAISS_STATIC=OFF -DRAFT_ENABLE_NN_DEPENDENCIES=ON -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX ../ +cmake -D BUILD_TESTS=ON -DRAFT_COMPILE_LIBRARIES=ON -DRAFT_ENABLE_NN_DEPENDENCIES=ON -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX ../ make install ``` -For python: + +RAFT's cmake has the following configurable flags available:. + +| Flag | Possible Values | Default Value | Behavior | +| --- | --- | --- | --- | +| BUILD_TESTS | ON, OFF | ON | Compile Googletests | +| RAFT_COMPILE_LIBRARIES | ON, OFF | OFF | Compiles all `libraft` shared libraries (these are required for Googletests) | +| RAFT_COMPILE_NN_LIBRARY | ON, OFF | ON | Compiles the `libraft-nn` shared library | +| RAFT_COMPILE_DIST_LIBRARY | ON, OFF | ON | Compiles the `libraft-distance` shared library | +| RAFT_ENABLE_NN_DEPENDENCIES | ON, OFF | OFF | Searches for dependencies of nearest neighbors API, such as FAISS, and compiles them if not found. | +| RAFT_USE_FAISS_STATIC | ON, OFF | OFF | Statically link FAISS into `libraft-nn` | +| DETECT_CONDA_ENV | ON, OFF | ON | Enable detection of conda environment for dependencies | +| NVTX | ON, OFF | OFF | Enable NVTX Markers | +| CUDA_ENABLE_KERNELINFO | ON, OFF | OFF | Enables `kernelinfo` in nvcc. This is useful for `compute-sanitizer` | +| CUDA_ENABLE_LINEINFO | ON, OFF | OFF | Enable the -lineinfo option for nvcc | +| CUDA_STATIC_RUNTIME | ON, OFF | OFF | Statically link the CUDA runtime | + +Shared libraries are provided for the `libraft-nn` and `libraft-distance` components currently. The `libraft-nn` component depends upon [FAISS](https://github.com/facebookresearch/faiss) and the `RAFT_ENABLE_NN_DEPENDENCIES` option will build it from source if it is not already installed. + + + +### Python + +Conda environment scripts are provided for installing the necessary dependencies for building and using the Python APIs. It is preferred to use `mamba`, as it provides significant speedup over `conda`. The following example will install create and install dependencies for a CUDA 11.5 conda environment: +```bash +conda env create --name raft_env -f conda/environments/raft_dev_cuda11.5.yml +``` + +The Python API can be built using the `build.sh` script: + +```bash +./build.sh pyraft +``` + +To run Python tests, if `install` setup.py target is not run: + +`setup.py` can also be used to build the Python API manually: ```bash cd python python setup.py build_ext --inplace python setup.py install ``` +To run the Python tests: +```bash +cd python +python -m pytest raft +``` + ## Using RAFT in downstream projects ### C++ header-only integration using cmake diff --git a/README.md b/README.md index 47c653c484..9260c755dd 100755 --- a/README.md +++ b/README.md @@ -76,9 +76,10 @@ The folder structure mirrors other RAPIDS repos (cuDF, cuML, cuGraph...), with t - `ci`: Scripts for running CI in PRs - `conda`: Conda recipes and development conda environments - `cpp`: Source code for all C++ code. + - `docs`: Doxygen configuration - `include`: The C++ API is fully-contained here - `src`: Compiled template specializations for the shared libraries -- `docs`: Source code and scripts for building library documentation +- `docs`: Source code and scripts for building library documentation (doxygen + pydocs) - `python`: Source code for all Python source code. ## Contributing From 1f403be7f9b97cb39f6645132f06aa8681f28368 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Fri, 4 Feb 2022 18:26:39 -0500 Subject: [PATCH 08/10] Adding ToC to build.md --- BUILD.md | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/BUILD.md b/BUILD.md index ab77bb034f..8462fa4595 100644 --- a/BUILD.md +++ b/BUILD.md @@ -1,14 +1,21 @@ # RAFT Build and Development Guide - [Building and installing RAFT](#build_install) + - [CUDA/GPU Requirements](#cuda_gpu_req) + - [Header-only C++](#nstall_header_only_cpp) + - [C++ Shared Libraries](#shared_cpp_libs) + - [Googletests](#gtests) + - [C++ Using Cmake](#cpp_using_cmake) + - [Python](#python) - [Using RAFT in downstream projects](#use_raft) - - [C++ Integration](#cxx_integration) + - [Cmake Header-only Integration](#cxx_integration) + - [Using Shared Libraries in Cmake](#use_shared_libs) - [Building RAFT C++ from source](#build_cxx_source) - [Python/Cython Integration](#py_integration) ## Building and installing RAFT -### CUDA/GPU Requirements +### CUDA/GPU Requirements - CUDA 11.0+ - NVIDIA driver 450.80.02+ - Pascal architecture of better (Compute capability >= 6.0) @@ -17,7 +24,7 @@ C++ RAFT is a header-only library but provides the option of building shared lib The recommended way to build and install RAFT is to use the `build.sh` script in the root of the repository. This script can build both the C++ and Python code and provides options for building and installing the headers, Googletests, and individual shared libraries. -### Header-only C++ +### Header-only C++ RAFT depends on many different core libraries such as `thrust`, `cub`, `cucollections`, and `rmm`, which will be downloaded automatically by `cmake` even when only installing the headers. It's important to note that while all the headers will be installed and available, some parts of the RAFT API depend on libraries like `FAISS`, which can also be downloaded in the RAFT build but will need to be told to do so. @@ -26,7 +33,7 @@ The following example builds and installs raft in header-only mode: ./build.sh libraft --nogtest ``` -### Shared C++ Libraries (optional) +###C++ Shared Libraries (optional) Shared libraries are provided to speed up compile times for larger libraries which may heavily utilize some of the APIs. These shared libraries can also significantly improve re-compile times while developing against the APIs. @@ -41,7 +48,7 @@ To remain flexible, the individual shared libraries have their own flags and mul ./build.sh libraft --compile-nn --compile-dist --nogtest ``` -### Googletests +###Googletests Compile the Googletests by removing the `--nogtest` flag from `build.sh`: ```bash @@ -54,7 +61,7 @@ To run C++ tests: ./test_raft ``` -### Build C++ Using cmake +### C++ Using Cmake To install RAFT into a specific location, use `CMAKE_INSTALL_PREFIX`. The snippet below will install it into the current conda environment. ```bash @@ -86,7 +93,7 @@ Shared libraries are provided for the `libraft-nn` and `libraft-distance` compon -### Python +### Python Conda environment scripts are provided for installing the necessary dependencies for building and using the Python APIs. It is preferred to use `mamba`, as it provides significant speedup over `conda`. The following example will install create and install dependencies for a CUDA 11.5 conda environment: @@ -100,8 +107,6 @@ The Python API can be built using the `build.sh` script: ./build.sh pyraft ``` -To run Python tests, if `install` setup.py target is not run: - `setup.py` can also be used to build the Python API manually: ```bash cd python @@ -121,7 +126,7 @@ python -m pytest raft Use RAFT in cmake projects with `find_package(raft)` for header-only operation and the `raft::raft` target will be available for configuring linking and `RAFT_INCLUDE_DIR` will be available for includes. Note that if any packages are used which require downstream dependencies, such as the `libraft-nn` package requiring FAISS, these dependencies will have be installed and configured in cmake independently. -### Using pre-compiled shared libraries +### Using pre-compiled shared libraries Use `find_package(raft COMPONENTS nn, distance)` to enable the shared libraries and pass dependencies through separate targets for each component. In this example, `raft::distance` and `raft::nn` targets will be available for configuring linking paths. These targets will also pass through any transitive dependencies (such as FAISS in the case of the `nn` package). @@ -133,7 +138,7 @@ The following example shows how to use the `libraft-distance` API with the pre-c #include ``` -### Building RAFT C++ from source +### Building RAFT C++ from source in cmake RAFT uses the [RAPIDS cmake](https://github.com/rapidsai/rapids-cmake) library, so it can be easily included into downstream projects. RAPIDS cmake provides a convenience layer around the [Cmake Package Manager (CPM)](https://github.com/cpm-cmake/CPM.cmake). The following example is similar to building RAFT itself from source but allows it to be done in cmake, providing the `raft::raft` link target and `RAFT_INCLUDE_DIR` for includes. The `COMPILE_LIBRARIES` option enables the building of the shared libraries From b05d76594f0ed1de8dae454d89be1f0baf6482dc Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Fri, 4 Feb 2022 18:34:42 -0500 Subject: [PATCH 09/10] Ading closing anchors --- BUILD.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/BUILD.md b/BUILD.md index 8462fa4595..2ea38205a4 100644 --- a/BUILD.md +++ b/BUILD.md @@ -33,7 +33,7 @@ The following example builds and installs raft in header-only mode: ./build.sh libraft --nogtest ``` -###C++ Shared Libraries (optional) +###C++ Shared Libraries (optional) Shared libraries are provided to speed up compile times for larger libraries which may heavily utilize some of the APIs. These shared libraries can also significantly improve re-compile times while developing against the APIs. @@ -48,7 +48,7 @@ To remain flexible, the individual shared libraries have their own flags and mul ./build.sh libraft --compile-nn --compile-dist --nogtest ``` -###Googletests +###Googletests Compile the Googletests by removing the `--nogtest` flag from `build.sh`: ```bash @@ -61,7 +61,7 @@ To run C++ tests: ./test_raft ``` -### C++ Using Cmake +### C++ Using Cmake To install RAFT into a specific location, use `CMAKE_INSTALL_PREFIX`. The snippet below will install it into the current conda environment. ```bash From ef77921568f04f9b542295be47f6079031715d33 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Fri, 4 Feb 2022 18:35:41 -0500 Subject: [PATCH 10/10] iTypot --- BUILD.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BUILD.md b/BUILD.md index 2ea38205a4..b10dc87f89 100644 --- a/BUILD.md +++ b/BUILD.md @@ -93,7 +93,7 @@ Shared libraries are provided for the `libraft-nn` and `libraft-distance` compon -### Python +### Python Conda environment scripts are provided for installing the necessary dependencies for building and using the Python APIs. It is preferred to use `mamba`, as it provides significant speedup over `conda`. The following example will install create and install dependencies for a CUDA 11.5 conda environment: