Critical doc fixes and updates for 23.08

README.md

@@ -1,19 +1,20 @@
-# <div align="left"><img src="https://rapids.ai/assets/images/rapids_logo.png" width="90px"/>&nbsp;RAFT: Reusable Accelerated Functions and Tools</div>
+# <div align="left"><img src="https://rapids.ai/assets/images/rapids_logo.png" width="90px"/>&nbsp;RAFT: Reusable Accelerated Functions and Tools for Vector Search and More</div>
 
-![Navigating the canyons of accelerated possibilities](img/raft.png)
+![RAFT tech stack](img/raft-tech-stack-vss.png)
 
 ## Resources
 
 - [RAFT Reference Documentation](https://docs.rapids.ai/api/raft/stable/): API Documentation.
 - [RAFT Getting Started](./docs/source/quick_start.md): Getting started with RAFT.
 - [Build and Install RAFT](./docs/source/build.md): Instructions for installing and building RAFT.
+- [Example Notebooks](./notebooks): Example jupyer notebooks
 - [RAPIDS Community](https://rapids.ai/community.html): Get help, contribute, and collaborate.
 - [GitHub repository](https://github.com/rapidsai/raft): Download the RAFT source code.
 - [Issue tracker](https://github.com/rapidsai/raft/issues): Report issues or request features.
 
 ## Overview
 
-RAFT contains fundamental widely-used algorithms and primitives for data science and machine learning. The algorithms are CUDA-accelerated and form building blocks for rapidly composing analytics.
+RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.
 
 By taking a primitives-based approach to algorithm development, RAFT
 - accelerates algorithm construction time
@@ -22,16 +23,16 @@ By taking a primitives-based approach to algorithm development, RAFT
 
 While not exhaustive, the following general categories help summarize the accelerated functions in RAFT:
 #####
-| Category | Examples |
-| --- | --- |
-| **Data Formats** | sparse & dense, conversions, data generation |
+| Category | Examples                                                                                                                          |
+| --- |-----------------------------------------------------------------------------------------------------------------------------------|
+| **Data Formats** | sparse & dense, conversions, data generation                                                                                      |
 | **Dense Operations** | linear algebra, matrix and vector operations, reductions, slicing, norms, factorization, least squares, svd & eigenvalue problems |
-| **Sparse Operations** | linear algebra, eigenvalue problems, slicing, norms, reductions, factorization, symmetrization, components & labeling |
-| **Spatial** | pairwise distances, nearest neighbors, neighborhood graph construction |
-| **Basic Clustering** | spectral clustering, hierarchical clustering, k-means |
-| **Solvers** | combinatorial optimization, iterative solvers |
-| **Statistics** | sampling, moments and summary statistics, metrics |
-| **Tools & Utilities** | common utilities for developing CUDA applications, multi-node multi-gpu infrastructure |
+| **Sparse Operations** | linear algebra, eigenvalue problems, slicing, norms, reductions, factorization, symmetrization, components & labeling             |
+| **Spatial** | pairwise distances, nearest neighbors and vector search, neighborhood graph construction                                          |
+| **Basic Clustering** | spectral clustering, hierarchical clustering, k-means                                                                             |
+| **Solvers** | combinatorial optimization, iterative solvers                                                                                     |
+| **Statistics** | sampling, moments and summary statistics, metrics                                                                                 |
+| **Tools & Utilities** | common utilities for developing CUDA applications, multi-node multi-gpu infrastructure                                            |
 
 
 RAFT is a C++ header-only template library with an optional shared library that
@@ -44,6 +45,29 @@ In addition being a C++ library, RAFT also provides 2 Python libraries:
 
 ![RAFT is a C++ header-only template library with optional shared library and lightweight Python wrappers](img/arch.png)
 
+## Use cases
+
+### Vector Similarity Search
+
+RAFT contains state-of-the-art implementations of approximate nearest neighbors algorithms on the GPU that enable vector similarity search. Vector similarity search applications often require fast online queries done one-at-a-time and RAFT's graph-based [CAGRA](https://docs.rapids.ai/api/raft/nightly/pylibraft_api/neighbors/#cagra) algorithm outperforms the state-of-the art on the CPU (hierarchical navigable small-world graph or HNSW).
+
+In addition to CAGRA, RAFT contains other state-of-the-art GPU-accelerated implementations of popular algorithms for vector similarity search, such as [IVF-Flat](https://docs.rapids.ai/api/raft/nightly/pylibraft_api/neighbors/#ivf-flat) and [IVF-PQ](https://docs.rapids.ai/api/raft/nightly/pylibraft_api/neighbors/#ivf-pq) algorithms originally popularized by the [FAISS](https://github.com/facebookresearch/faiss) library.
+
+### Information Retrieval
+
+RAFT also contains a catalog of reusable primitives for composing algorithms that require fast neighborhood computations, such as
+
+1. Computing distances between vectors and computing kernel gramm matrices
+2. Performing ball radius queries for constructing epsilon neighborhoods
+3. Clustering points to partition a space for smaller and faster searches
+4. Constructing neighborhood "connectivities" graphs from dense vectors
+
+As an example, computations such as the above list are critical for information retrieval, data mining, and machine learning applications such as clustering, manifold learning, and dimensionality reduction.
+
+## Is RAFT right for me?
+
+RAFT contains low level primitives for accelerating applications and workflows. Data source providers and application developers may find specific tools -- like ANN algorithms -- very useful. RAFT is not intended to be used directly by data scientists for discovery and experimentation. For data science tools, please see the [RAPIDS website](https://rapids.ai/).
+
 ## Getting started
 
 ### RAPIDS Memory Manager (RMM)
@@ -323,3 +347,14 @@ If citing the sparse pairwise distances API, please consider using the following
   year={2021}
 }
 ```
+
+If citing the single-linkage agglomerative clustering APIs, please consider the following bibtex:
+```bibtex
+@misc{nolet2023cuslink,
+      title={cuSLINK: Single-linkage Agglomerative Clustering on the GPU}, 
+      author={Corey J. Nolet and Divye Gala and Alex Fender and Mahesh Doijade and Joe Eaton and Edward Raff and John Zedlewski and Brad Rees and Tim Oates},
+      year={2023},
+      eprint={2306.16354},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG}
+}

cpp/doxygen/Doxyfile

@@ -893,11 +893,8 @@ EXCLUDE                = ../include/raft/sparse/linalg/symmetrize.hpp \
                          ../include/raft/util/device_utils.cuh \
                          ../include/raft/core/error.hpp \
                          ../include/raft/core/handle.hpp \
-                         ../include/raft/util/integer_utils.hpp \
-                         ../include/raft/core/interruptible.hpp \
-                         ../include/raft/core/mdarray.hpp \
+                         ../include/raft/util/integer_utils.hpp
                          ../include/raft/util/pow2_utils.cuh \
-                         ../include/raft/core/span.hpp \
                          ../include/raft/util/vectorized.cuh \
                          ../include/raft/raft.hpp \
                          ../include/raft/core/cudart_utils.hpp \

cpp/include/raft/core/coo_matrix.hpp

@@ -23,6 +23,11 @@
 
 namespace raft {
 
+/**
+ * \defgroup coo_matrix COO Matrix
+ * @{
+ */
+
 template <typename RowType, typename ColType, typename NZType, int is_device>
 class coordinate_structure_t : public sparse_structure<RowType, ColType, NZType, is_device> {
  public:
@@ -289,4 +294,7 @@ class coo_matrix
     }
   }
 };
+
+/** @} */
+
 }  // namespace raft

cpp/include/raft/core/csr_matrix.hpp

@@ -22,6 +22,11 @@
 
 namespace raft {
 
+/**
+ * \defgroup csr_matrix CSR Matrix
+ * @{
+ */
+
 template <typename IndptrType, typename IndicesType, typename NZType, int is_device>
 class compressed_structure_t : public sparse_structure<IndptrType, IndicesType, NZType, is_device> {
  public:
@@ -301,4 +306,7 @@ class csr_matrix
     }
   }
 };
+
+/** @} */
+
 }  // namespace raft

cpp/include/raft/core/device_coo_matrix.hpp

@@ -23,6 +23,11 @@
 
 namespace raft {
 
+/**
+ * \defgroup device_coo_matrix Device COO Matrix
+ * @{
+ */
+
 /**
  * Specialization for a sparsity-preserving coordinate structure view which uses device memory
  */
@@ -388,4 +393,6 @@ auto make_device_coordinate_structure_view(raft::device_span<RowType> rows,
   return device_coordinate_structure_view<RowType, ColType, NZType>(rows, cols, n_rows, n_cols);
 }
 
+/** @} */
+
 };  // namespace raft

cpp/include/raft/core/device_csr_matrix.hpp

@@ -25,6 +25,11 @@
 
 namespace raft {
 
+/**
+ * \defgroup device_csr_matrix Device CSR Matrix Types
+ * @{
+ */
+
 /**
  * Specialization for a sparsity-preserving compressed structure view which uses device memory
  */
@@ -414,4 +419,6 @@ auto make_device_compressed_structure_view(raft::device_span<IndptrType> indptr,
   return device_compressed_structure_view<IndptrType, IndicesType, NZType>(indptr, indices, n_cols);
 }
 
+/** @} */
+
 };  // namespace raft

cpp/include/raft/core/host_coo_matrix.hpp

@@ -22,6 +22,11 @@
 
 namespace raft {
 
+/**
+ * \defgroup host_coo_matrix Host COO Matrix
+ * @{
+ */
+
 /**
  * Specialization for a sparsity-preserving coordinate structure view which uses host memory
  */
@@ -386,4 +391,6 @@ auto make_host_coordinate_structure_view(raft::host_span<RowType> rows,
   return host_coordinate_structure_view<RowType, ColType, NZType>(rows, cols, n_rows, n_cols);
 }
 
+/** @} */
+
 };  // namespace raft

cpp/include/raft/core/host_csr_matrix.hpp

@@ -24,6 +24,11 @@
 
 namespace raft {
 
+/**
+ * \defgroup host_csr_matrix Host CSR Matrix
+ * @{
+ */
+
 /**
  * Specialization for a sparsity-preserving compressed structure view which uses host memory
  */
@@ -415,4 +420,6 @@ auto make_host_compressed_structure_view(raft::host_span<IndptrType> indptr,
   return host_compressed_structure_view<IndptrType, IndicesType, NZType>(indptr, indices, n_cols);
 }
 
+/** @} */
+
 };  // namespace raft

cpp/include/raft/core/host_span.hpp

@@ -21,7 +21,7 @@
 namespace raft {
 
 /**
- * @defgroup device_span one-dimensional device span type
+ * @defgroup host_span one-dimensional device span type
  * @{
  */
 

cpp/include/raft/core/interruptible.hpp

@@ -303,7 +303,7 @@ class interruptible {
 };
 
 /**
- * @}
+ * @} // end doxygen group interruptible
  */
 
 }  // namespace raft

cpp/include/raft/core/math.hpp

@@ -30,7 +30,7 @@
 namespace raft {
 
 /**
- * @defgroup Absolute Absolute value
+ * @defgroup math_functions Mathematical Functions
  * @{
  */
 template <typename T>
@@ -55,12 +55,7 @@ constexpr RAFT_INLINE_FUNCTION auto abs(T x)
 {
   return x < T{0} ? -x : x;
 }
-/** @} */
 
-/**
- * @defgroup Trigonometry Trigonometry functions
- * @{
- */
 /** Inverse cosine */
 template <typename T>
 RAFT_INLINE_FUNCTION auto acos(T x)
@@ -199,12 +194,7 @@ RAFT_INLINE_FUNCTION auto tanh(T x)
   return std::tanh(x);
 #endif
 }
-/** @} */
 
-/**
- * @defgroup Exponential Exponential and logarithm
- * @{
- */
 /** Exponential function */
 template <typename T,
           std::enable_if_t<CUDA_CONDITION_ELSE_TRUE(((!std::is_same_v<T, __half> &&
@@ -286,12 +276,9 @@ log(T x)
 #endif
 }
 #endif
-/** @} */
 
 /**
- * @defgroup Maximum Maximum of two or more values.
- *
- * The CUDA Math API has overloads for all combinations of float/double. We provide similar
+ * @brief The CUDA Math API has overloads for all combinations of float/double. We provide similar
  * functionality while wrapping around std::max, which only supports arguments of the same type.
  * However, though the CUDA Math API supports combinations of unsigned and signed integers, this is
  * very error-prone so we do not support that and require the user to cast instead. (e.g the max of
@@ -408,10 +395,8 @@ max(T x)
 }
 #endif
 
-/** @} */
-
 /**
- * @defgroup Minimum Minimum of two or more values.
+ * @brief Minimum Minimum of two or more values.
  *
  * The CUDA Math API has overloads for all combinations of float/double. We provide similar
  * functionality while wrapping around std::min, which only supports arguments of the same type.
@@ -530,12 +515,7 @@ min(T x)
 #endif
 }
 #endif
-/** @} */
 
-/**
- * @defgroup Power Power and root functions
- * @{
- */
 /** Power */
 template <typename T1, typename T2>
 RAFT_INLINE_FUNCTION auto pow(T1 x, T2 y)
@@ -587,7 +567,6 @@ sqrt(T x)
 #endif
 }
 #endif
-/** @} */
 
 /** Sign */
 template <typename T>
@@ -596,4 +575,6 @@ RAFT_INLINE_FUNCTION auto sgn(T val) -> int
   return (T(0) < val) - (val < T(0));
 }
 
+/** @} */
+
 }  // namespace raft

cpp/include/raft/core/mdarray.hpp

@@ -34,7 +34,7 @@
 namespace raft {
 
 /**
- * @defgroup mdarray multi-dimensional memory-owning type
+ * @defgroup mdarray_apis multi-dimensional memory-owning type
  * @{
  */
 
@@ -343,9 +343,7 @@ class mdarray
   container_type c_;
 };
 
-/**
- * @}
- */
+/** @} */
 
 /**
  * @defgroup mdarray_reshape Row- or Col-norm computation
@@ -387,8 +385,6 @@ auto reshape(const array_interface_type& mda, extents<IndexType, Extents...> new
   return reshape(mda.view(), new_shape);
 }
 
-/**
- * }@
- */
+/** @} */
 
 }  // namespace raft