Generate unified Python/C++ docs

ci/build_docs.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# Copyright (c) 2023, NVIDIA CORPORATION.
+# Copyright (c) 2023-2024, NVIDIA CORPORATION.
 
 set -euo pipefail
 
@@ -40,8 +40,8 @@ popd
 
 rapids-logger "Build Python docs"
 pushd docs/cudf
-make dirhtml
-make text
+make dirhtml O="-j 4"
+make text O="-j 4"
 mkdir -p "${RAPIDS_DOCS_DIR}/cudf/"{html,txt}
 mv build/dirhtml/* "${RAPIDS_DOCS_DIR}/cudf/html"
 mv build/text/* "${RAPIDS_DOCS_DIR}/cudf/txt"

conda/environments/all_cuda-118_arch-x86_64.yaml

@@ -12,6 +12,7 @@ dependencies:
 - benchmark==1.8.0
 - boto3>=1.21.21
 - botocore>=1.24.21
+- breathe>=4.35.0
 - c-compiler
 - cachetools
 - clang-tools=16.0.6

conda/environments/all_cuda-120_arch-x86_64.yaml

@@ -12,6 +12,7 @@ dependencies:
 - benchmark==1.8.0
 - boto3>=1.21.21
 - botocore>=1.24.21
+- breathe>=4.35.0
 - c-compiler
 - cachetools
 - clang-tools=16.0.6

cpp/doxygen/developer_guide/BENCHMARKING.md

@@ -23,7 +23,7 @@ benchmarks in `cpp/benchmarks/copying/scatter.cu`.
 In the interest of improving compile time, whenever possible, test source files should be `.cpp`
 files because `nvcc` is slower than `gcc` in compiling host code. Note that `thrust::device_vector`
 includes device code, and so must only be used in `.cu` files. `rmm::device_uvector`,
-`rmm::device_buffer` and the various `column_wrapper` types described in [Testing](TESTING.md)
+`rmm::device_buffer` and the various `column_wrapper` types described in [Testing](testing)
 can be used in `.cpp` files, and are therefore preferred in test code over `thrust::device_vector`.
 
 ## CUDA Asynchrony and benchmark accuracy
@@ -37,7 +37,7 @@ performance in repeated iterations.
 
 ## Data generation
 
-For generating benchmark input data, helper functions are available at [cpp/benchmarks/common/generate_input.hpp](/cpp/benchmarks/common/generate_input.hpp). The input data generation happens on device, in contrast to any `column_wrapper` where data generation happens on the host.
+For generating benchmark input data, helper functions are available at ``cpp/benchmarks/common/generate_input.hpp``. The input data generation happens on device, in contrast to any `column_wrapper` where data generation happens on the host.
 * `create_sequence_table` can generate sequence columns starting with value 0 in first row and increasing by 1 in subsequent rows.
 * `create_random_column` can generate a column filled with random data. The random data parameters are configurable.
 * `create_random_table` can generate a table of columns filled with random data. The random data parameters are configurable.

cpp/doxygen/developer_guide/DEVELOPER_GUIDE.md

@@ -1,11 +1,6 @@
 # libcudf C++ Developer Guide {#DEVELOPER_GUIDE}
 
-This document serves as a guide for contributors to libcudf C++ code. Developers should also refer
-to these additional files for further documentation of libcudf best practices.
-
-* [Documentation Guide](DOCUMENTATION.md) for guidelines on documenting libcudf code.
-* [Testing Guide](TESTING.md) for guidelines on writing unit tests.
-* [Benchmarking Guide](BENCHMARKING.md) for guidelines on writing unit benchmarks.
+This document serves as a guide for contributors to libcudf C++ code.
 
 # Overview
 
@@ -135,7 +130,7 @@ Additional style guidelines for libcudf code include:
    `1'234'567`. Hexadecimal values should use separators every 4 characters,
    like `0x0123'ABCD`.
 
-Documentation is discussed in the [Documentation Guide](DOCUMENTATION.md).
+Documentation is discussed in the [Documentation Guide](documentation).
 
 ### Includes
 
@@ -434,7 +429,7 @@ inputs. The types of those exceptions (e.g. `cudf::logic_error`) are part of the
 However, the explanatory string returned by the `what` method of those exceptions is not part of the
 API and is subject to change. Calling code should not rely on the contents of libcudf error
 messages to determine the nature of the error. For information on the types of exceptions that
-libcudf throws under different circumstances, see the [section on error handling](#errors).
+libcudf throws under different circumstances, see the [section on error handling](#error-handling).
 
 # libcudf API and Implementation
 
@@ -894,6 +889,7 @@ the "breaking" tag. This ensures that the "Breaking" section of the release note
 description of what has broken from the past release. Label pull requests that contain deprecations
 with the "non-breaking" tag.
 
+(error-handling)=
 
 # Error Handling {#errors}
 
@@ -1342,6 +1338,8 @@ within the string.
 
 `cudf::type_dispatcher` dispatches to the `string_view` data type when invoked on a `STRING` column.
 
+(utf-8)=
+
 #### UTF-8
 
 The libcudf strings column only supports UTF-8 encoding for strings data.

cpp/doxygen/developer_guide/DOCUMENTATION.md

@@ -40,7 +40,7 @@ Doxygen recognizes and parses block comments and performs specialized output for
 There are almost 200 commands (also called tags in this document) that doxygen recognizes in comment blocks.
 This document provides guidance on which commands/tags to use and how to use them in the libcudf C++ source code.
 
-The doxygen process can be customized using options in the [Doxyfile](../doxygen/Doxyfile).
+The doxygen process can be customized using options in the Doxyfile.
 
 Here are some of the custom options in the Doxyfile for libcudf.
 | Option | Setting | Description |
@@ -234,10 +234,10 @@ The following tags should appear near the end of function comment block in the o
 
 | Command | Description |
 | ------- | ----------- |
-| [\@throw](#throw) | Specify the conditions in which the function may throw an exception |
-| [\@tparam](#tparam) | Description for each template parameter |
-| [\@param](#param) | Description for each function parameter |
-| [\@return](#return) | Short description of object or value returned |
+| \@throw | Specify the conditions in which the function may throw an exception |
+| \@tparam | Description for each template parameter |
+| \@param | Description for each function parameter |
+| \@return | Short description of object or value returned |
 
 #### \@throw
 
@@ -293,6 +293,8 @@ Include a brief description of what is returned.
 
 Do not include the type of the object returned with the `@return` comment.
 
+(inline-examples)=
+
 ### Inline Examples
 
 It is usually helpful to include a source code example inside your comment block when documenting a function or other declaration.
@@ -378,12 +380,12 @@ The doxygen output includes a _Modules_ page that organizes items into groups sp
 These commands can group common functions across header files, source files, and even namespaces.
 Groups can also be nested by defining new groups within existing groups.
 
-For libcudf, all the group hierarchy is defined in the [doxygen_groups.h](../include/doxygen_groups.h) header file.
-The [doxygen_groups.h](../include/doxygen_groups.h) file does not need to be included in any other source file, because the definitions in this file are used only by the doxygen tool to generate groups in the _Modules_ page.
+For libcudf, all the group hierarchy is defined in the ``cpp/include/doxygen_groups.h`` header file.
+The doxygen_groups.h file does not need to be included in any other source file, because the definitions in this file are used only by the doxygen tool to generate groups in the _Modules_ page.
 Modify this file only to add or update groups.
 The existing groups have been carefully structured and named, so new groups should be added thoughtfully.
 
-When creating a new API, specify its group using the [\@ingroup](https://www.doxygen.nl/manual/commands.html#cmdingroup) tag and the group reference id from the [doxygen_groups.h](../include/doxygen_groups.h) file.
+When creating a new API, specify its group using the [\@ingroup](https://www.doxygen.nl/manual/commands.html#cmdingroup) tag and the group reference id from the doxygen_groups.h file.
 
     namespace cudf {
 
@@ -426,7 +428,7 @@ So include the `@addtogroup` and `@{ ... @}` between the namespace declaration b
 Summary of groups tags
 | Tag/Command | Where to use |
 | ----------- | ------------ |
-| `@defgroup` | For use only in [doxygen_groups.h](../include/doxygen_groups.h) and should include the group's title. |
+| `@defgroup` | For use only in doxygen_groups.h and should include the group's title. |
 | `@ingroup` | Use inside individual doxygen block comments for declaration statements in a header file. |
 | `@addtogroup` | Use instead of `@ingroup` for multiple declarations in the same file within a namespace declaration. Do not specify a group title. |
 | `@{ ... @}` |  Use only with `@addtogroup`. |

cpp/include/cudf/io/json.hpp

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2020-2023, NVIDIA CORPORATION.
+ * Copyright (c) 2020-2024, NVIDIA CORPORATION.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -72,8 +72,10 @@ enum class json_recovery_mode_t {
  *
  * Parameters in PANDAS that are unavailable or in cudf:
  *
+ *
+ * +----------------------+--------------------------------------------------+
  * | Name                 | Description                                      |
- * | -------------------- | ------------------------------------------------ |
+ * +======================+==================================================+
  * | `orient`             | currently fixed-format                           |
  * | `typ`                | data is always returned as a cudf::table         |
  * | `convert_axes`       | use column functions for axes operations instead |
@@ -84,6 +86,7 @@ enum class json_recovery_mode_t {
  * | `date_unit`          | only millisecond units are supported             |
  * | `encoding`           | only ASCII-encoded data is supported             |
  * | `chunksize`          | use `byte_range_xxx` for chunking instead        |
+ * +----------------------+--------------------------------------------------+
  */
 class json_reader_options {
   source_info _source;