diff --git a/docs/cudf/source/io-gds-integration.rst b/docs/cudf/source/io-gds-integration.rst new file mode 100644 index 00000000000..9ccf773b2e4 --- /dev/null +++ b/docs/cudf/source/io-gds-integration.rst @@ -0,0 +1,22 @@ +GPUDirect Storage Integration +============================= + +Many IO APIs can use GPUDirect Storage (GDS) library to optimize IO operations. +GDS enables a direct data path for direct memory access (DMA) transfers between GPU memory and storage, which avoids a bounce buffer through the CPU. +GDS also has a compatibility mode that allows the library to fall back to copying through a CPU bounce buffer. +The SDK is available for download `here `_. + +Use of GPUDirect Storage in cuDF is disabled by default, and can be enabled through environment variable ``LIBCUDF_CUFILE_POLICY``. +This variable also controls the GDS compatibility mode. There are two special values for the environment variable: + +- "GDS": Use of GDS is enabled; GDS compatibility mode is *off*. +- "ALWAYS": Use of GDS is enabled; GDS compatibility mode is *on*. + +Any other value (or no value set) will keep the GDS disabled for use in cuDF and IO will be done using cuDF's CPU bounce buffers. + +This environment variable also affects how cuDF treats GDS errors. +When ``LIBCUDF_CUFILE_POLICY`` is set to "GDS" and a GDS API call fails for any reason, cuDF falls back to the internal implementation with bounce buffers. +When ``LIBCUDF_CUFILE_POLICY`` is set to "ALWAYS" and a GDS API call fails for any reason (unlikely, given that the compatibility mode is on), +cuDF throws an exception to propagate the error to te user. + +NOTE: current GDS integration is not fully optimized and enabling GDS will not lead to performance improvements in all cases. \ No newline at end of file diff --git a/docs/cudf/source/io.rst b/docs/cudf/source/io.rst index 5186473ae10..e88162d8f52 100644 --- a/docs/cudf/source/io.rst +++ b/docs/cudf/source/io.rst @@ -8,4 +8,5 @@ This page contains Input / Output related APIs in cuDF. :maxdepth: 2 :caption: Contents: - io-supported-types.rst \ No newline at end of file + io-supported-types.rst + io-gds-integration.rst \ No newline at end of file