Skip to content

Commit

Permalink
cudf.pandas documentation improvement (#15948)
Browse files Browse the repository at this point in the history
Added some more about the generality of the fast-slow proxy scheme from a suggestion from @wence-

Authors:
  - Matthew Murray (https://github.com/Matt711)

Approvers:
  - Matthew Roeschke (https://github.com/mroeschke)

URL: #15948
  • Loading branch information
Matt711 authored Jun 7, 2024
1 parent 39c5b86 commit 0067444
Showing 1 changed file with 10 additions and 2 deletions.
12 changes: 10 additions & 2 deletions docs/cudf/source/developer_guide/cudf_pandas.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,16 @@ The use of the cuDF pandas accelerator mode (`cudf.pandas`) is explained [in the
The purpose of this document is to explain how the fast-slow proxy mechanism works and document internal environment variables that can be used to debug `cudf.pandas` itself.

## fast-slow proxy mechanism
`cudf.pandas` works by wrapping each Pandas type and its corresponding cuDF type in a new proxy type also known as a fast-slow proxy type.
The purpose of proxy types is to attempt computations on the fast (cuDF) object first, and then fall back to running on the slow (Pandas) object if the fast version fails.
The core of `cudf.pandas` is implemented through proxy types defined in [`fast_slow_proxy.py`](https://github.com/rapidsai/cudf/blob/5f45803b2a68b49d330d94e2f701791a7590612a/python/cudf/cudf/pandas/fast_slow_proxy.py), which link a pair of "fast" and "slow" libraries.
`cudf.pandas` works by wrapping each "slow" type and its corresponding "fast" type in a new proxy type, also known as a fast-slow proxy type.
The purpose of these proxy types is so we can first attempt computations on the fast object, and then fall back to the slow object if the fast version fails.
While the core wrapping functionality is generic, the current usage mainly involves providing a proxy pair using cuDF and Pandas.
In the rest of this document, to maintain a concrete pair of libraries in mind, we use cuDF and Pandas interchangeably as names for the "fast" and "slow" libraries, respectively, with the understanding that any pair of API-matching libraries could be used.
For example, future support could include pairs such as CuPy (as the "fast" library) and NumPy (as the "slow" library).

```{note}
We currently do not wrap the entire NumPy library because it exposes a C API. But we do wrap NumPy's `numpy.ndarray` and CuPy's `cupy.ndarray` in a proxy type.
```

### Types:
#### Wrapped Types and Proxy Types
Expand Down

0 comments on commit 0067444

Please sign in to comment.