Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cudf.options #11193

Merged
merged 18 commits into from
Jul 28, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/cudf/source/api_docs/config.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
.. _api.config:

============
CUDF Configs
isVoid marked this conversation as resolved.
Show resolved Hide resolved
============

.. autosummary::
:toctree: api/

cudf.register_config
cudf.get_config
cudf.set_config
cudf.describe_config
cudf.describe_configs
1 change: 1 addition & 0 deletions docs/cudf/source/api_docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,4 @@ This page provides a list of all publicly accessible modules, methods and classe
io
subword_tokenize
string_handling
config
20 changes: 20 additions & 0 deletions docs/cudf/source/developer_guide/config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# CUDF Configuration

Configurations are stored as a dictionary in `cudf.config` module.
Each configuration name is also its key in the dictionary.
The value of the configuration is an instance of a `CUDFConfiguration` object.

A `CUDFConfiguration` object inherits from `dataclass` and consists 4 attributes:
isVoid marked this conversation as resolved.
Show resolved Hide resolved
- `name`: the name and the key of the configuration
- `value`: the current value of the configuration
- `description`: a text description of the configuration
- `validator`: a boolean function that returns `True` if value is valid,
`False` otherwise.

Developers can use `cudf.register_config` to add configurations to the registry.
`cudf.get_config` is provided to get config value from the registry.

When testing the behavior of certain configuration,
it is advised to use [yield fixture](https://docs.pytest.org/en/7.1.x/how-to/fixtures.html#yield-fixtures-recommended) to setup and cleanup certain configuration for the test.

See [API reference](api.config) for detail.
1 change: 1 addition & 0 deletions docs/cudf/source/developer_guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@
:maxdepth: 2

library_design
config
isVoid marked this conversation as resolved.
Show resolved Hide resolved
8 changes: 8 additions & 0 deletions docs/cudf/source/user_guide/config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# CUDF Configuration

CUDF configurations are a set of key value pairs stored in a global dictionary.

User may get the full list of cudf configurations with ``cudf.describe_configs``.
To set value to a configuration, use ``cudf.set_config``.

See [API reference](api.config) for detail.
1 change: 1 addition & 0 deletions docs/cudf/source/user_guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,6 @@ groupby
guide-to-udfs
cupy-interop
dask-cudf
config
PandasCompat
```
8 changes: 8 additions & 0 deletions python/cudf/cudf/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,14 @@
from cudf.utils.dtypes import _NA_REP
from cudf.utils.utils import set_allocator

from .config import (
register_config,
get_config,
set_config,
describe_config,
describe_configs,
)

try:
from ptxcompiler.patch import patch_numba_codegen_if_needed
except ImportError:
Expand Down
100 changes: 100 additions & 0 deletions python/cudf/cudf/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Copyright (c) 2022, NVIDIA CORPORATION.

from dataclasses import dataclass
from typing import Any, Callable, Dict


@dataclass
class CUDFConfiguration:
name: str
description: str
value: Any
validator: Callable


_CUDF_CONFIG: Dict[str, CUDFConfiguration] = {}


def register_config(
isVoid marked this conversation as resolved.
Show resolved Hide resolved
name: str, default_value: Any, description: str, validator: Callable
):
"""Add a registry to the configuration dictionary.
isVoid marked this conversation as resolved.
Show resolved Hide resolved

Parameters
----------
name : str
The name of the configuration. Also used as the key in the dictionary.
isVoid marked this conversation as resolved.
Show resolved Hide resolved

default_value : Any
The default value of the configuration.

description : str
A text description of the configuration.

validator : Callable
A function that returns ``True`` is a given value is valid for the
isVoid marked this conversation as resolved.
Show resolved Hide resolved
configuration, ``False`` otherwise.
"""
if not validator(default_value):
raise ValueError(f"Invalid default value: {default_value}")

_CUDF_CONFIG[name] = CUDFConfiguration(
name, default_value, description, validator
)


def get_config(key: str) -> Any:
"""Get the value of configuration.

Parameters
----------
key : str
The name of the configuration.

Returns
-------
The value of the configuration.
"""
return _CUDF_CONFIG[key].value


def set_config(key: str, val: Any):
"""Set the value of configuration.

Raises ``ValueError`` if val is invalid to the configuration.
isVoid marked this conversation as resolved.
Show resolved Hide resolved

Parameters
----------
key : str
The name of the configuration.
val : Any
The value to set.
"""
config = _CUDF_CONFIG[key]
if not config.validator(val):
raise ValueError(f"Invalid configuration {val}")
config.value = val


def describe_config(key: str) -> str:
"""Returns the description of the configuration.

Parameters
----------
key : str
The name of the configuration.
"""
return _CUDF_CONFIG[key].description


def describe_configs() -> Dict[str, str]:
isVoid marked this conversation as resolved.
Show resolved Hide resolved
"""Returns all descriptions available in cudf.

Returns
-------
descriptions : Dict[str, str]
A dictionary mapping the name of the config to their descriptions.
"""
return {
config.name: config.description for config in _CUDF_CONFIG.values()
}
48 changes: 48 additions & 0 deletions python/cudf/cudf/tests/test_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Copyright (c) 2022, NVIDIA CORPORATION.

import pytest

import cudf


@pytest.fixture(scope="module")
def configuration_demo():
isVoid marked this conversation as resolved.
Show resolved Hide resolved
cudf.register_config(
"odd_config",
1,
"An odd configuration.",
lambda x: x % 2 == 1,
)
yield
cudf.config._CUDF_CONFIG.pop("odd_config")


@pytest.fixture(scope="module")
def configuration_demo2():
cudf.register_config(
"even_config", 0, "An even configuration.", lambda x: x % 2 == 0
)
yield
cudf.config._CUDF_CONFIG.pop("even_config")


def test_config_get_set(configuration_demo):
assert cudf.get_config("odd_config") == 1
cudf.config.set_config("odd_config", 101)
assert cudf.get_config("odd_config") == 101


def test_config_set_invalid(configuration_demo):
with pytest.raises(ValueError, match="Invalid configuration 0"):
cudf.set_config("odd_config", 0)


def test_config_description(configuration_demo):
assert cudf.describe_config("odd_config") == "An odd configuration."


def test_config_description_multi(configuration_demo, configuration_demo2):
assert cudf.describe_configs() == {
"odd_config": "An odd configuration.",
"even_config": "An even configuration.",
}