diff --git a/docs/developer/meta/config-models.md b/docs/developer/meta/config-models.md index 9984937a8f9ea..affae1cc79672 100644 --- a/docs/developer/meta/config-models.md +++ b/docs/developer/meta/config-models.md @@ -2,9 +2,9 @@ ----- -All integrations use [pydantic](https://github.com/samuelcolvin/pydantic) models as the primary way to validate and interface with configuration. +All integrations use [pydantic](https://github.com/pydantic/pydantic) models as the primary way to validate and interface with configuration. -As config spec data types are based on OpenAPI 3, we [automatically generate](https://github.com/koxudaxi/datamodel-code-generator) the necessary code. +As [config spec](config-specs.md) data types are based on OpenAPI 3, we [automatically generate](https://github.com/koxudaxi/datamodel-code-generator) the necessary code. The models reside in a package named `config_models` located at the root of a check's namespaced package. For example, a new integration named `foo`: @@ -26,25 +26,22 @@ foo There are 2 possible models: -- `SharedConfig` (ID: `shared`) that corresponds to the `init_config` section - `InstanceConfig` (ID: `instance`) that corresponds to a check's entry in the `instances` section +- `SharedConfig` (ID: `shared`) that corresponds to the `init_config` section that is shared by all instances All models are defined in `<ID>.py` and are available for import directly under `config_models`. ## Default values The default values for optional settings are populated in `defaults.py` and are derived from the -[value](../meta/config-specs.md#values) property of config spec options. - -The precedence is: - -1. the `default` key -2. the `example` key, if it appears to represent a real value rather than an illustrative example and the `type` is a primitive -3. the default value of the `type` e.g. `string` -> `str()`, `object` -> `dict()`, etc. +[value](config-specs.md#values) property of config spec options. The precedence is the `default` key +followed by the `example` key (if it appears to represent a real value rather than an illustrative example +and the `type` is a primitive). In all other cases, the default is `None`, which means there is no default +getter function. ## Validation -The validation of fields for every model occurs in 6 stages. +The validation of fields for every model occurs in three high-level stages, as described in this section. ### Initial @@ -56,46 +53,55 @@ def initialize_<ID>(values: dict[str, Any], **kwargs) -> dict[str, Any]: If such a validator exists in `validators.py`, then it is called once with the raw config that was supplied by the user. The returned mapping is used as the input config for the subsequent stages. -### Default value population +### Field -If a field was not supplied by the user nor during the initialization stage, then its default value is -taken from `defaults.py`. This stage is skipped for required fields. +The value of each field goes through the following steps. -### Default field validators +#### Default value population -At this point `pydantic` will parse the values and perform validation of types, etc. +If a field was not supplied by the user nor during the [initialization stage](#initial), then its default value is +taken from `defaults.py`. This stage is skipped for required fields. -### Custom field validators +#### Custom field validators The contents of `validators.py` are entirely custom and contain functions to perform extra validation if necessary. ```python -def <ID>_<OPTION_NAME>(value: Any, *, field: pydantic.fields.ModelField, **kwargs) -> Any: +def <ID>_<OPTION_NAME>(value: Any, *, field: pydantic.fields.FieldInfo, **kwargs) -> Any: ... ``` -Such validators are called for the appropriate field of the proper model if the option was supplied by the user. +Such validators are called for the appropriate field of the proper model. The returned value is used as the +new value of the option for the subsequent stages. -The returned value is used as the new value of the option for the subsequent stages. +!!! note + This only occurs if the option was supplied by the user. -### Pre-defined field validators +#### Pre-defined field validators -A new `validators` key under the [value](https://datadoghq.dev/integrations-core/meta/config-specs/#values) property of config -spec options is considered. Every entry will refer to a relative import path to a [field validator](#custom-field-validators) +A `validators` key under the [value](https://datadoghq.dev/integrations-core/meta/config-specs/#values) property of config +spec options is considered. Every entry refers to a relative import path to a [field validator](#custom-field-validators) under `datadog_checks.base.utils.models.validation` and is executed in the defined order. -The last returned value is used as the new value of the option for the [final](#final) stage. +!!! note + This only occurs if the option was supplied by the user. + +#### Conversion to immutable types + +Every `list` is converted to `tuple` and every `dict` is converted to [types.MappingProxyType](https://docs.python.org/3/library/types.html#types.MappingProxyType). + +!!! note + A field or nested field would only be a `dict` when it is defined as a mapping with arbitrary keys. Otherwise, it would be a model with its own properties as usual. ### Final ```python -def finalize_<ID>(values: dict[str, Any], **kwargs) -> dict[str, Any]: +def check_<ID>(model: pydantic.BaseModel) -> pydantic.BaseModel: ... ``` -If such a validator exists in `validators.py`, then it is called with the cumulative result of all fields. - -The returned mapping is used to instantiate the model. +If such a validator exists in `validators.py`, then it is called with the final constructed model. At this point, it cannot +be mutated, so you can only raise errors. ## Loading @@ -116,12 +122,11 @@ class Check(AgentCheck, ConfigMixin): ... ``` -It exposes the instantiated `InstanceConfig` model at `self.config` and `SharedConfig` model at `self.shared_config`. +It exposes the instantiated `InstanceConfig` model as `self.config` and `SharedConfig` model as `self.shared_config`. ## Immutability -All generated models are [configured as immutable](https://pydantic-docs.helpmanual.io/usage/models/#faux-immutability). -Additionally, every `list` is converted to `tuple` and every `dict` is converted to [immutables.Map](https://github.com/MagicStack/immutables). +In addition to each field being [converted to an immutable type](#conversion-to-immutable-types), all generated models are [configured as immutable](https://docs.pydantic.dev/2.0/usage/models/#faux-immutability). ## Deprecation @@ -129,4 +134,4 @@ Every option marked as deprecated in the config spec will log a warning with inf ## Enforcement -A validation command `ddev validate models` runs in our CI. To locally generate the proper files, run `ddev validate models [CHECK] --sync`. +A validation command [`validate models`](../ddev/cli.md#ddev-validate-models) runs in our CI. To locally generate the proper files, run `ddev validate models [INTEGRATION] --sync`.