Status: Experimental
File configuration provides a mechanism for configuring OpenTelemetry which is more expressive and full-featured than the environment variable based scheme, and language agnostic in a way not possible with programmatic configuration.
File configuration defines a Configuration Model, which can be expressed in a configuration file. Configuration files can be validated against the Configuration Schema, and interpreted to produce configured OpenTelemetry components.
The configuration model is defined in opentelemetry-configuration using the JSON Schema.
TODO: define stability guarantees and backwards compatibility
A configuration file is a serialized file-based representation of the Configuration Model.
Configuration files SHOULD use one the following serialization formats:
YAML configuration files SHOULD follow YAML spec revision >= 1.2.
YAML configuration files MUST use file extensions .yaml
or .yml
.
TODO: decide if JSON file format is required
Configuration files support environment variables substitution for references which match the following regular expression:
\$\{(?<ENV_NAME>[a-zA-Z_][a-zA-Z0-9_]*)}
The ENV_NAME
MUST start with an alphabetic or _
character, and is followed
by 0 or more alphanumeric or _
characters.
For example, ${API_KEY}
is valid, while ${1API_KEY}
and ${API_$KEY}
are
invalid.
Environment variable substitution MUST only apply to scalar values. Mapping keys are not candidates for substitution.
If a referenced environment variable is not defined, it MUST be replaced with an empty value.
Node types MUST be interpreted after environment variable substitution takes place. This ensures the environment string representation of boolean, integer, or floating point fields can be properly converted to expected types.
It MUST NOT be possible to inject YAML structures by environment variables. For
example, references to INVALID_MAP_VALUE
environment variable below.
For example, consider the following environment variables, and YAML configuration file:
export STRING_VALUE="value"
export BOOl_VALUE="true"
export INT_VALUE="1"
export FLOAT_VALUE="1.1"
export INVALID_MAP_VALUE"value\nkey:value" # An invalid attempt to inject a map key into the YAML
string_key: ${STRING_VALUE} # Valid reference to STRING_VALUE
other_string_key: "${STRING_VALUE}" # Valid reference to STRING_VALUE inside double quotes
another_string_key: "${BOOl_VALUE}" # Valid reference to BOOl_VALUE inside double quotes
yet_another_string_key: ${INVALID_MAP_VALUE} # Valid reference to INVALID_MAP_VALUE, but YAML structure from INVALID_MAP_VALUE MUST NOT be injected
bool_key: ${BOOl_VALUE} # Valid reference to BOOl_VALUE
int_key: ${INT_VALUE} # Valid reference to INT_VALUE
float_key: ${FLOAT_VALUE} # Valid reference to FLOAT_VALUE
combo_string_key: foo ${STRING_VALUE} ${FLOAT_VALUE} # Valid reference to STRING_VALUE and FLOAT_VALUE
undefined_key: ${UNDEFINED_KEY} # Invalid reference, UNDEFINED_KEY is not defined and is replaced with ""
${STRING_VALUE}: value # Invalid reference, substitution is not valid in mapping keys and reference is ignored
Environment variable substitution results in the following YAML:
string_key: value # Interpreted as type string, tag URI tag:yaml.org,2002:str
other_string_key: "value" # Interpreted as type string, tag URI tag:yaml.org,2002:str
another_string_key: "true" # Interpreted as type string, tag URI tag:yaml.org,2002:str
yet_another_string_key: "value\nkey:value" # Interpreted as type string, tag URI tag:yaml.org,2002:str
bool_key: true # Interpreted as type bool, tag URI tag:yaml.org,2002:bool
int_key: 1 # Interpreted as type int, tag URI tag:yaml.org,2002:int
float_key: 1.1 # Interpreted as type float, tag URI tag:yaml.org,2002:float
combo_string_key: foo value 1.1 # Interpreted as type string, tag URI tag:yaml.org,2002:str
undefined_key: # Interpreted as type null, tag URI tag:yaml.org,2002:null
${STRING_VALUE}: value # Interpreted as type string, tag URI tag:yaml.org,2002:str
SDK configuration defines the interfaces and operations that SDKs are expected to expose to enable file based configuration.
SDKs SHOULD provide an in-memory representation of
the Configuration Model. In general, SDKs are encouraged
to provide this in-memory representation in a manner that is idiomatic for their
language. If an SDK needs to expose a class or interface, the
name Configuration
is RECOMMENDED.
The SDK supports a variety of extension plugin interfaces, allowing users and libraries to customize behaviors including the sampling, processing, and exporting of data. In general, the configuration model defines specific types for built-in implementations of these plugin interfaces. For example, the BatchSpanProcessor type refers to the built-in Batching span processor. The schema SHOULD also support the ability to specify custom implementations of plugin interfaces defined by libraries or users.
For example, a custom span exporter might be configured as follows:
tracer_provider:
processors:
- batch:
exporter:
my-exporter:
config-parameter: value
Here we specify that the tracer provider has a batch span processor
paired with a custom span exporter named my-exporter
, which is configured
with config-parameter: value
. For this configuration to succeed,
a component provider must
be registered with type: SpanExporter
,
and name: my-exporter
. When parse is called, the implementation will
encounter my-exporter
and translate the corresponding configuration to an
equivalent generic properties
representation (
i.e. properties: {config-parameter: value}
). When create is called,
the implementation will encounter my-exporter
and
invoke create plugin on the registered component provider
with the configuration properties
determined during parse
.
Given the inherent differences across languages, the details of extension component mechanisms are likely to vary to a greater degree than is the case with other APIs defined by OpenTelemetry. This is to be expected and is acceptable so long as the implementation results in the defined behaviors.
A component provider is responsible for interpreting configuration and returning an implementation of a particular type of SDK extension plugin interface.
Component providers are registered with an SDK implementation of configuration via register. This MAY be done automatically or require manual intervention by the user based on what is possible and idiomatic in the language ecosystem. For example in Java, component providers might be registered automatically using the service provider interface (SPI) mechanism.
See create, which details component provider usage in file configuration interpretation.
Interpret configuration to create a instance of a SDK extension plugin interface.
Parameters:
properties
- The configuration properties. Properties MUST fully represent the configuration as specified in the configuration file, including the ability to access scalars, mappings, and sequences (of scalars and other structures). It MUST be possible to determine if a particular property is present. It SHOULD be possible to access properties in a type safe manner, based on what is idiomatic in the language.
Returns: A configured SDK extension plugin interface implementation.
The plugin interface MAY have properties which are optional or required, and have specific requirements around type or format. The set of properties a component provider accepts, along with their requirement level and expected type, comprise a configuration schema. A component provider SHOULD document its configuration schema.
When Create Plugin is invoked, the component provider interprets properties
and attempts to extract data according to its configuration schema. If this
fails (e.g. a required property is not present, a type is mismatches, etc.),
Create Plugin SHOULD return an error.
SDK implementations of configuration MUST provide the following operations.
Note: Because these operations are stateless pure functions, they are not defined as part of any type, class, interface, etc. SDKs may organize them in whatever manner is idiomatic for the language.
TODO: Add operation to update SDK components with new configuration for usage with OpAmp
Parse and validate a configuration file.
Parameters:
file
: The configuration file to parse. This MAY be a file path, or language specific file data structure, or a stream of a file's content.file_format
: The file format of thefile
(e.g. yaml). Implementations MAY accept afile_format
parameter, or infer it from thefile
extension, or include file format specific overloads ofparse
, e.g.parseYaml(file)
. Ifparse
acceptsfile_format
, the API SHOULD be structured so a user is obligated to provide it.
Returns: configuration model
Parse MUST perform environment variable substitution.
Parse MUST interpret null as equivalent to unset.
When encountering a reference to
a SDK extension component which is not built in to
the SDK, Parse MUST resolve corresponding configuration to a
generic properties
representation as described
in Create Plugin.
Parse SHOULD return an error if:
- The
file
doesn't exist or is invalid - The parsed
file
content does not conform to the configuration model schema.
Interpret configuration model and return SDK components.
Parameters:
configuration
- The configuration model.
Returns: Top level SDK components:
TracerProvider
MeterProvider
LoggerProvider
Propagators
The multiple responses MAY be returned using a tuple, or some other data structure encapsulating the components.
If a field is null or unset and a default value is defined, Create MUST ensure
the SDK component is configured with the default value. If a field is null or
unset and no default value is defined, Create SHOULD return an error. For
example, if configuring
the span batching processor and
the scheduleDelayMillis
field is null or unset, the component is configured
with the default value of 5000
. However, if the exporter
field is null or
unset, Create fails fast since there is no default value for exporter
.
When encountering a reference to
a SDK extension component which is not built in to
the SDK, Create MUST resolve the component using Create Plugin
of the component provider of the corresponding type
and name
used to register, including the
configuration properties
as an argument. If no component provider is
registered with the type
and name
, Create SHOULD return an error.
If Create Plugin returns an error, Create SHOULD propagate the
error.
This SHOULD return an error if it encounters an error in configuration
(i.e.
fail fast) in accordance with
initialization error handling principles.
TODO: define behavior if some portion of configuration model is not supported
The file configuration implementation MUST provide a mechanism to register component providers.
Parameters:
component_provider
- The component provider.type
- The type of plugin interface it provides (e.g. SpanExporter, Sampler, etc).name
- The name used to identify the type of component. This is used in configuration files to specify that the correspondingcomponent_provider
is to provide the component.
The type
and name
comprise a unique key. Register MUST return an error if it
is called multiple times with the same type
and name
combination.
- Configuration proposal (OTEP #225)