diff --git a/CHANGELOG.md b/CHANGELOG.md index 1a184f9578a..297a3477160 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,10 @@ release. ### Traces +- Add generalized attribute count and attribute value length limits and relevant + environment variables. + ([#1130](https://github.com/open-telemetry/opentelemetry-specification/pull/1130)) +- Adding environment variables for event and link attribute limits. ([#1751](https://github.com/open-telemetry/opentelemetry-specification/pull/1751)) - Adding SDK configuration for Jaeger remote sampler ([#1791](https://github.com/open-telemetry/opentelemetry-specification/pull/1791)) ### Metrics diff --git a/spec-compliance-matrix.md b/spec-compliance-matrix.md index 191ef1d1d19..2cfdd2d748b 100644 --- a/spec-compliance-matrix.md +++ b/spec-compliance-matrix.md @@ -79,6 +79,7 @@ formats is required. Implementing more than one format is optional. | [IdGenerators](specification/trace/sdk.md#id-generators) | | + | + | | + | + | | | + | + | | + | | [SpanLimits](specification/trace/sdk.md#span-limits) | X | + | + | | + | + | | | | - | | + | | [Built-in `SpanProcessor`s implement `ForceFlush` spec](specification/trace/sdk.md#forceflush-1) | | | | | + | + | | | + | + | | | +| [Attribute Limits](specification/common/common.md#attribute-limits) | X | | | | | | | | | | | | ## Baggage @@ -141,12 +142,15 @@ Note: Support for environment variables is optional. |OTEL_TRACES_EXPORTER | - | + | | + | + | + | | - | - | | | |OTEL_METRICS_EXPORTER | - | + | | + | - | - | | - | - | - | - | |OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT | - | + | | + | + | - | | + | - | | | +|OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT | | | | | | | | | | | | |OTEL_SPAN_EVENT_COUNT_LIMIT | - | + | | + | + | - | | + | - | | | |OTEL_SPAN_LINK_COUNT_LIMIT | - | + | | + | + | - | | + | - | | | |OTEL_EVENT_ATTRIBUTE_COUNT_LIMIT | | | | | | | | | | | | |OTEL_LINK_ATTRIBUTE_COUNT_LIMIT | | | | | | | | | | | | |OTEL_TRACES_SAMPLER | - | + | | + | + | + | | - | - | | | |OTEL_TRACES_SAMPLER_ARG | - | + | | + | + | + | | - | - | | | +|OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT | | | | | | | | | | | | +|OTEL_ATTRIBUTE_COUNT_LIMIT | | | | | | | | | | | | ## Exporters diff --git a/specification/common/common.md b/specification/common/common.md index 1b34c4f9f8c..6cb26c12f99 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -8,6 +8,8 @@ Table of Contents - [Attributes](#attributes) + - [Attribute Limits](#attribute-limits) + - [Exempt Entities](#exempt-entities) @@ -41,3 +43,52 @@ both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). See [Attribute Naming](attribute-naming.md) for naming guidelines. + +### Attribute Limits + +Execution of erroneous code can result in unintended attributes. If there are no +limits placed on attributes, they can quickly exhaust available memory, resulting +in crashes that are difficult to recover from safely. + +By default an SDK SHOULD apply truncation as per the list of +[configurable parameters](#attribute-limits-configuration) below. + +If an SDK provides a way to: + +- set an attribute value length limit such that for each + attribute value: + - if it is a string, if it exceeds that limit (counting any character in it as + 1), SDKs MUST truncate that value, so that its length is at most equal + to the limit, + - if it is an array of strings, then apply the above rule to each of the + values separately, + - otherwise a value MUST NOT be truncated; +- set a limit of unique attribute keys such that: + - for each unique attributes key, addition of which would result in exceeding + the limit, SDK MUST discard that key/value pair. + +There MAY be a log emitted to indicate to the user that an attribute was +truncated or discarded. To prevent excessive logging, the log MUST NOT be +emitted more than once per record on which an attribute is set. + +If the SDK implements the limits above, it MUST provide a way to change these +limits programmatically. Names of the configuration options SHOULD be the same as +in the list below. + +An SDK MAY implement model-specific limits, for example +`SpanAttributeCountLimit`. If both a general and a model-specific limit are +implemented, then the SDK MUST first attempt to use the model-specific limit, if +it isn't set and doesn't have a default, then the SDK MUST attempt to use the +general limit. + + +**Configurable parameters:** + +* `AttributeCountLimit` (Default=128) - Maximum allowed attribute count per record; +* `AttributeValueLengthLimit` (Default=Infinity) - Maximum allowed attribute value length; + +#### Exempt Entities + +Attributes, which belong to Metrics, are exempt from the limits described above +at this time, as discussed in +[Metrics Attribute Limits](../metrics/sdk.md#attribute-limits). diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 7af4f22c6d3..d14a11a43bb 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -23,6 +23,7 @@ Table of Contents * [MeterProvider](#meterprovider) +* [Attribute Limits](#attribute-limits) * [MeasurementProcessor](#measurementprocessor) * [MetricProcessor](#metricprocessor) * [MetricExporter](#metricexporter) @@ -207,6 +208,13 @@ meter_provider .set_exporter(ConsoleExporter()) ``` +## Attribute Limits + +Attributes which belong to Metrics are exempt from the +[common rules of attribute limits](../common/common.md#attribute-limits) at this +time. Attribute truncation or deletion could affect identitity of metric time +series and it requires further analysis. + ## MeasurementProcessor `MeasurementProcessor` is an interface which allows hooks when a diff --git a/specification/sdk-environment-variables.md b/specification/sdk-environment-variables.md index 75cc3d1a265..459c511c401 100644 --- a/specification/sdk-environment-variables.md +++ b/specification/sdk-environment-variables.md @@ -78,19 +78,32 @@ Depending on the value of `OTEL_TRACES_SAMPLER`, `OTEL_TRACES_SAMPLER_ARG` may b | OTEL_BSP_MAX_QUEUE_SIZE | Maximum queue size | 2048 | | | OTEL_BSP_MAX_EXPORT_BATCH_SIZE | Maximum batch size | 512 | Must be less than or equal to OTEL_BSP_MAX_QUEUE_SIZE | -## Span Collection Limits +## Attribute Limits + +SDKs SHOULD only offer environment variables for the types of attributes, for +which that SDK implements truncation mechanism. + +See the SDK [Attribute Limits](common/common.md#attribute-limits) section for the definition of the limits. + +| Name | Description | Default | Notes | +| --------------------------------- | ------------------------------------ | ------- | ----- | +| OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT | Maximum allowed attribute value size | | Empty value is treated as infinity. Non-integer and negative values are invalid. | +| OTEL_ATTRIBUTE_COUNT_LIMIT | Maximum allowed span attribute count | 128 | | + +## Span Limits **Status**: [Stable](document-status.md) See the SDK [Span Limits](trace/sdk.md#span-limits) section for the definition of the limits. -| Name | Description | Default | Notes | -| -------------------------------- | ---------------------------------------------- | ------- | ----- | -| OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT | Maximum allowed span attribute count | 128 | | -| OTEL_SPAN_EVENT_COUNT_LIMIT | Maximum allowed span event count | 128 | | -| OTEL_SPAN_LINK_COUNT_LIMIT | Maximum allowed span link count | 128 | | -| OTEL_EVENT_ATTRIBUTE_COUNT_LIMIT | Maximum allowed attribute per span event count | 128 | | -| OTEL_LINK_ATTRIBUTE_COUNT_LIMIT | Maximum allowed attribute per span link count | 128 | | +| Name | Description | Default | Notes | +| -------------------------------------- | ---------------------------------------------- | ------- | ----- | +| OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT | Maximum allowed attribute value size | | Empty value is treated as infinity. Non-integer and negative values are invalid. | +| OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT | Maximum allowed span attribute count | 128 | | +| OTEL_SPAN_EVENT_COUNT_LIMIT | Maximum allowed span event count | 128 | | +| OTEL_SPAN_LINK_COUNT_LIMIT | Maximum allowed span link count | 128 | | +| OTEL_EVENT_ATTRIBUTE_COUNT_LIMIT | Maximum allowed attribute per span event count | 128 | | +| OTEL_LINK_ATTRIBUTE_COUNT_LIMIT | Maximum allowed attribute per span link count | 128 | | ## OTLP Exporter diff --git a/specification/trace/sdk.md b/specification/trace/sdk.md index 980eb2d8cc0..f6114d4ed87 100644 --- a/specification/trace/sdk.md +++ b/specification/trace/sdk.md @@ -319,20 +319,16 @@ Optional parameters: ## Span Limits -Erroneous code can add unintended attributes, events, and links to a span. If -these collections are unbounded, they can quickly exhaust available memory, -resulting in crashes that are difficult to recover from safely. +Span attributes MUST adhere to the [common rules of attribute limits](../common/common.md#attribute-limits). -To protect against such errors, SDK Spans MAY discard attributes, links, and -events that would increase the number of elements of each collection beyond -the configured limit. +SDK Spans MAY also discard links and events that would increase the number of +elements of each collection beyond the configured limit. If the SDK implements the limits above it MUST provide a way to change these limits, via a configuration to the TracerProvider, by allowing users to configure individual limits like in the Java example bellow. -The name of the configuration options SHOULD be `AttributeCountLimit`, -`EventCountLimit` and `LinkCountLimit`. The options MAY be bundled in a class, +The name of the configuration options SHOULD be `EventCountLimit` and `LinkCountLimit`. The options MAY be bundled in a class, which then SHOULD be called `SpanLimits`. Implementations MAY provide additional configuration such as `AttributePerEventCountLimit` and `AttributePerLinkCountLimit`. @@ -354,7 +350,7 @@ public final class SpanLimits { **Configurable parameters:** -* `AttributeCountLimit` (Default=128) - Maximum allowed span attribute count; +* [all common options applicable to attributes](../common/common.md#attribute-limits-configuration) * `EventCountLimit` (Default=128) - Maximum allowed span event count; * `LinkCountLimit` (Default=128) - Maximum allowed span link count; * `AttributePerEventCountLimit` (Default=128) - Maximum allowed attribute per span event count;