Skip to content

Commit

Permalink
[pkg/ottl] Replace instances of "query" with "statement" (#14680)
Browse files Browse the repository at this point in the history
* Replace instances of "query" with "statement"

* Add PR as issue reference

* Fix link in README

* Remove ottlconfig package

Co-authored-by: Evan Bradley <[email protected]>
  • Loading branch information
evan-bradley and evan-bradley authored Oct 5, 2022
1 parent 13f5098 commit 8ad476c
Show file tree
Hide file tree
Showing 16 changed files with 138 additions and 129 deletions.
16 changes: 16 additions & 0 deletions .chloggen/ottl-statements-rename.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: breaking

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: pkg/ottl

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Rename the `Queries` signal configuration field to `Statements` and remove the `ottlconfig` package.

# One or more tracking issues related to the change
issues: [14680]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
16 changes: 16 additions & 0 deletions .chloggen/transformprocessor-statements-rename.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: breaking

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: processor/transform

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Rename the `queries` config key to `statements`.

# One or more tracking issues related to the change
issues: [14680]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
2 changes: 1 addition & 1 deletion pkg/ottl/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The OpenTelemetry Transformation Language is a language for transforming open telemetry data based on the [OpenTelemetry Collector Processing Exploration](https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/processing.md).

This package reads in OTTL queries and converts them to invokable Booleans and functions based on the OTTL's grammar.
This package reads in OTTL statements and converts them to invokable Booleans and functions based on the OTTL's grammar.

The OTTL is signal agnostic; it is not aware of the type of telemetry on which it will operate. Instead, the Booleans and functions returned by the package must be passed a `TransformContext`, which provide access to the signal's telemetry. Telemetry data can be accessed and updated through [Getters and Setters](#getters-and-setters).

Expand Down
27 changes: 0 additions & 27 deletions pkg/ottl/ottlconfig/config.go

This file was deleted.

2 changes: 1 addition & 1 deletion pkg/ottl/ottlfuncs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,7 +262,7 @@ The `set` function allows users to set a telemetry field using a value.

`target` is a path expression to a telemetry field. `value` is any value type. If `value` resolves to `nil`, e.g. it references an unset map value, there will be no action.

How the underlying telemetry field is updated is decided by the path expression implementation provided by the user to the `ottl.ParseQueries`.
How the underlying telemetry field is updated is decided by the path expression implementation provided by the user to the `ottl.ParseStatements`.

Examples:

Expand Down
6 changes: 3 additions & 3 deletions pkg/ottl/parser.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ func NewParser[K any](functions map[string]interface{}, pathParser PathExpressio
}

func (p *Parser[K]) ParseStatements(statements []string) ([]Statement[K], error) {
var queries []Statement[K]
var parsedStatements []Statement[K]
var errors error

for _, statement := range statements {
Expand All @@ -56,7 +56,7 @@ func (p *Parser[K]) ParseStatements(statements []string) ([]Statement[K], error)
errors = multierr.Append(errors, err)
continue
}
queries = append(queries, Statement[K]{
parsedStatements = append(parsedStatements, Statement[K]{
Function: function,
Condition: expression,
})
Expand All @@ -65,7 +65,7 @@ func (p *Parser[K]) ParseStatements(statements []string) ([]Statement[K], error)
if errors != nil {
return nil, errors
}
return queries, nil
return parsedStatements, nil
}

var parser = newParser()
Expand Down
20 changes: 10 additions & 10 deletions processor/transformprocessor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,16 @@
| Warnings | [Unsound Transformations, Identity Conflict, Orphaned Telemetry, Other](#warnings) |

The transform processor modifies telemetry based on configuration using the [OpenTelemetry Transformation Language](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl).
The processor takes a list of queries for each signal type and executes the queries against the incoming telemetry in the order specified in the config. Each statement can access and transform telemetry using functions and allow the use of a condition to help decide whether the function should be executed.
The processor takes a list of statements for each signal type and executes the statements against the incoming telemetry in the order specified in the config. Each statement can access and transform telemetry using functions and allow the use of a condition to help decide whether the function should be executed.

## Config

The transform processor allows configuring queries for traces, metrics, and logs. Each signal specifies a list of string queries that get passed to the OTTL for interpretation.
The transform processor allows configuring statements for traces, metrics, and logs. Each signal specifies a list of string statements that get passed to the OTTL for interpretation.

```yaml
transform:
<traces|metrics|logs>:
queries:
statements:
- string
- string
- string
Expand All @@ -29,7 +29,7 @@ Example configuration:
```yaml
transform:
traces:
queries:
statements:
- set(status.code, 1) where attributes["http.path"] == "/health"
- keep_keys(resource.attributes, "service.name", "service.namespace", "cloud.region", "process.command_line")
- set(name, attributes["http.route"])
Expand All @@ -40,7 +40,7 @@ transform:
- truncate_all(attributes, 4096)
- truncate_all(resource.attributes, 4096)
metrics:
queries:
statements:
- set(metric.description, "Sum") where metric.type == "Sum"
- keep_keys(resource.attributes, "host.name")
- limit(attributes, 100, "host.name")
Expand All @@ -49,7 +49,7 @@ transform:
- convert_sum_to_gauge() where metric.name == "system.processes.count"
- convert_gauge_to_sum("cumulative", false) where metric.name == "prometheus_metric"
logs:
queries:
statements:
- set(severity_text, "FAIL") where body == "request failed"
- replace_all_matches(attributes, "/user/*/list/*", "/user/{userId}/list/{listId}")
- replace_all_patterns(attributes, "/account/\\d{4}", "/account/{accountId}")
Expand All @@ -65,7 +65,7 @@ You can learn more in-depth details on the capabilities and limitations of the O
The transform processor utilizes the OTTL's standard contexts for Traces, Metrics and Logs. The contexts allow the OTTL to interact with the underlying telemetry data in its pdata form.
- [Traces Context](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/contexts/ottltraces)
- [Metrics Context](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/contexts/ottlmetrics)
- [Metrics Context](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/contexts/ottldatapoints)
- [Logs Context](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/contexts/ottllogs)
## Supported functions:
Expand Down Expand Up @@ -119,7 +119,7 @@ The `convert_summary_count_val_to_sum` function creates a new Sum metric from a

`aggregation_temporality` is a string (`"cumulative"` or `"delta"`) representing the desired aggregation temporality of the new metric. `is_monotonic` is a boolean representing the monotonicity of the new metric.

The name for the new metric will be `<summary metric name>_count`. The fields that are copied are: `timestamp`, `starttimestamp`, `attibutes`, and `description`. The new metric that is created will be passed to all functions in the metrics queries list. Function conditions will apply.
The name for the new metric will be `<summary metric name>_count`. The fields that are copied are: `timestamp`, `starttimestamp`, `attibutes`, and `description`. The new metric that is created will be passed to all functions in the metrics statements list. Function conditions will apply.

**NOTE:** This function may cause a metric to break semantics for [Sum metrics](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/data-model.md#sums). Use at your own risk.

Expand All @@ -138,7 +138,7 @@ The `convert_summary_sum_val_to_sum` function creates a new Sum metric from a Su

`aggregation_temporality` is a string (`"cumulative"` or `"delta"`) representing the desired aggregation temporality of the new metric. `is_monotonic` is a boolean representing the monotonicity of the new metric.

The name for the new metric will be `<summary metric name>_sum`. The fields that are copied are: `timestamp`, `starttimestamp`, `attibutes`, and `description`. The new metric that is created will be passed to all functions in the metrics queries list. Function conditions will apply.
The name for the new metric will be `<summary metric name>_sum`. The fields that are copied are: `timestamp`, `starttimestamp`, `attibutes`, and `description`. The new metric that is created will be passed to all functions in the metrics statements list. Function conditions will apply.

**NOTE:** This function may cause a metric to break semantics for [Sum metrics](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/data-model.md#sums). Use at your own risk.

Expand All @@ -156,7 +156,7 @@ See [CONTRIBUTING.md](https://github.com/open-telemetry/opentelemetry-collector-

## Warnings

The transform processor's implementation of the [OpenTelemetry Transformation Language](https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/processing.md#telemetry-query-language) (OTTL) allows users to modify all aspects of their telemetry. Some specific risks are listed below, but this is not an exhaustive list. In general, understand your data before using the transform processor.
The transform processor's implementation of the [OpenTelemetry Transformation Language]https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/processing.md#opentelemetry-transformation-language) (OTTL) allows users to modify all aspects of their telemetry. Some specific risks are listed below, but this is not an exhaustive list. In general, understand your data before using the transform processor.

- [Unsound Transformations](https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/standard-warnings.md#unsound-transformations): Several Metric-only functions allow you to transform one metric data type to another or create new metrics from an existing metrics. Transformations between metric data types are not defined in the [metrics data model](https://github.com/open-telemetry/opentelemetry-specification/blob/main//specification/metrics/data-model.md). These functions have the expectation that you understand the incoming data and know that it can be meaningfully converted to a new metric data type or can meaningfully be used to create new metrics.
- Although the OTTL allows the `set` function to be used with `metric.data_type`, its implementation in the transform processor is NOOP. To modify a data type you must use a function specific to that purpose.
Expand Down
19 changes: 14 additions & 5 deletions processor/transformprocessor/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ import (
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/contexts/ottldatapoints"
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/contexts/ottllogs"
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/contexts/ottltraces"
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/ottlconfig"
"github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor/internal/logs"
"github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor/internal/metrics"
"github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor/internal/traces"
Expand All @@ -32,7 +31,17 @@ import (
type Config struct {
config.ProcessorSettings `mapstructure:",squash"`

ottlconfig.Config `mapstructure:",squash"`
OTTLConfig `mapstructure:",squash"`
}

type OTTLConfig struct {
Traces SignalConfig `mapstructure:"traces"`
Metrics SignalConfig `mapstructure:"metrics"`
Logs SignalConfig `mapstructure:"logs"`
}

type SignalConfig struct {
Statements []string `mapstructure:"statements"`
}

var _ config.Processor = (*Config)(nil)
Expand All @@ -41,19 +50,19 @@ func (c *Config) Validate() error {
var errors error

ottltracesp := ottltraces.NewParser(traces.Functions(), component.TelemetrySettings{Logger: zap.NewNop()})
_, err := ottltracesp.ParseStatements(c.Traces.Queries)
_, err := ottltracesp.ParseStatements(c.Traces.Statements)
if err != nil {
errors = multierr.Append(errors, err)
}

ottlmetricsp := ottldatapoints.NewParser(metrics.Functions(), component.TelemetrySettings{Logger: zap.NewNop()})
_, err = ottlmetricsp.ParseStatements(c.Metrics.Queries)
_, err = ottlmetricsp.ParseStatements(c.Metrics.Statements)
if err != nil {
errors = multierr.Append(errors, err)
}

ottllogsp := ottllogs.NewParser(logs.Functions(), component.TelemetrySettings{Logger: zap.NewNop()})
_, err = ottllogsp.ParseStatements(c.Logs.Queries)
_, err = ottllogsp.ParseStatements(c.Logs.Statements)
if err != nil {
errors = multierr.Append(errors, err)
}
Expand Down
16 changes: 7 additions & 9 deletions processor/transformprocessor/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,6 @@ import (
"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/config"
"go.opentelemetry.io/collector/confmap/confmaptest"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/ottlconfig"
)

func TestLoadConfig(t *testing.T) {
Expand All @@ -38,21 +36,21 @@ func TestLoadConfig(t *testing.T) {
id: config.NewComponentIDWithName(typeStr, ""),
expected: &Config{
ProcessorSettings: config.NewProcessorSettings(config.NewComponentID(typeStr)),
Config: ottlconfig.Config{
Traces: ottlconfig.SignalConfig{
Queries: []string{
OTTLConfig: OTTLConfig{
Traces: SignalConfig{
Statements: []string{
`set(name, "bear") where attributes["http.path"] == "/animal"`,
`keep_keys(attributes, "http.method", "http.path")`,
},
},
Metrics: ottlconfig.SignalConfig{
Queries: []string{
Metrics: SignalConfig{
Statements: []string{
`set(metric.name, "bear") where attributes["http.path"] == "/animal"`,
`keep_keys(attributes, "http.method", "http.path")`,
},
},
Logs: ottlconfig.SignalConfig{
Queries: []string{
Logs: SignalConfig{
Statements: []string{
`set(body, "bear") where attributes["http.path"] == "/animal"`,
`keep_keys(attributes, "http.method", "http.path")`,
},
Expand Down
21 changes: 10 additions & 11 deletions processor/transformprocessor/factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ import (
"go.opentelemetry.io/collector/consumer"
"go.opentelemetry.io/collector/processor/processorhelper"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/ottlconfig"
"github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor/internal/logs"
"github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor/internal/metrics"
"github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor/internal/traces"
Expand All @@ -49,15 +48,15 @@ func NewFactory() component.ProcessorFactory {
func createDefaultConfig() config.Processor {
return &Config{
ProcessorSettings: config.NewProcessorSettings(config.NewComponentID(typeStr)),
Config: ottlconfig.Config{
Logs: ottlconfig.SignalConfig{
Queries: []string{},
OTTLConfig: OTTLConfig{
Logs: SignalConfig{
Statements: []string{},
},
Traces: ottlconfig.SignalConfig{
Queries: []string{},
Traces: SignalConfig{
Statements: []string{},
},
Metrics: ottlconfig.SignalConfig{
Queries: []string{},
Metrics: SignalConfig{
Statements: []string{},
},
},
}
Expand All @@ -71,7 +70,7 @@ func createLogsProcessor(
) (component.LogsProcessor, error) {
oCfg := cfg.(*Config)

proc, err := logs.NewProcessor(oCfg.Logs.Queries, logs.Functions(), set.TelemetrySettings)
proc, err := logs.NewProcessor(oCfg.Logs.Statements, logs.Functions(), set.TelemetrySettings)
if err != nil {
return nil, fmt.Errorf("invalid config for \"transform\" processor %w", err)
}
Expand All @@ -92,7 +91,7 @@ func createTracesProcessor(
) (component.TracesProcessor, error) {
oCfg := cfg.(*Config)

proc, err := traces.NewProcessor(oCfg.Traces.Queries, traces.Functions(), set.TelemetrySettings)
proc, err := traces.NewProcessor(oCfg.Traces.Statements, traces.Functions(), set.TelemetrySettings)
if err != nil {
return nil, fmt.Errorf("invalid config for \"transform\" processor %w", err)
}
Expand All @@ -113,7 +112,7 @@ func createMetricsProcessor(
) (component.MetricsProcessor, error) {
oCfg := cfg.(*Config)

proc, err := metrics.NewProcessor(oCfg.Metrics.Queries, metrics.Functions(), set.TelemetrySettings)
proc, err := metrics.NewProcessor(oCfg.Metrics.Statements, metrics.Functions(), set.TelemetrySettings)
if err != nil {
return nil, fmt.Errorf("invalid config for \"transform\" processor %w", err)
}
Expand Down
Loading

0 comments on commit 8ad476c

Please sign in to comment.