Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[processor/routing] OTTL statements doesn't work without from_attribute #16555

Closed
vchirikov opened this issue Nov 30, 2022 · 4 comments · Fixed by #16582
Closed

[processor/routing] OTTL statements doesn't work without from_attribute #16555

vchirikov opened this issue Nov 30, 2022 · 4 comments · Fixed by #16582
Assignees
Labels
bug Something isn't working processor/routing Routing processor

Comments

@vchirikov
Copy link

vchirikov commented Nov 30, 2022

Component(s)

processor/routing

What happened?

Description

According to the docs
We should be able to use OTTL statements without from_attribute and other "conventional routing items"

```yaml
processors:
routing:
default_exporters:
- jaeger
table:
- statement: route() where resource.attributes["X-Tenant"] == "acme"
exporters: [jaeger/acme]
- statement: delete_key(resource.attributes, "X-Tenant") where IsMatch(resource.attributes["X-Tenant"], ".*corp") == true
exporters: [jaeger/ecorp]
exporters:
jaeger:
endpoint: localhost:14250
jaeger/acme:
endpoint: localhost:24250
jaeger/ecorp:
endpoint: localhost:34250
```

But if I try to use it without from_attribute the collector won't start.

p.s. and it will be useful to add an example of filtering k8s logs for example into 2 elasticsearchexporters based on OTTL, because it's unclear how to not route to all exporters in case of similar attributes (for example how to route only one container from the pod into exporter)

Collector version

0.66.0

Environment information

Environment

Docker image otel/opentelemetry-collector-contrib:0.66.0

OpenTelemetry Collector configuration

processors:
  routing:
    default_exporters: [elasticsearch/default]
    table:
      - statement: route() where (resource.attributes["k8s.namespace.name"] == "infra" and resource.attributes["k8s.container.name"] != "access-logs") or resource.attributes["k8s.namespace.name"] == "flux-system"
        exporters: [elasticsearch/infra]
      - statement: route() where resource.attributes["k8s.namespace.name"] == "infra" and resource.attributes["k8s.container.name"] == "access-logs"
        exporters: [elasticsearch/ingress]
      - statement: route() where resource.attributes["k8s.namespace.name"] == "staging" and resource.attributes["k8s.container.name"] == "app"
        exporters: [elasticsearch/apps]

Log output

Error: invalid configuration: processor "routing" has invalid configuration: invalid attribute to read the route's value from: the FromAttribute property is empty
2022-11-30 13:55:57.699034 I | collector server run finished with error: invalid configuration: processor "routing" has invalid configuration: invalid attribute to read the route's value from: the FromAttribute property is empty

Additional context

#3746 looks like similar

@vchirikov vchirikov added bug Something isn't working needs triage New item requiring triage labels Nov 30, 2022
@github-actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@jpkrohling
Copy link
Member

@kovrus, are you able to handle this one?

@jpkrohling jpkrohling added processor/routing Routing processor and removed needs triage New item requiring triage labels Dec 1, 2022
@vchirikov
Copy link
Author

vchirikov commented Dec 1, 2022

I checked with 2 otel collectors, with config like this:

  receivers:
    jaeger: null
    otlp:
      protocols:
        grpc:
          endpoint: 0.0.0.0:4317
        http: null
    prometheus: null
    zipkin: null

  processors:
    batch: {}
    memory_limiter: null
    routing:
      from_attribute: service.name
      default_exporters: [logging]
      table:
        - statement: route() where resource.attributes["service.name"] == "123"
          exporters: [otlp]

  exporters:
    logging:
      verbosity: detailed
      sampling_initial: 1000
      sampling_thereafter: 2000
    otlp:
      endpoint: otel-2:4317
      tls:
        insecure: true
        insecure_skip_verify: true
  service:
    extensions:
      - health_check
    pipelines:
      logs:
        receivers:
          - otlp
        processors:  []
        exporters:
          - logging
      metrics: null
      traces:
        receivers:
          - otlp
        processors:
          - routing
        exporters:
          - otlp
          - logging

and all traces were in logging exporter:

2022-12-01T16:22:56.741Z	info	ResourceSpans #0
Resource SchemaURL: 
Resource attributes:
+     -> service.name: Str(123)
     -> service.instance.id: Str(5464a388-9ee0-4c63-b858-3d910597adba)
     -> k8s.pod.ip: Str(127.0.0.1)
ScopeSpans #0
ScopeSpans SchemaURL: 
InstrumentationScope opentelemetry 1.0.0
Span #0
    Trace ID       : e6b6b12325d04a35c368ae67b629bece
    Parent ID      : 
    ID             : 4c8c253d2b3f97f3
    Name           : Parent_01
    Kind           : Internal
    Start time     : 2022-12-01 16:22:56.7316525 +0000 UTC
    End time       : 2022-12-01 16:22:56.7316554 +0000 UTC
    Status code    : Unset
    Status message : 
	{"kind": "exporter", "data_type": "traces", "name": "logging"}

Please check the config, because seems like the OTTL feature doesn't work at all in routing processor

upd:

Well, I found out that that OTTL doesn't work without these settings:

      from_attribute: __ # required to start processor, could be anything
      attribute_source: resource # statements on a resource don't work without this

@kovrus
Copy link
Member

kovrus commented Dec 2, 2022

@jpkrohling please assign it to me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working processor/routing Routing processor
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants