Populate SpanIDs and TraceIDs from logs picked via filelog receiver

[ceevaaa]

Component(s)

processor/transform, receiver/filelog

What happened?

Description

I want to populate the TraceID and SpanID fields (in the exported OTLP logs) from the logs picked by the filelog receiver.

An entry from a log looks like this -
{"level":"info","traceId":"f5f510d734d6df8e51ffa998d2045199","spanId":"923e0d2c44e6b43a","additionalData":null,"requestId":"","location":"main.(*application).history","time":"2023-09-27T19:18:44.034049083Z","message":"blah"}

The output from filelog receiver looks something like -

Sep 30 22:34:38 LMAO otelcol-contrib[16706]: LogRecord #0
Sep 30 22:34:38 LMAO otelcol-contrib[16706]: ObservedTimestamp: 2023-09-30 17:04:38.184352971 +0000 UTC
Sep 30 22:34:38 LMAO otelcol-contrib[16706]: Timestamp: 2023-09-27 19:18:44.034049083 +0000 UTC
Sep 30 22:34:38 LMAO otelcol-contrib[16706]: SeverityText: info
Sep 30 22:34:38 LMAO otelcol-contrib[16706]: SeverityNumber: Info(9)
Sep 30 22:34:38 LMAO otelcol-contrib[16706]: Body: Str(blah)
Sep 30 22:34:38 LMAO otelcol-contrib[16706]: Attributes:
Sep 30 22:34:38 LMAO otelcol-contrib[16706]:      -> requestId: Str()
Sep 30 22:34:38 LMAO otelcol-contrib[16706]:      -> level: Str(info)
Sep 30 22:34:38 LMAO otelcol-contrib[16706]:      -> traceId: Str(f5f510d734d6df8e51ffa998d2045199)
Sep 30 22:34:38 LMAO otelcol-contrib[16706]:      -> additionalData: Str(<nil>)
Sep 30 22:34:38 LMAO otelcol-contrib[16706]:      -> spanId: Str(923e0d2c44e6b43a)
Sep 30 22:34:38 LMAO otelcol-contrib[16706]:      -> fyId: Str()
Sep 30 22:34:38 LMAO otelcol-contrib[16706]:      -> location: Str(main.(*application).historyWebHandler.func1[history_web.go:16])
Sep 30 22:34:38 LMAO otelcol-contrib[16706]:      -> time: Str(2023-09-27T19:18:44.034049083Z)
Sep 30 22:34:38 LMAO otelcol-contrib[16706]:      -> message: Str(History Web Handler)
Sep 30 22:34:38 LMAO otelcol-contrib[16706]:      -> log.file.name: Str(gg.log)
Sep 30 22:34:38 LMAO otelcol-contrib[16706]: Trace ID:
Sep 30 22:34:38 LMAO otelcol-contrib[16706]: Span ID:

Can I do this via the filelog receiver itself ? Or i will need to do this using the transform processor ?
Either way can someone please point me to the exact expression I will need to write to populate these ?

Thanks and Regards

Collector version

v0.83.0

Environment information

Environment

OS: (e.g., "Ubuntu 22.04")

OpenTelemetry Collector configuration

filelog:
    include:
      - /gg.log
    start_at: beginning
    operators:
      - type: filter
        expr: 'body matches "GIN"'
      - type: filter
        expr: 'body matches "gin"'
      - type: filter
        expr: 'body matches "INF"'
      - type: json_parser
      - type: severity_parser
        parse_from: attributes["level"]
        preset: default
      - type: time_parser
        parse_from: attributes["time"]
        layout: "%Y-%m-%dT%H:%M:%S.%LZ"

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• processor/transform: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley
• receiver/filelog: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[ceevaaa]

/label -bug -needs-triage

[ceevaaa]

/label +question -bug

[ceevaaa]

This worked for me.

- set(trace_id.string, attributes["traceId"]) where attributes["traceId"] != nil and IsString(attributes["traceId"]) == true
- set(span_id.string, attributes["spanId"]) where attributes["spanId"] != nil and IsString(attributes["spanId"]) == true

[djaglowski]

You can also do this within the receiver. See https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/stanza/docs/types/trace.md#trace-parsing

[ceevaaa]

God! I missed that. Thanks for pointing that out.