Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[otel log] Handle logs without Timestamp field better. #5311

Closed
c0per opened this issue Aug 13, 2024 · 2 comments · Fixed by #5366
Closed

[otel log] Handle logs without Timestamp field better. #5311

c0per opened this issue Aug 13, 2024 · 2 comments · Fixed by #5366
Assignees
Labels
bug Something isn't working

Comments

@c0per
Copy link

c0per commented Aug 13, 2024

Describe the bug

Currently, quickwit is ignoring otel logs with a null (or 0) Timestamp.

// quickwit-opentelemetry::otlp::logs.rs line 441
for log_record in scope_log.log_records {
    if log_record.time_unix_nano == 0 {
        rate_limited_error!(limit_per_min = 10, "skipping record");
        num_parse_errors += 1;
        continue;
    }
    // ...
}

Some instrumentation only set ObservedTimestamp and leave Timestamp null.
Maybe we should default to ObservedTimestamp when Timestamp is not set.

Steps to reproduce
Use any Otel logs exporter which doesn't set the timestamp field.
In my case, I'm using rust tracing + opentelemetry-appender-tracing.

Expected behavior

Other people have encountered this problem using the same setup with the official otel collector (open-telemetry/opentelemetry-collector-contrib#11752 (comment)).

The official otel collector exporter use ObservedTimestamp when Timestamp is not set.
The PR: open-telemetry/opentelemetry-collector-contrib#31844

Maybe Quickwit should also do this. Default to ObservedTimestamp when Timestamp is not set, instead of throwing away logs without a timestamp.

@c0per c0per added the bug Something isn't working label Aug 13, 2024
@rdettai rdettai self-assigned this Aug 19, 2024
@spetz
Copy link

spetz commented Aug 25, 2024

I've just encountered the same issue - traces work great, but the logs are silently ignored with the same error.

@mfaerevaag
Copy link

Experiencing the same issue as @spetz. Fingers crossed for #5366

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants