Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[filebeat] Drop __REALTIME_TIMESTAMP from the output that caused the new field "journald.custom.realtime_timestamp" in the final event document #40658

Merged
merged 5 commits into from
Aug 30, 2024
Merged

[filebeat] Drop __REALTIME_TIMESTAMP from the output that caused the new field "journald.custom.realtime_timestamp" in the final event document #40658

Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a7da306
[filebeat] Drop __REALTIME_TIMESTAMP from the output that caused the …
aleksmaus Aug 27, 2024
3897878
Merge branch 'main' into fix/filebeat_journald_realtime_timestamp
aleksmaus Aug 29, 2024
7b622ea
Merge branch 'main' into fix/filebeat_journald_realtime_timestamp
aleksmaus Aug 29, 2024
ff6d125
Merge branch 'main' into fix/filebeat_journald_realtime_timestamp
aleksmaus Aug 29, 2024
3ecec14
Replace misleading comment
aleksmaus Aug 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions filebeat/input/journald/pkg/journalfield/default.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,4 +85,5 @@ var journaldEventFields = FieldConversion{
"_SOURCE_REALTIME_TIMESTAMP": ignoredField, // saved in the registry
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the comment on L84 inaccurate? It also says it is used for the timestamp.

Copy link
Member Author

@aleksmaus aleksmaus Aug 29, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like it is not accurate, as far as I understand looking at the current code elastic/integrations#10757 (comment)
@belimawr am I missing some place where it is used?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment on L84 was there before we started using journalctl to read the journal. Looking at the old code, the timestamp field was coming from the library we used, here is the code that reads the journal and sets the timestamps.

When I re-wrote it to use journalctl, I looked at the code and __REALTIME_TIMESTAMP seemed the best choice to keep the behaviour of the old code/library.

So the comment there might be misleading :/.

I did write a test that compares both implementations reading the same journal file, so I believe the behaviour has not changed with the new implementation.

"__CURSOR": ignoredField, // saved in the registry
"__MONOTONIC_TIMESTAMP": ignoredField, // saved in the registry
"__REALTIME_TIMESTAMP": ignoredField, // dropped from the output, is used for the @timestamp
}
Loading