Journalbeat has no ability to not process old journal events

[timp87]

Describe the enhancement:
Journalbeat has no ability to not process journal events older than N ${time_units}.
For example, filebeat has ignore_older option which relies on logfile modification time (general approach for plain text logfiles).
Journalbeat has no similar option even for the same approach. It reads the latest event from any journal file it finds even in "tail" mode.

Describe a specific use case for the enhancement or feature:
We have k8s cluster and want to see logs in ELK not only from containers, but from daemons running on host system also.
We reconfigure docker's log-driver option from json-file to journald. Then we deploy journalbeat daemonset instead of filebeat, of couse with mounted /var/log/journal and other valuable dirs from host system.
Journalbeat pulls and processes all events from journal, even if they are a year old. Then events go to logstash.
We don't want to grow our ELK in size uncontrolled. So we split indexes on a per day basis. And keep open indexes for the last week only. Older indexes are closed.
Now logstash tries to write events to the closed indexes. This leads to a filedescriptor leak on logstash (long-living known logstash bug^W feature ;)).
I want a way to tell journalbeat to not process journal event older than a week.

Describe possible workaround
We configure journald to rotate its journal files every day and keep them only for a week. That's it.
So nothing in journalbeat configuration unfortunately.

[elasticmachine]

Pinging @elastic/integrations-services (Team:Services)

[andrewkroh]

I think it might be possible to seek the cursor based on time by using sd_journal_seek_realtime_usec. If so, then when a new reader starts for the first time (no previous state is found) then it could start reading at the time derived from ignore_older. If ignore_older: 24h was used then it would seek to now() - 24h and then start reading. Any restarts afterwards would resume from the last persisted cursor position.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[timp87]

It's relevant

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[timp87]

Still need this

[andrewkroh]

I want a way to tell journalbeat to not process journal event older than a week.

(journalbeat is no longer around and the functionality is part of Filebeat.)

This is possible. The following configuration will at most initially backfill -168h of data (one week). Then after that it will resume from the saved cursor when restarted (because a cursor for all_events will exist in the registry).

filebeat.inputs:
- type: journald
  id: all_events
  seek: cursor
  cursor_seek_fallback: since
  since: -168h

• https://www.elastic.co/guide/en/beats/filebeat/8.12/filebeat-input-journald.html#filebeat-input-journald-seek
• filebeat/input/journald: allow specifying since when to read journald entries #35408