[Journalbeat] Still no multiline support after 3 years

[yupm]

May I know if there is any progress on the support for multiline in Journalbeats?
This feature has been pending for 3 years already: #8323

Currently we are writing kubernetes/docker logs to journald. However, some of the stack traces are multi line and this gets split up.
It seems that Filebeat has the ability to manage multiline https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html. Might it be easy to port over this feature to Journalbeat?

I have looked into using Logstash to process file events, but the document states that:

If you are using a Logstash input plugin that supports multiple hosts, such as the beats input plugin, you should not use the multiline codec to handle multiline events. Doing so may result in the mixing of streams and corrupted event data. In this situation, you need to handle multiline events before sending the event data to Logstash.

This request was also raised here: #10114

[elasticmachine]

Pinging @elastic/agent (Team:Agent)

[nimarezainia]

Hello @yupm,

As of 7.16 journald is provided as an input to Filebeat: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-journald.html which supports multi-line. Consequently journalbeat has been deprecated from that release onwards.

To future proof your deployment we suggest using filebeat. Would that work for you in this case?

[SpComb]

per https://discuss.elastic.co/t/filebeat-inputs-journald-multiline-not-working/294422 and #29907, the filebeat journald input does not support multi-line.

Should the beats issue be re-opened? Is it a bug, or more of a feature request?

Compared to the existing filebeat file input, multi-line support for journald might need add some additional complexity to account for multiple processes writing interleaved multi-line messages to the same journal stream... for filebeat file inputs, I assume that the multi-line implementation assumes that only one process will be writing per file, but that is generally not the case for journald. I think I discussed this in the older third-party journalbeat repo, but it looks like https://github.com/mheese/journalbeat/issues/37 is now gone 😕

[l0calhost]

From a user perspective i currently see no functional way to handle multiline messages (e.g. jav stack traces) from the journal either with filebeat or journalbeat. The discussion/FR for filebeat is closed (see #29907), same here for journalbeat.

I would very much appreciate it to have multiline support in at least one of the beats to have a technical way to solve this.