Forward input + multi-line parser consumes more than 100% CPU and lost of enriched data

[quanghungb]

Bug Report

During the evaluation of Fluent Bit, I am trying to have the following architecture (critical for our use case):
|fluent-bit-shipper| -> |fluent-bit-aggregator| -> |final destination|

But I got those 2 blocking issues:

• CPU high usage (more than 100%)
• Lost of enriched data (that have been done at the source)

Everything (configurations, instructions to reproduce, observations) provided here:

• v2.2: https://github.com/quanghungb/fluent-bit-forward
• v3: https://github.com/quanghungb/fluent-bit-forward/tree/fluent-bit-v3.0.0

(I also shared this issue with Eduardo and Anurag as they asked)

Thanks for your help & feedback

[drbugfinder-work]

Maybe related: #8473

[drbugfinder-work]

@drbugfinder-work FYI I just tried with the latest version 3.0.2 and still observed the same issues :(

Does it make a difference, when you disable the multiline or/and the regex filter? Can you isolate the issue further?

[quanghungb]

@drbugfinder-work FYI I tried the newest version 3.0.2 and can observe the same issues:

• CPU usage growing up over 100% after few moment
• the enriched data are still missing

Regarding your question about the multiline parser/regex, the issues appear only with forward input, because with the tail input everything is working well

[quanghungb]

hey @drbugfinder-work I was running the same scenario (with the latest 3.0.2) on my Macbook Air (much less power than my workstation) and did not see the issues (the CPU usage stays low and the enriched data are present in the output).
So I assumes that on my Macbook Air the throughput is less important so it does not flood the aggregator forward input.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.