-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to parse multiline logs with very long lines which are in containerd/crio log format #4512
Comments
We need 2 steps to parse this log.
test.log:
output:
|
My configuration also doesn't output what you expected.
Is this line a valid java exception log? |
@nokute78 I know that logs in presented format could be parsed using two steps as you presented (even if logs do not match to java parser then custom parser could be created) and this configuration could work for simple scenario but the issue is that there is not guarantee that the full multiline log will be sent in the same chunk and multiline filter plugin cannot be used to join multiline logs because it does not perform buffering that persists across different chunks. Related docs:
-https://docs.fluentbit.io/manual/administration/buffering-and-storage
|
@nokute78 Any other feedbacks? This solution is not working for me. This subject is very important for me also. Seems to be exactly the same problem like here https://github.com/fluent/fluent-bit/issues/4551. Thanks in advance |
Thank you for reporting. FYI, we are discussing about Multiline Filter Re-Design. Could you take a look the issue ? |
This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the |
This issue was closed because it has been stalled for 5 days with no activity. |
I'm not able to parse multiline logs with long lines (with partial logs) which are in containred/crio log format using new multiline parser.
Example log file:
Above log file is created by Pod with following specification:
Expected log format:
I see that default
cri
parser is able to extract log content and join logs which are split into multiple lines taking into account log tag (P
andF
) but it is not able to deal with multiline logs in crio/containerd format.As I understand multiline filter plugin cannot be used to join multiline logs because it does not perform buffering that persists across different Chunks and there is not guarantee that the full multiline log will be sent in the same chunk.
The text was updated successfully, but these errors were encountered: