Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increase lineScanLimit in log rotation logic #4699

Closed
jippi opened this issue Sep 19, 2018 · 5 comments · Fixed by #4765
Closed

Increase lineScanLimit in log rotation logic #4699

jippi opened this issue Sep 19, 2018 · 5 comments · Fixed by #4765
Labels
theme/client type/enhancement

Comments

@jippi
Copy link
Contributor

jippi commented Sep 19, 2018

This is a follow-up on #1048 and #4282

We have some applications that are logging quite long lines of JSON (up to 25KB) - and we see our files being split across files mid-line, which causes all kind of bad error and unreadable logs in Graylog.

I would like to have lineScanLimit increased from 16K to 25K - if it doesn't affect performance in any significant way.

I'll submit the change if its okay with the Nomad team
@jippi jippi changed the title Increase scan length Increase lineScanLimit in log rotation logic Sep 19, 2018
@nickethier
Copy link
Member

Hey @jippi

We'd support this change, however let make it 32K.

@preetapan
Copy link
Contributor

@jippi we would take a PR for this for the 0.9 release if you want to get this in!

@jippi
Copy link
Contributor Author

jippi commented Oct 9, 2018

@preetapan thanks! I'll get it wrapped up today

@jippi jippi mentioned this issue Oct 9, 2018
Merged
@jippi
Copy link
Contributor Author

jippi commented Oct 9, 2018

PR open in #4765

jippi added a commit to jippi/nomad that referenced this issue Oct 9, 2018
@jippi
fix: increase log rotator line scan limit
5314bfc 
In case where gelf/json logging is used, its fairly easy to exceed the 16k limit, resulting in json output being cut up into multiple strings

the result is invalid json lines which can create all kind of badness in the logging server

This fixes hashicorp#4699

Signed-off-by: Christian Winther <[email protected]>
@jippi jippi mentioned this issue Nov 2, 2018
Closed
@github-actions
Copy link

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
theme/client type/enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: increase log rotator line scan limit jippi/nomad
3 participants
@jippi @preetapan @nickethier