Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NAS-129384 / 24.10 / Fix netdata spamming logs #13860

Merged
merged 2 commits into from
Jun 13, 2024
Merged

NAS-129384 / 24.10 / Fix netdata spamming logs #13860

merged 2 commits into from
Jun 13, 2024

Conversation

Qubad786
Copy link
Contributor

@Qubad786 Qubad786 commented Jun 7, 2024

Problem

We were seeing logs being continuously written to, after investigation, we found that only the UPS plugin of netdata is causing issues. When UPS is not configured, the failed upsc -l command returns an error, spamming the Netdata logs and causing the log file size to increase continuously.

Solution

Modify the Netdata plugin to avoid executing the upsc command when upsmon process is not running. Additionally, disable access logs since only the errors.log is used for debugging, thereby preventing unnecessary log file growth.

@Qubad786 Qubad786 requested a review from a team June 7, 2024 16:19
@bugclerk bugclerk changed the title Fix netdata spamming logs NAS-129384 / 24.10 / Fix netdata spamming logs Jun 7, 2024
@bugclerk
Copy link
Contributor

bugclerk commented Jun 7, 2024

Copy link
Contributor

@yocalebo yocalebo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again sorry, but no. We're shell'ing out every 1/2 seconds for this stuff. We can't do this on our enterprise systems so we're going to have to come up with a better solution. If that means we have to disable UPS reporting for now, then so be it.

@Qubad786 Qubad786 requested a review from anodos325 June 11, 2024 05:21
@Qubad786 Qubad786 dismissed yocalebo’s stale review June 11, 2024 05:21

Addressed Review

@Qubad786 Qubad786 merged commit e697d43 into master Jun 13, 2024
3 checks passed
@Qubad786 Qubad786 deleted the NAS-129384 branch June 13, 2024 13:38
@bugclerk
Copy link
Contributor

This PR has been merged and conversations have been locked.
If you would like to discuss more about this issue please use our forums or raise a Jira ticket.

@truenas truenas locked as resolved and limited conversation to collaborators Jun 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants