Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make basicstats faster #4723

Closed
markusr opened this issue Sep 19, 2018 · 2 comments · Fixed by #4741
Closed

Make basicstats faster #4723

markusr opened this issue Sep 19, 2018 · 2 comments · Fixed by #4741
Labels
performance problems with decreased performance or enhancements that improve performance

Comments

@markusr
Copy link

markusr commented Sep 19, 2018

Steps to reproduce:

  1. Config basicstats to only do
stats = ["count", "min" , "max" , "mean"]

Expected behavior:

"mean" is calculated at the Push by dividing

v.sum / v.count

Actual behavior:

All values also variance are calculated on each Add
https://github.com/influxdata/telegraf/blob/master/plugins/aggregators/basicstats/basicstats.go#L107

Additional info:

I tested this on a raspberry pi 3.

After removing the "online" calculation part the cpu usage dropped from 72% to 35%. (3000 metric per second input and 5s aggregation time).

Another idea would be to add sum and mean to the minmax aggregator or a complete new aggregator.

@danielnelson
Copy link
Contributor

@markusr I think we need the online mean for the online variance, but we should get a big performance gain by switching from Fields() to FieldList(). I made this change in #4741, can you compare the performance with this code?

@danielnelson danielnelson added the performance problems with decreased performance or enhancements that improve performance label Sep 24, 2018
@markusr
Copy link
Author

markusr commented Sep 26, 2018

@danielnelson
You are right. #4741 gives the same speed up. The CPU usage I measured was on the same level (35.41%) as my test before.

It can be merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance problems with decreased performance or enhancements that improve performance
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants