-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The histogram aggregator plugin was added. #2387
Merged
Merged
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
5fcd278
The histogram aggregator plugin was added.
9f9327f
The gofmt tool was applied.
d24d011
The problem with test was fixed.
53b63fa
Changed the config format of histogram.
232c432
The using uint64 instead int64. The checking of overflow of bucket hi…
17ea04e
The go fmt tool was applied.
79ea10f
The tests were fixed.
30aa7f9
Added the additional information about buckets to README.md.
2d94051
Added item to Features section in the file CHANGELOG.md.
af29572
Changes by code-review.
023f3f6
Changes by code-review. Part 2.
ca40497
Using copying of tags in the method: addFields
d69f245
Some small changes in tests.
d902a12
The converting of bucket to string was fixed.
2873efc
The grouping metric fields by value was added.
vlamug f4e3867
The unit test were fixed.
vlamug 36f356c
The changelog file was changed.
vlamug e8f4476
Histogram aggregator now groups metric fields by bucket
abc2324
Merge pull request #1 from vlamug/aggregator-history-groupFieldsCount…
vlamug 700c52b
The info about histogram plugin was deleted from CHANGELOG file.
vlamug File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
package all | ||
|
||
import ( | ||
_ "github.com/influxdata/telegraf/plugins/aggregators/histogram" | ||
_ "github.com/influxdata/telegraf/plugins/aggregators/minmax" | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,128 @@ | ||
# Histogram Aggregator Plugin | ||
|
||
#### Goal | ||
|
||
This plugin was added for ability to build histograms. | ||
|
||
#### Description | ||
|
||
The histogram aggregator plugin aggregates values of specified metric's | ||
fields. The metric is emitted every `period` seconds. All you need to do | ||
is to specify borders of histogram buckets and fields, for which you want | ||
to aggregate histogram. | ||
|
||
#### How it works | ||
|
||
The each metric is passed to the aggregator and this aggregator searches | ||
histogram buckets for those fields, which have been specified in the | ||
config. If buckets are found, the aggregator will put +1 to appropriate | ||
bucket. Otherwise, nothing will happen. Every `period` seconds these data | ||
will be pushed to output. | ||
|
||
Note, that the all hits of current bucket will be also added to all next | ||
buckets in final result of distribution. Why does it work this way? In | ||
configuration you define right borders for each bucket in a ascending | ||
sequence. Internally buckets are presented as ranges with borders | ||
(0..bucketBorder]: 0..1, 0..10, 0..50, …, 0..+Inf. So the value "+1" will be | ||
put into those buckets, in which the metric value fell with such ranges of | ||
buckets. | ||
|
||
This plugin creates cumulative histograms. It means, that the hits in the | ||
buckets will always increase from the moment of telegraf start. But if you | ||
restart telegraf, all hits in the buckets will be reset to 0. | ||
|
||
Also, the algorithm of hit counting to buckets was implemented on the base | ||
of the algorithm, which is implemented in the Prometheus | ||
[client](https://github.com/prometheus/client_golang/blob/master/prometheus/histogram.go). | ||
|
||
### Configuration | ||
|
||
```toml | ||
# Configuration for aggregate histogram metrics | ||
[[aggregators.histogram]] | ||
## General Aggregator Arguments: | ||
## The period on which to flush & clear the aggregator. | ||
period = "30s" | ||
## If true, the original metric will be dropped by the | ||
## aggregator and will not get sent to the output plugins. | ||
drop_original = false | ||
|
||
## The example of config to aggregate histogram for all fields of specified metric. | ||
[[aggregators.histogram.config]] | ||
## The set of buckets. | ||
buckets = [0.0, 15.6, 34.5, 49.1, 71.5, 80.5, 94.5, 100.0] | ||
## The name of metric. | ||
metric_name = "cpu" | ||
|
||
## The example of config to aggregate histogram for concrete fields of specified metric. | ||
[[aggregators.histogram.config]] | ||
## The set of buckets. | ||
buckets = [0.0, 10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0] | ||
## The name of metric. | ||
metric_name = "diskio" | ||
## The concrete fields of metric. | ||
metric_fields = ["io_time", "read_time", "write_time"] | ||
``` | ||
|
||
#### Explanation | ||
|
||
The field `metric_fields` is the list of metric fields. For example, the | ||
metric `cpu` has the following fields: usage_user, usage_system, | ||
usage_idle, usage_nice, usage_iowait, usage_irq, usage_softirq, usage_steal, | ||
usage_guest, usage_guest_nice. | ||
|
||
Note that histogram metrics will be pushed every `period` seconds. | ||
As you know telegraf calls aggregator `Reset()` func each `period` seconds. | ||
Histogram aggregator ignores `Reset()` and continues to count hits. | ||
|
||
#### Use cases | ||
|
||
You can specify fields using two cases: | ||
|
||
1. The specifying only metric name. In this case all fields of metric | ||
will be aggregated. | ||
2. The specifying metric name and concrete field. | ||
|
||
#### Some rules | ||
|
||
- The setting of each histogram must be in separate section with title | ||
`aggregators.histogram.config`. | ||
|
||
- The each value of bucket must be float value. | ||
|
||
- Don\`t include the border bucket `+Inf`. It will be done automatically. | ||
|
||
### Measurements & Fields: | ||
|
||
The postfix `bucket` will be added to each field. | ||
|
||
- measurement1 | ||
- field1_bucket | ||
- field2_bucket | ||
|
||
### Tags: | ||
|
||
All measurements have tag `le`. This tag has the border value of bucket. It | ||
means that the metric value is less or equal to the value of this tag. For | ||
example, let assume that we have the metric value 10 and the following | ||
buckets: [5, 10, 30, 70, 100]. Then the tag `le` will have the value 10, | ||
because the metrics value is passed into bucket with right border value `10`. | ||
|
||
### Example Output: | ||
|
||
The following output will return to the Prometheus client. | ||
|
||
``` | ||
cpu,cpu=cpu1,host=localhost,le=0.0 usage_idle_bucket=0i 1486998330000000000 | ||
cpu,cpu=cpu1,host=localhost,le=10.0 usage_idle_bucket=0i 1486998330000000000 | ||
cpu,cpu=cpu1,host=localhost,le=20.0 usage_idle_bucket=1i 1486998330000000000 | ||
cpu,cpu=cpu1,host=localhost,le=30.0 usage_idle_bucket=2i 1486998330000000000 | ||
cpu,cpu=cpu1,host=localhost,le=40.0 usage_idle_bucket=2i 1486998330000000000 | ||
cpu,cpu=cpu1,host=localhost,le=50.0 usage_idle_bucket=2i 1486998330000000000 | ||
cpu,cpu=cpu1,host=localhost,le=60.0 usage_idle_bucket=2i 1486998330000000000 | ||
cpu,cpu=cpu1,host=localhost,le=70.0 usage_idle_bucket=2i 1486998330000000000 | ||
cpu,cpu=cpu1,host=localhost,le=80.0 usage_idle_bucket=2i 1486998330000000000 | ||
cpu,cpu=cpu1,host=localhost,le=90.0 usage_idle_bucket=2i 1486998330000000000 | ||
cpu,cpu=cpu1,host=localhost,le=100.0 usage_idle_bucket=2i 1486998330000000000 | ||
cpu,cpu=cpu1,host=localhost,le=+Inf usage_idle_bucket=2i 1486998330000000000 | ||
``` |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
each bucket is an "upper inclusive bound", correct? you should document that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you're right. Sure
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Added to documentation.