Add support for histograms to metrics intake #5360
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
axw
force-pushed
the
modeldecoder-metric-type-unit
branch
from
e2b6d5c to
2fc5e46
Compare
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪 |
|
@beniwohli @felixbarny FYI. Would be good to get your eyes at least on the changes in docs/schema and testdata: https://github.com/elastic/apm-server/pull/5360/files?file-filters%5B%5D=.json&file-filters%5B%5D=.ndjson |
axw
force-pushed
the
modeldecoder-metric-type-unit
branch
from
2fc5e46 to
89ee1e2
Compare
axw
force-pushed
the
modeldecoder-metric-type-unit
branch
from
89ee1e2 to
1dca4c8
Compare
stuartnelson3
approved these changes
May 31, 2021
|
jenkins run the tests please |
mergify bot
pushed a commit
that referenced
this pull request
Jun 1, 2021
* model/modeldecoder: add metric type and unit * systemtest: test histogram metrics * Update changelog * systemtest: fix min docs expectation in test (cherry picked from commit 0744428) # Conflicts: # changelogs/head.asciidoc
This was referenced Jun 1, 2021
axw
added a commit
that referenced
this pull request
Jun 2, 2021
…5381) * Add support for histograms to metrics intake (#5360) * model/modeldecoder: add metric type and unit * systemtest: test histogram metrics * Update changelog * systemtest: fix min docs expectation in test (cherry picked from commit 0744428) # Conflicts: # changelogs/head.asciidoc * Delete head.asciidoc Co-authored-by: Andrew Wilkins <[email protected]>
beniwohli
added a commit
to beniwohli/apm-agent-python
that referenced
this pull request
Jun 14, 2021
The spec for histograms has been merged in elastic/apm-server#5360
|
Tested with BC2 with an example request received from @beniwohli (testing with opbeans app did not yet fully work): histogram data are indexed in ES. |
beniwohli
added a commit
to elastic/apm-agent-python
that referenced
this pull request
Jul 19, 2021
* catch and log exceptions in the interval timer function Instead of letting the thread die, we log the exception with a stack trace to ease debugging. * implement histogram metrics and a prometheus histogram bridge The spec for histograms has been merged in elastic/apm-server#5360 * trying to debug failure that only happens on CI... * adapt prometheus histograms to use centroids instead of upper limit buckets * move midpoint calculation into base metrics
beniwohli
added a commit
to beniwohli/apm-agent-python
that referenced
this pull request
Sep 14, 2021
* catch and log exceptions in the interval timer function Instead of letting the thread die, we log the exception with a stack trace to ease debugging. * implement histogram metrics and a prometheus histogram bridge The spec for histograms has been merged in elastic/apm-server#5360 * trying to debug failure that only happens on CI... * adapt prometheus histograms to use centroids instead of upper limit buckets * move midpoint calculation into base metrics
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation/summary
Extend the metricset schema with additional fields on "samples": type, unit, values, and counts. For now, the only type that is honoured is "histogram", and units are not honoured at all. We need a resolution on elastic/elasticsearch#72536 to handle gauge/counter and units.
If type is "histogram", then you are expected send "counts" and "values". Counts must be a list of non-negative integers. Values must be a list of (any) numbers; the values must be provided in ascending order.
With this change, one of "value" or "values" must be supplied. Previously "value" was required, so sending a histogram to an older server will result in a validation error. If "values" is supplied, "counts" must also be supplied.
Checklist
- [ ] Documentation has been updatedHow to test these changes
Once one of our agents has implemented support for sending histogram metrics (probably via the Prometheus API?), use one to send to APM Server and ensure the metric is stored in a histogram field with expected counts/values.
Related issues
Closes #4884
Closes #3195