Support Openmetrics metrics collection #10752
Conversation
ghost
added
ChristineTChen
force-pushed
the
cc/envoy-prom
branch
from
a2ac015 to
42bcff2
Compare
|
The |
|
The |
|
The |
|
The |
envoy/datadog_checks/envoy/check.py
Outdated
| if sample.name.endswith("_total"): | ||
| parsed_sample_name = re.match("(.*)_total$", sample.name).groups()[0] | ||
|
|
||
| compiled_name = re.compile(metric_pattern) |
There was a problem hiding this comment.
Let's cache the compiled patterns in the outer scope using collections.defaultdict
| # | ||
| # cache_metrics: true |
There was a problem hiding this comment.
Is cache_metrics no longer supported? It looks like it's still used in the original implementation:
There was a problem hiding this comment.
I'm documenting openmetrics options only.
There was a problem hiding this comment.
Can we have the legacy options still be in spec.yaml, but just have them hidden? Similar to when Istio was updated to only include OpenMetricsV2 options but legacy options were still included albeit hidden.
| 'included_metrics': [r'envoy\.cluster\.'], | ||
| 'excluded_metrics': [r'envoy\.cluster\.out\.'], | ||
| }, | ||
| 'collect_server_info': { |
There was a problem hiding this comment.
I see that collect_server_info is still kept for tests, but it's no longer in your conf.yaml.example. It's still in the legacy implementation. Should this be included as a config option?
There was a problem hiding this comment.
see above comment, the readme links to the legacy implementation conf.yaml.example.
There was a problem hiding this comment.
We'll need to collect the version still
Co-authored-by: Ofek Lev <[email protected]>
|
@ChristineTChen I'm unable to tell if this functionality is present in any of the My datadog:
confd:
envoy.yaml: |-
init_config:
instances:
- openmetrics_endpoint: http://localhost:15090/stats/prometheus?usedonlyYet the check still chokes on
Then I found the source code leading me to this PR. Can someone point me to an agent image that contains this? |
* Add logic for Envoy Openmetricsv2 * Add label remapper * Add new metrics * Finish adding other metrics * reorganize metrics that should be transformed * Introduce openmetrics_endpoint config option * Add watchdog metrics transformers * Add some more label extraction metrics * refactor tests to move to legacy * Add legacy and non legacy fixtures to test files * Update readme * Bump base req * Fix style * Mark legacy metrics * Fix watchdog counter name * document prometheus metrics in metadata csv * Fix metadata csv * Add e2e test * FIx style * Fix metadata format for validation * Flaky metrics * Only support openmetrics in latest api v3 * Fix test imports * Enable Openmetrics option by default * Fix import * Fix style * Update readme * Update config stats_url wording * Fix envoy import * Remove py27 for openmetrics version * Openmetrics endpoint should be optional * Account for flaky metrics * Document service checks * Use unique name * Update envoy/tests/legacy/test_bench.py Co-authored-by: Ofek Lev <[email protected]> * Move metrics map to metrics.py * Update with feedback * Use lambda * simplify match * Refactor metadata utils * Support metadata collection in V2 * Use urlunparse * Reintroduce legacy config options as hidden Co-authored-by: Ofek Lev <[email protected]>
What does this PR do?
This PR adds support to collect metrics via Openmetrics/Prometheus
Py3 only.
Waiting on #10753
Metrics:
Motivation
Benefits to OpenmetricsV2:
Other:
Istio users can benefit from collecting Prometheus metrics because Envoy admin port is reserved.
Additional Notes
Review checklist (to be filled by reviewers)
changelog/andintegration/labels attached