Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support reporting ExtraMetrics in prometheus client internally used by PrometheusReceiver in otel collector #21040

Closed
premendrasingh opened this issue Apr 18, 2023 · 11 comments
Labels

Comments

@premendrasingh
Copy link
Contributor

Component(s)

receiver/prometheus

Is your feature request related to a problem? Please describe.

We use prometheus receiver to scrape metrics, however many times the prometheus endpoints return huge number of time series. We use body_size_limit to drop data but there is no way to report for which endpoints the data has been dropped.

Prometheus client has a featureFlag options.ExtraMetrics which reports content size metric (scrape_body_size_bytes) for successful scrapes and -1 for dropped data. This metric will help us collect data and setup alert for dropped data. However the prometheus receiver does not have a way to enable this feature flag. Please suggest a solution to enable this metric - scrape_body_size_bytes

Feature flag used for collecting metric
https://github.com/prometheus/prometheus/blob/main/scrape/scrape.go#L1798-L1808

Describe the solution you'd like

New scrape manager is created in PrometheusReceiver here, it can be enhanced to pass a flag to enable ExtraMetrics in Prometheus client.

For example something like
r.scrapeManager = scrape.NewManager(&scrape.Options{PassMetadataInContext:true, ExtraMetrics:r.cfg.ReportExtraMetrics}, logger, store)

Describe alternatives you've considered

No response

Additional context

No response

@premendrasingh premendrasingh added enhancement New feature or request needs triage New item requiring triage labels Apr 18, 2023
@github-actions github-actions bot added the receiver/prometheus Prometheus receiver label Apr 18, 2023
@github-actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@JaredTan95 JaredTan95 removed the needs triage New item requiring triage label Apr 18, 2023
@JaredTan95
Copy link
Member

JaredTan95 commented Apr 18, 2023

@premendrasingh makes sense, and would you be interested in contributing this flag?

@premendrasingh
Copy link
Contributor Author

Thanks @JaredTan95 I will submit a PR for this change.

@premendrasingh
Copy link
Contributor Author

@JaredTan95 Please help review the PR #21064 Thanks

@github-actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Jul 10, 2023
@premendrasingh
Copy link
Contributor Author

@JaredTan95 Please help review PR and let me know whether any more changes are needed. Thanks.

@github-actions github-actions bot removed the Stale label Jul 11, 2023
mx-psi added a commit that referenced this issue Aug 1, 2023
 (#21763)

Feature: Enable reporting of additional metrics for Prometheus client
used in PrometheusReceiver

**Description:**
Prometheus client has a featureFlag options.ExtraMetrics which reports
content size metric
([scrape_body_size_bytes](https://github.com/prometheus/prometheus/blob/main/scrape/scrape.go#L1770))
for successful scrapes and -1 for dropped data. This metric will help us
collect data and setup alert for dropped data. However the prometheus
receiver does not have a way to enable this feature flag.

This PR passes the flag to scrape.NewManager() to enable reporting of
metric -
[scrape_body_size_bytes](https://github.com/prometheus/prometheus/blob/main/scrape/scrape.go#L1770)

**Link to tracking Issue:**
#21040 

**Testing:**
Added check for validating the value set in testdata/config.yaml

---------

Co-authored-by: Jared Tan <[email protected]>
Co-authored-by: Pablo Baeyens <[email protected]>
@github-actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Sep 11, 2023
@mx-psi mx-psi removed the Stale label Sep 11, 2023
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Nov 13, 2023
@dashpole dashpole removed the Stale label Nov 13, 2023
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Jan 15, 2024
@dashpole dashpole removed the Stale label Jan 16, 2024
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Mar 18, 2024
Copy link
Contributor

This issue has been closed as inactive because it has been stale for 120 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants