Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scraper fails on prometheus NaN metrics #23287

Closed
andig opened this issue Apr 21, 2022 · 3 comments
Closed

Scraper fails on prometheus NaN metrics #23287

andig opened this issue Apr 21, 2022 · 3 comments

Comments

@andig
Copy link

andig commented Apr 21, 2022

Steps to reproduce:

  1. Add a scrape target the has NaN values
  2. Scrape it

Expected behavior:
No errors

Actual behavior:
Log file:

ts=2022-04-21T06:05:39.347453Z lvl=error msg="Unable to marshal json" log_id=0_zV5qh0000 error="json: unsupported value: NaN"

At this point, no metric is collected.

It seems this behaviour has recently been introduced with the prometheus golang library in 1.12.1: prometheus/client_golang#981. According to spec (see that issue), NaN is a value.

A previous issue for Inf values was fixed in #10490.

Environment info:

InfluxDB 2.0.9
Server: d1233b7
Frontend: 9c89e22

May be related to #4089

@andig andig changed the title Scraper fails on NaN metrics Scraper fails on prometheus NaN metrics Apr 21, 2022
andig added a commit to evcc-io/evcc that referenced this issue Apr 21, 2022
@andig
Copy link
Author

andig commented Apr 30, 2022

No feedback? This would impact anyone scraping prometheus metrics produced by a recent prometheus library?

dontbyte pushed a commit to dontbyte/evcc that referenced this issue Aug 2, 2022
@Sniper91
Copy link

Sniper91 commented Sep 8, 2022

influxdb don't support NaN by design.
When encoding float value, it uses NaN as a sentinel to mark the end of data block.
https://github.com/influxdata/influxdb/blob/master/tsdb/engine/tsm1/float.go

@andig
Copy link
Author

andig commented Sep 8, 2022

influxdb don't support NaN by design.

...and that exactly is the problem when using Influx to scrape prometheus. It seems though, that prometheus has reverted the change for enabling NaNs.

@andig andig closed this as completed Jan 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants