-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
prometheus api and +/-Inf values #10490
Comments
QuickFix, at least identifying the culprit: diff --git a/prometheus/converters.go b/prometheus/converters.go
index f3f438587..3cb9a5064 100644
--- a/prometheus/converters.go
+++ b/prometheus/converters.go
@@ -57,6 +57,7 @@ func WriteRequestToPoints(req *remote.WriteRequest) ([]models.Point, error) {
for _, s := range ts.Samples {
// skip NaN values, which are valid in Prometheus
if math.IsNaN(s.Value) {
+ // FIXME: Add some logging here?
droppedNaN = ErrNaNDropped
continue
}
@@ -66,7 +67,8 @@ func WriteRequestToPoints(req *remote.WriteRequest) ([]models.Point, error) {
fields := map[string]interface{}{fieldName: s.Value}
p, err := models.NewPoint(measurement, models.NewTags(tags), fields, t)
if err != nil {
- return nil, err
+ // FIXME: Add some logging here?
+ continue
}
points = append(points, p) |
This is tripping me up in 1.7.4 as well |
e-dard
added a commit
that referenced
this issue
Mar 21, 2019
This commit extends the Prometheus remote write endpoint to drop unsupported Prometheus values, rather than reject the entire batch. InfluxDB does not support NaN, -Inf or +Inf, but Prometheus does. The remote write endpoint will now drop these and write valid values in the provided batch. If the user enabled write trace logging (`[http] write-tracing = true`) then summaries of any dropped values within a batch will be logged. If a batch of values contains any values that are subsequently dropped, the returned status code will be `204`.
Fixed via #12813 |
e-dard
added a commit
that referenced
this issue
Apr 4, 2019
This commit extends the Prometheus remote write endpoint to drop unsupported Prometheus values, rather than reject the entire batch. InfluxDB does not support NaN, -Inf or +Inf, but Prometheus does. The remote write endpoint will now drop these and write valid values in the provided batch. If the user enabled write trace logging (`[http] write-tracing = true`) then summaries of any dropped values within a batch will be logged. If a batch of values contains any values that are subsequently dropped, the returned status code will be `204`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
System info: [Include InfluxDB version, operating system name, and other relevant details
InfluxDB v1.7.0 (git: 1.7 dac4c6f)
Steps to reproduce:
Send +/-Inf values with prometheus' remote_write.
Expected behavior: [What you expected to happen]
Ignore +/-Inf values
Actual behavior: [What actually happened]
The whole batch fails.
Additional info: [Include gist of relevant config, logs, etc.]
Here is an example setup with docker-compose:
https://gist.github.com/datamuc/d6085cc4c4190619796f0ea5a8661dce
All metrics sent in this POST request are lost because of a single +Inf value.
The text was updated successfully, but these errors were encountered: