Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[exporter/prometheusremotewrite] Hash labels using xxhash for perform…
…ance (#31385) **Description:** Improve performance in pkg/translator/prometheusremotewrite by using the same [xxhash](https://github.com/cespare/xxhash) based time series signature algorithm as Prometheus itself ([`labels.StableHash()`](https://github.com/prometheus/prometheus/blob/c6c8f63516741fd38ece5387254afee798cfc8d7/model/labels/sharding.go#L24), which is guaranteed to not change over time). I became aware of time series signature calculation being a bit of a bottleneck when profiling Grafana Mimir's OTLP endpoint. This change involves moving from a string hash, to a `uint64` one. My thesis is that if Prometheus uses this algorithm to identify time series/label sets, it should be suitable for this translation logic too. #### Hash collisions I've attempted to handle hash collisions in the same way as Prometheus: `PrometheusConverter` has a `unique` field, which is its main `map` from hash to `TimeSeries`, as well as a `conflicts` field, being its secondary `map` from hash to a slice of `TimeSeries` in case of hash collisions. If a label set should hash to an existing entry in `unique`, but not be equal to the existing entry's label set, the label set is attempted matched to the `conflicts` `map` instead. If its equal is not found among the conflicts for the hash in question either, it's added to the conflicts slice (for the hash). **Testing:** I've run `make test`/`make lint` and run the `BenchmarkFromMetrics` benchmark. Benchmark stats included below, they show an average speedup of 3.68% and an average memory reduction of 17.13%. **NB:** The benchmark stats reveal performance regressions in a few cases, because of using the `prometheusConverter` API _via_ the `FromMetrics` function. --------- Signed-off-by: Arve Knudsen <[email protected]> Co-authored-by: Anthony Mirabella <[email protected]>
- Loading branch information