feat: add metricscardinality to heartbeat

[alexcastilio]

Description

Add metricscardinality to heartbeat. This will give visibility on the number of time series being exposed by retina.

Related Issue

#1040

Checklist

• I have read the contributing documentation.
• I signed and signed-off the commits (git commit -S -s ...). See this documentation on signing commits.
• I have correctly attributed the author(s) of the code.
• I have tested the changes locally.
• I have followed the project's style guidelines.
• I have updated the documentation, if necessary.
• I have added tests, if applicable.

Screenshots (if applicable) or Testing Completed

Metrics exported with heartbeat:

Additional Notes

Metrics of types histogram and summary expose multiple time series during a scrape. Code is counting according to number of time series exposed at /metrics endpoint.

Ref:
https://prometheus.io/docs/concepts/metric_types/#histogram
https://prometheus.io/docs/concepts/metric_types/#summary

---

Please refer to the CONTRIBUTING.md file for more information on how to contribute to this project.

[matmerr]

I think a simpler solution may just be to enumerate over the metrics and take the length of the metrics slice

https://gist.github.com/matmerr/f2e31d9be0b5af0eec750fa944edc617#file-counter_test-go-L58

---

// count all metrics and combination of labels currently exposed on
// prometheus endpoint
metrics, err := registry.Gather()
if err != nil {
	fmt.Println("Error gathering metrics: ", err)
}

// count all metrics exposed for each metric
var metriccounter int
for _, metric := range metrics {
	metriccounter += len(metric.Metric)
}

---

where the metrics endpoint looks like this:

and in stdout we see the same, total metrics is 4

[alexcastilio]

@matmerr if I only loop through the metrics and add the length of each, this will cause a difference between the count seen through scraping and by this code. Metrics of type histogram and summary expose more time series for quantile/bucket, count and sum. Is it ok to exist this difference in the counting between what's seen in the /metrics endpoint and by summing the length of metrics in code?

[anubhabMajumdar]

@alexcastilio How much is the difference? Is it 2x or a constant number?

[alexcastilio]

@anubhabMajumdar the difference is:

• for each histogram: number of buckets + 3 (le="+Inf", _sum, _count)
• for each summary: number of quantiles + 2 (_sum, _count)

[matmerr]

The issue like @alexcastilio showed me is that histogram's create more metrics that can't be summed by just length, and our apiserver metrics latency uses histogram type.

retina/pkg/exporter/prometheusexporter.go

Line 79 in 23c34b8

func CreatePrometheusHistogramWithLinearBucketsForMetric(r prometheus.Registerer, name, desc string, start, width float64, count int) prometheus.Histogram {

Confirmed with him offline that this diff outputs the correct results. Sample endpoint, including metric of histogram type:

Results of his diff:

[anubhabMajumdar]

CombinedGatherer can be nil.

[alexcastilio]

@anubhabMajumdar addressed in #1293

[anubhabMajumdar]

GetHistogram can return nil.

[alexcastilio]

@anubhabMajumdar addressed in #1293