statsd exporter currently relies on LastValue for emitting gauge metrics

[zlozano]

The new statsd exporter looks for a LastValue aggregator to emit gauge metrics. This is problematic because the NewExportPipeline factory uses simple.NewWithExactMeasure as its aggregator selector. This selector chooses the points aggregation for Observer types.

func (selectorExact) AggregatorFor(descriptor *metric.Descriptor) export.Aggregator {
	switch descriptor.MetricKind() {
	case metric.ObserverKind:
		fallthrough
	case metric.MeasureKind:
		return array.New()
	default:
		return sum.New()
	}
}

In most cases, I would imagine those using DataDog would expect Observer results to be output as traditional gauge metrics. The recommendation here says

This also allows aggregating over time to export a distribution, but the "last" value is still well defined in each collection interval for a given label set. I imagine that an exporter that wants traditional gauge semantics could just export the sum, under the assumption that count==1, and that would be all that's needed.

However, this exporter does not consider this. Also IIUC under certain circumstances (e.g. short push interval, and low traffic) there may be situations where true histogram values come in with a single count. In that case it would be impossible to determine what is desired to be exported as a gauge, vs. what is desired to be exported as a histogram.

In light of this, the two solutions I see are to make the ExactMeasure selector choose a LastValue aggregator for the metric.ObserverKind case, or provide the ability to construct our own export pipeline with out own selector. The latter is currently not possible because Exporter.labelEncoder is not present.

If a new selector is in fact required, I think it makes sense to add to this library since the exporter expects a LastValue aggregator to be used.

@jmacd I'll defer to you here on any options I probably missed since you're most familiar in this space.

[zlozano]

This was fixed by #14. Thanks!