Enable optional trust domain label for all metrics

[graceajibade]

From the telemetry doc, there are some labels that accompany metrics, we would find it useful in our deployment to be able to set a custom label to distinguish different sets of spire-servers from each other.

To this end, we could enable the use of ServiceName and EnableServiceLabel (which is from go-metric config) in metrics.go, which would give the ability to optionally pass a ServiceName and enable that ServiceName to be turned to a label. Looking at go-metric's implementation, it looks like it always uses "service" as the label. It would be nice if we could get it to be some kind of custom label, for example support for global labels could be introduced in metrics.go.

I would be happy to implement this if it gets accepted.

[sorindumitru]

Some more details, this is to help with having multiple trust domains publish metrics to the same destination/namespace. Having the trust domain as a label makes it easier to use, rather than including it in the metric namespace.

I don't think anything else apart from the trust domain id/name is needed.

[rturner3]

That sounds ok to me. I think we would just want to make this an optional label that is disabled by default and can be enabled via a config property in the telemetry config section to avoid unintentionally increasing the volume of telemetry data emitted for existing larger deployments.

[graceajibade]

That sounds ok to me. I think we would just want to make this an optional label that is disabled by default and can be enabled via a config property in the telemetry config section to avoid unintentionally increasing the volume of telemetry data emitted for existing larger deployments.

Making it an optional label works well. I will edit the title of this issue to reflect that.