[TraceQL Metrics] Unexpected error when rating by rootName/rootServiceName

[joe-elliott]

Reproduced with the same images as here:
#3428

If I include a filter in the query it works:

[mdisibio]

Confirmed this is a Tempo-side bug: it's an issue with the series where rootName=empty string "". I didn't realize it but when we set labels here it treats empty as missing and removes it. Need to think a bit about how to solve this - how should we represent blanks? The concept doesn't exist in metrics.

Ran into a related scenario with nils which is another way Tempo differs. {} | rate() by(.foo) doesn't only meter spans with a .foo. It counts all spans and includes a <nil> series. Perhaps we should do similar and return <empty> ?

To muddy the waters a bit more, this is only really necessary when all labels are empty or nil. See the comment here describing the ways we can drop nil labels and produce good "prometheus-like" series.