label breakdown API

[trevorwhitney]

Is your feature request related to a problem? Please describe.
To improve the experience of exploring logs, it is useful to show a few different histograms, beyond the default aggregate by level that we have now in explore. However, in order for frontends to be able to provide this functionality, they needs a subset of indexed labels that are "interesting" or "useful". For example, breaking logs down by GUIDs is not as useful as breaking down by component or job.

It would be nice if Loki had an endpoint that leveraged the information about streams in the index to return a list of "used" labels, ordered by "usefulness"

Describe the solution you'd like
An API that returns an array of label names, ordered by usefulness, where the criteria to determine usefulness may include:

• less than a configurable number of values
• more than a configurable minimum of values (ie. 2?)
• known common labels (ie. namespace, app, cluster) are prioritzed
• redundant labels are removed (ie. job and component overlap by ~80%)
• ids, shas, and other non human readable/understandable values are removed

This endpoint needs to be quick, so the queryable time range might need to be hard coded, and it should leveraging some form of caching.

Describe alternatives you've considered
Determine this in the frontend, based on # of series being over some threshold, or a metrics query returning a 500. Problem here is the frontend still has to issue the metric query, thus not preventing the load on the backend.

Tasks

Beta Give feedback

1. Setup API plumbing for detected fields
2. Return all ingester labels
3. Filter labels <= configurable values <=
4. Figure out how to filter out labels non-human understandable values
5. Return labels from the store
6. Sort

API Contract:

POST /api/alphav1/detected_fields

    start: <timestamp>
    end: <timestamp>
    query: <string>

Response:
[ ... labels ]

[cyriltovena]

Moving to done for now but there's still some outstanding work.