Detected field values: Response sometimes has space between digits and units, breaking queries

[gtk-grafana]

Sometimes we get responses like this:

{"limit":1000,"values":["4.4 kB","568 kB","916 B","949 B","761 B","7.2 kB","865 B","1.7 kB","292 B","575 B","3.7 kB","17 kB","1.9 kB","866 kB","2.5 kB","2.1 kB","479 B","708 B","768 B","799 B","1.1 kB","79 kB","5.4 kB","3.2 kB","681 B","62 kB","596 B","921 B","297 B","3.3 kB","241 kB","6.4 kB","699 B","899 B","515 B","3.4 kB","550 B","2.7 kB","1.2 kB","432 B","1.5 kB","438 B","425 B","29 kB","5.2 kB","793 B","189 B","166 kB","38 kB","562 B","5.7 kB","175 kB","9.7 kB","0 B","13 kB","4.1 kB","654 B","3.1 kB","7.1 kB","30 kB","1.4 kB","409 kB","20 kB","7.6 kB","11 kB","1.8 kB","1.3 kB","504 B","2.6 kB"]}

And other times like this:

{"limit":1000,"values":["514B","116B","136B","154B","807B","908B","642B","775B"]}

Currently the spaces before the unit is making it into the query, which breaks the query as that is not valid logQL.
Please strip out white-spaces before the units and bring up to Loki folks to make sure this isn't a bug Loki side.

[gtk-grafana]

So this might be a bit more work then I was hoping in the frontend. We'll need to store the fieldType (e.g. bytes, float, duration, etc) in the JSON encoded value for the ad-hoc filter (like we currently store the parser) to only strip out white-space when the field is of type duration or bytes.

[gtk-grafana]

We're going to push on a Loki side change grafana/loki#15052 instead of needing to encode the field type with every field value for now.

[trevorwhitney]

so, after digging into this a bit more, I'm not sure what the desired behavior is? the response will reflect whatever the raw data has. if there's a space before the unit in the raw data, that's what we'll return. is the preference to be changing people's log lines, or should we look for a way to honor whichever way they've chosen to express the value in their log lines?

[gtk-grafana]

If the response returns whatever the user has in the raw data, then that needs to be queryable in Loki, right now querying 4.4 kB blows up the query, and querying 4.4kB returns the logs that contain 4.4 kB

[gtk-grafana]

Fixed by grafana/loki#15272, which will be released in loki v3.3.2 or maybe v3.4

[trevorwhitney]

It will go out in 3.3.2 (grafana/loki#15326)