Feature request: select key, value pairs in variable queries

[TimUnderhay]

First, thank you @marcusolsson for this fantastic plugin. Top notch!

I'd like to request a feature, namely the ability to select multiple values in dashboard variable queries, and give the ability to rename their values to __value and __text, so that the selection may be a key / value pair. Currently, only a single value is selectable in the variable query options, i.e. the + / - buttons are missing.

Scenario: My JSON query returns an array of key / value pairs, such as [ { "mykey": "somekey", "myvalue": "somevalue"} ]. I want my dashboard selector to show "somevalue", but when I select a value, I want __value to be set to "somekey". Grafana supports this for values named __value and __text.

Which enters the second part: once I've defined my field, the ability to rename it is required, ideally to some arbitrary value of my choosing (similar to a SELECT mykey as __value, myvalue as __text).

Please see this blog post for how this scenario is accomplished using mysql: https://grafana.com/blog/2019/07/17/ask-us-anything-how-to-alias-dashboard-variables-in-grafana-in-sql/

Thank you!

[marcusolsson]

Hi, and thanks for the kind words! ❤️

I think it would make sense to be able to use a different field as a label for the values in a dashboard variable, and I'd like to add it.

That being said, I'd like to explore different ways to configure this. Instead of using keywords like __value, __text, time, or metric, I'd like to make this more explicit so it's easier for users to discover.

My initial thought is to have the user select the field to use for the value and for the label (text) using drop-downs. I believe this would make renaming redundant, since the field could have any name. Would that work for your use case?

[TimUnderhay]

Thanks for the quick reply! That's a good suggestion, though after giving it a bit of a think, that approach might be a bit limiting, as well.

With SQL queries, again as an example, I can write SELECT something AS somethingElse, and this works well to rename a field no matter whether you're defining a variable query or a panel query. It would be handy to have an optional field, say to the right of the type dropdown, where one could just type in the name that the field should be called. Should this field not be defined, the plugin would then retain the current behaviour of not renaming the field.

This would accommodate both the variable query and panel query scenarios, with the added benefit of no longer needing to use a transform in order to rename a field in a panel query.

Also, it would accommodate the variable query scenario slightly better than using predefined dropdowns, as AFAIK, there are only those limited variables. This would allow for adding as many arbitrary fields as desired to those variable queries. And as variable queries don't have the ability to apply transformations, it would accommodate more scenarios than just those few predefined in the dropdown.

[marcusolsson]

This would allow for adding as many arbitrary fields as desired to those variable queries.

I'm not sure I understand this. When would you want to add an arbitrary number of fields to a variable query? Could you give me an example? AFAIK there wouldn't be any use for anything else than a a value field, and an optional label field?

Regarding renaming fields—Earlier versions of the plugin actually had support for aliasing fields, but I decided to remove the feature since it's now possible to do this in Grafana itself. Although I'll be the first one to admit that it's not as user-friendly, since it's a feature that all data source would benefit from I'd prefer to improve this upstream rather than reimplement it. That being said, if there are enough people who want to bring the alias feature back, I'll definitely consider it.

However, I don't think(?) that aliases and explicitly configuring value/label fields for variable queries are dependent on each other? I think that we could implement the value/label field selection while we discuss whether to bring aliases back in a separate issue?

My main reason for not wanting to use "magic keywords", like __value and __text, for core features is that they require the user to read the documentation to even know they exist. I'd like for users—as much as possible—to be able to understand and use the data source without having to consult the documentation. Other than that, I'm totally open to suggestions on how to actually configure it.

Hope I'm making sense here :)

[TimUnderhay]

A fair question :). Forgive me if this still doesn't make sense, as I've only been using Grafana for about a week, but I'll take a stab at it.

Additional / arbitrary fields would be helpful if the user wanted to reference them from within panel queries, such as in a SELECT or a WHERE clause. RESTful API's can return any number of data points, and in a given query they may prove effectual in helping to narrow a query further or define additional fields to extract, join on, or display.

Re: "I think that we could implement the value/label field selection while we discuss whether to bring aliases back in a separate issue?", it pertains here IMHO because transformations aren't available within a variable query. Users would either be locked into using whatever the field is keyed as in the JSON, or whatever the predefined options in the dropdown are. If one wanted to preload data for panel queries, as in the last paragraph, it might be limiting.

There's another reason why native aliases would be helpful: absent the ability to alias natively, field name collisions are possible. Take the following JSON structure:

{ "pet" : { "name": "Fido" }, "owner": { "name": "Joe" } }

If one wanted to extract both pet.name and owner.name, I don't believe this would be possible as they would both be keyed as name, though correct me if I'm wrong.

On your last point, a few possible ideas to address your concerns might be:

1. Add a placeholder to the input field, showing something like __value, __text, etc
2. and / or placing a handy link to the documentation page next to the input field (if that's indeed possible with Grafana's plugin architecture)
3. A tooltip summarising these fields

Cheers

[marcusolsson]

Additional / arbitrary fields would be helpful if the user wanted to reference them from within panel queries

It sounds like you'd like to pass a multi-dimensional value within variable, essentially an object, to the panel query. AFAIK this isn't supported by Grafana. The API only allows plugins to send a single value (string or number) and an optional label for that value.

because transformations aren't available within a variable query

This is a very valid reason for why we should allow aliases in the queries. Name collisions shouldn't be a problem, as Grafana makes them unique by adding a sequence number to the field name. Either way, you make good points on why we should reintroduce aliases.

Even if we would introduce aliases, I'd still like to explore other options to configure an optional text label for variable queries. Indeed aliases would be necessary if we were to go with the __value and __text keywords, though I'd still like to separate the alias discussion into a separate issue as I think that it's a valid feature request on its own. Would you mind creating the issue?

As for the configuration of variable labels, I'd like to experiment a little first. I'm thinking that this could set a precedent for how we implement #36 which would require similar __metric configuration. In both cases, we're looking to configure an optional "role" for a field. This feels similar to how Grafana itself lets you configure annotations:

[sasha-blip]

Hi @marcusolsson ,

I could really benefit from this feature. Is there any timeline on when this feature would be available?

[marcusolsson]

I've just published v1.1.0 to the marketplace, which includes experimental support for this. Would you mind checking it out and letting me know whether it addresses your use case?

[TimUnderhay]

Hi @marcusolsson sorry for the late reply. Thank you! I'll be giving it a go and will report back in a few days, hopefully.

[TimUnderhay]

@marcusolsson I've given it a go but ran into an issue with #98 which is blocking me from testing further.

[TimUnderhay]

Right, I worked through #98 but found that this isn't working for me thus far. It appears to only be picking up __value but not __text.

[sasha-blip]

If you are using grafana 7.1.2+ you could use global variables $__from and $__to for this. Details at https://grafana.com/docs/grafana/latest/variables/variable-types/global-variables/

…

On Thu, Apr 22, 2021, 6:57 AM Tim Underhay ***@***.***> wrote: Right, I worked through #98 <#98> but found that this isn't working for me thus far. It appears to only be picking up __value but not __text. [image: image] <https://user-images.githubusercontent.com/15734900/115642047-45144200-a2e8-11eb-9eb1-8018eb558c70.png> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#79 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATUR2IW33GNM5RUSLZQTJIDTJ53PPANCNFSM4ZHFUQSA> .

[TimUnderhay]

@sasha-blip time range isn't the issue here -- maybe you meant to comment in #98. It's the variable extraction or return that appears not to be working.

[TimUnderhay]

Sorry, I wasn't using the feature properly -- was trying to set the field names using alias -- I found the settings under experimental tab. Seems to be working!