gNMI: Add support for static and dynamic tag mappings

[bewing]

Feature Request

Currently, the SNMP input plugin supports adding tags to measurements based on other data - it is common to add the interface description to interface counter readings with the following config

    [[inputs.snmp.table.field]]
      oid = "IF-MIB::ifDescr"
      name = "ifDescr"
      is_tag = true

Ideally, we should be able to do similar with cisco_telemetry_gnmi, via subscribing to openconfig-interfaces:/interfaces/interface/state/description to maintain an up-to-date map, and adding the tag when transmitting measurements

Proposal:

Be able to configure gNMI paths as tag-only, with additional configuration to identify the search key. We should be able to read the current value of those paths, and subscribe to updates to maintain an in-memory map to apply to measurements

Current behavior:

Only global tags are supported?

Desired behavior:

Dynamic tags based on measurement values

Use case:

When using InfluxDB as a backend, tags cannot be added to a measurement post-processing, so they need to be present in the initial reading. Being able to apply dynamic device metadata to each measurement improves the searchability of measurements (show me interface stats for all interfaces whose descriptions start with "TRANSIT", etc)

[bewing]

I'm not sure how feasible the request is in the current architecture -- I'll have to play around a bit with gNMI in my lab to see what's feasible utilizing the openconfig gnmi module.

[kb70]

Hi,

any chance to see that in a release any time soon?
That's exactly the feature I'm missing for the telemetry stuff of our Nokia SRs. Often, ie. for ports or BGP neighbors, the description can only be found in the /configure tree, not in the /state tree. So having a possibility to dynamically lookup tags in some other path would help a lot.

Thanks for your work,
Kurt

[purifp]

Any estimated date to include this feature? I really need it for iosxr telemetry

[veorlo]

Here's a workaround for this issue that will enable you to remap tags embedded in fields using the starlark processor plugin. Just replace the strings in the embedded_tags array with the ones you need.

[[processors.starlark]]
# Ensure embedded tag remapping is only done to the desired metrics
namepass = ["my_metric_name"]
source = '''
def apply(metric):
    # Paths listed in this array are copied from the metric fields to tags
    embedded_tags = ["my/embedded/tag/name1", "my/embedded/tag/name2"]
    for et in embedded_tags:
        val = metric.fields.get(et)
        if val:
            metric.tags[et] = val
    return metric
'''

Please note this duplicates the field data as tag data rather than moving the field data to tags.

[bewing]

Here's a workaround for this issue that will enable you to remap tags embedded in fields using the starlark processor plugin. Just replace the strings in the embedded_tags array with the ones you need.
...
Please note this duplicates the field data as tag data rather than moving the field data to tags.

I think this is the approach used in #8250 -- and it's not working for my device because of how the device itself reports metrics individually instead of batched/grouped.