[EBT] Success-rate telemetry

[afharo]

Since we are taking some shortcuts in the implementation of the MVP, ideally, we should report some telemetry around the success & drop rates we are getting with the current approach so that we can adapt in the future

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[afharo]

@Bamieh as you know best about UI/Usage Counters, I'd like to bring you in to discuss how to best map the telemetryCounters$ to UI/Usage Counters.

We need to think about how to map the type (succeeded/failed/dropped), source (client or shipper name), event_type and code into counterName and counterType

It feels like type and code could be concatenated and reported as counterType. However, I'm not entirely sure about how to deal with the other 2 levels.

For Usage Counters we have domainId, so I guess we could set the source to it?
However, on the UI Counters, appName and eventName are concatenated into the counterName so we'll have shipperName:eventType in the counterName and uiCounters in the domainId.

Is that OK?

@TinaHeiligers do you think we could add any piece of logic in the indexer to split uiCounters to break the appName:eventName concatenation into separate fields?

[TinaHeiligers]

do you think we could add any piece of logic in the indexer to split uiCounters to break the appName:eventName concatenation into separate fields?

My instinct and from the little I've worked with splitting events, I think it should be possible. I'd defer to @haywoood for an expert opinion though.

[haywoood]

it should be possible

Yea, not a problem 🙂

Feel free to hit me up whenever this gets worked on if you need a second pair of eyes, or to talk through implementation specifics.

[TinaHeiligers]

@afharo, for ui_counters we don't have a counterName though, we only have a counterName for usage_counters. Am I missing something from here?

[afharo]

@afharo, for ui_counters we don't have a counterName though, we only have a counterName for usage_counters. Am I missing something from here?

Sorry! I was looking at the Usage Counters definition. I meant appName. But I think we should be fine. Nevermind :)