[Lens] Telemetry for embeddable component usage

[flash1293]

Lens offers a component to be embedded into other solutions now. To track the usage in the final product, there should be a counter for these. The embeddable component should be extended by a prop app: string which identifies the calling application.

@afharo is this compatible with how we structure telemetry today? Ideally there should be one counter per app, but we don't know all apps in advance. is it possible to keep this part dynamic?

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[afharo]

@flash1293 we've got 2 ways to handle this:
Use a tree structure { [appId]: counter } or report an array of key-values [ {appId, counter}, ... ].

I would prefer the second approach because it avoids mapping field explosion, but it may require some work on the infra side to index it in an easy way to consume.

If you prefer to go through the first one, you can use the keyword DYNAMIC_KEY to tell the schema the keys will grow dynamically and are unknown.

I'll let @elastic/infra-telemetry share their thoughts on this :)

[flash1293]

We would probably need some post-processing to make the array consumable. For reference, the cardinality of app ids will stay low (<10)

[mindbat]

share their thoughts on this :)

Happy to! 😀

I suppose I have a slight preference for the first, simpler version (single hash-map of appId -> counter), as that's probably the easiest to query, and the easiest to index. With the caveat that the threat of mapping explosion is real, if this data is going to end up in the main telemetry index.

If this data will be indexed into a custom index, though, we don't have to worry so much about mapping explosion, especially if the number of appIds is bounded (as I think @flash1293 is saying?).

@afharo @flash1293 Does that make sense? Is my assumption about easier query-ability accurate?

cc @alexfrancoeur @thesmallestduck for insight on analysis/custom index suitability

[flash1293]

I think a custom index is overkill here. A map would be cool - aggregating these keys in Kibana is a big use case for us.

[stratoula]

@drewdaemon you might know, do we store this information in our ui counters?

[drewdaemon]

@stratoula I think our internal tracking dashboard basically covers this.

I'm not sure UI counters are a good fit here since this issue seems mostly to be about tracking API usage within our codebase. WDYT?

[stratoula]

I think we are ok for now, I will close it and if there is a specific need for it we can always reopen