[Lens] Optimize redundant formula function aggregations

[drewdaemon]

Describe the feature:
When a Lens formula calls the same function more than once, it is currently translated into two identical aggregations in the request to Elasticsearch. Elasticsearch does not optimize this, instead performing the exact same work as many times as there are aggregations.

#131875 added the flexibility to use a single Elasticsearch aggregation to power multiple Lens dimensions. It also introduced an expression optimization hook on the Operation class. We should be able to use this groundwork to merge all redundant formula function calls into a single aggregation request to Elasticsearch. This will improve performance and lessen cluster load.

Describe a specific use case for the feature:

Logistic function

[elasticmachine]

Pinging @elastic/kibana-vis-editors @elastic/kibana-vis-editors-external (Team:VisEditors)

[drewdaemon]

@flash1293 just to verify—this is the comprehensive list of operations that need to be optimized?

[flash1293]

Yes, that’s right. Technically an unfiltered count is “for free”, but I don’t see how this would simplify things. Also, we already have some special optimization for percentiles we should keep in mind.

[drewdaemon]

I believe the only formula function left to optimize is percentile_rank. To optimize it, we need to follow a similar pattern to what is currently done for percentile.

• implement getGroupByKey to deduplicate identical functions
• use optimizeEsAggs method to merge multiple different percentile rank values into the same aggregation as long as all other parameters match