Stats about bulk sizes

[jpountz]

The size of bulk requests in bytes in one of the main factors of indexing performance. Yet we don't have any stats about it, and users don't always know what is the actual size of the bulk requests that get actually sent to Elasticsearch, which is something that is automated in many shippers. For instance Logstash allows to configure the size of batches as a number of documents, but the pipeline.batch.delay may cause incomplete batches to be sent, so it would be good to know what is the size of bulk requests that got sent to Elasticsearch in practice.

The size of bulk requests per shard is probably the most relevant metric, but we could work with the per-index size as well if that proves easier to expose.

[elasticmachine]

Pinging @elastic/es-core-features

[jbaiera]

As a note here, it might make sense to track the average bulk sizes per shard as well as the average bulk request size, since a large bulk request may be chopped down into much smaller shard level bulk operation on an index with high numbers of shards. This makes more sense to me than just tracking at the shard level since most clients are not partitioning by shard already.

[dakrone]

This has been open for quite a while, and we haven't made much progress on this due to focus in other areas. For now I'm going to close this as something we aren't planning on implementing. We can re-open it later if needed.