Ensure APM Server makes full usage of available CPU resources

[simitt]

Previous benchmarks indicate that the APM Server performs better when scaling horizontally than vertically. This is a meta issue collecting steps necessary to ensure APM Server's throughput scales well when scaling the server's available CPU resources up.

Previous investigations identified the modelindexer to be a bottle neck, and two action items were identified:

• Move event compression outside of the modelindexer activeMu lock #9180
• Autoscale number of modelindexers to increase throughput and ensure full resource usage #9181

The improvements need to be benchmarked; the goal is to provide full resource usage and (almost) linear scaling for APM Server's up to 32GB of RAM (with aliquot CPU time) on ESS.

[kruskall]

I've reviewed the code in the linked PRs and looked at the trace files for specific benchmarks like BenchmarkModelIndexer.
I confirm the available CPU cores are being used efficiently and tasks seem to be parallelized correctly.

I could not find any issues but I noticed the flush goroutine for the active indexers (introduced in #9318) being always busy.
A goroutine being active might slow down GC and increase memory usage. I don't think that's going to be a big issue here due to the flush timer channel but I noticed some improvements and opened a PR to reduce the cpu time spent in the flush gorutine: #9760