Change ContextContainer to lazily initialize providers

[pgayvallet]

(obligatory comment: this is not a failed rebase)

Summary

Fix #84763
Enhancement of #78957

For performances purposes, we adapted core's request handler context to lazily instantiate its properties on access (#78957). This PR goes a step further, by ensuring that all registered context providers will only be invoked/instantiated when effectively used by the request handler.

This was done by changing the ContextContainer implementation to lazily create the context parts when they are individually accessed instead of eagerly instantiating them all before handler execution.

Due to the allowed asynchronous nature of some context providers, this implies that access to context parts within a handler has now to be done asynchronously.

E.g

const handler = async (ctx, req, res) {
   const esClient = ctx.core.elasticsearch.client;
   // do something async with the client
   return res.ok()
}

Now becomes

const handler = (ctx, req, res) {
   const esClient = (await ctx.core).elasticsearch.client;
   // do something async with the client
   return res.ok()
}

Or

const handler = async (ctx, req, res) {
   const resolved = await ctx.resolve(['core']);
   const esClient = resolved.core.elasticsearch.client;
   // do something async with the client
   return res.ok()
}

This PR:

• implements the changes withint the ContextContainer
• update core RequestHandlerContext accordingly
• expose a new CustomRequestHandlerContext type to perform any potential sync context provider -> async context handler type conversion
• update all usages of request handler contexts in the code base and associated test files.

Notes to reviewers

The changes, even if impacting a lot of files, were for the most trivial, and given the overall excellent TS type checks around usages of contexts within request handlers (in addition to our FTR test coverage), the potential risks of having missed any usages is evaluated as fairly low.

[joeypoon]

one tiny thing for OLM otherwise 👌 ✅

[joeypoon]

I think this comment should stay above line 26?

[tsullivan]

App Services changes LGTM

[TinaHeiligers]

Do we want to reject everything on the first rejected promise and/or error?

[pgayvallet]

Yea, I think we do. Managing partially resolved contexts seems too complex and not really necessary given there's no real risk given the underlying providers. This is why I felt a simple Promise.all was good enough.

Also, that was the behavior when the context was implicitly awaited before the route handler execution.

[ari-aviran]

Approving for cloud security posture

[dasansol92]

Is this import * needed if we only use the IndexRequest ?

[pgayvallet]

We usually import * for the ES client types in the whole codebase, given this has no real impact (especially for test files), but I can use named imports instead if you prefer.

[dasansol92]

OLM changes LGTM!

[pgayvallet]

Some benchmarking:

Kibana instances: GCP n2d-standard-8 (created via https://github.com/elastic/kibana-remote-dev)
Elasticsearch: Cloud's default ES cluster configuration, same region as the GCP instances.
Load test machine: local (not ideal but that's a start until I find out how to reach the remote-dev instance from another VM created manually)
tool: kibana-load-testing - DemoJourney

Given the testing environment wasn't perfect, the results are to be taken with a grain of salt, but each scenario was ran 3 times with fairly consistent results.

Overall summary

Average response times

Response time distribution	reference - 200 users	PR - 200 users	reference - 300 users	PR - 300 users
t < 800 ms	41%	58%	27%	33%
800 ms < t < 1200 ms	21%	21%	11%	24%
t > 1200 ms	38%	21%	62%	43%

Observation: not the best indicator, but the PR's results are plain better.

Overall statistics

Scenario	Min	50th pct	75th pct	95th pct	99th pct	Mean
Reference - 200 users	230	965	1556	3079	4684	1215
PR - 200 users	230	692	1093	2138	3617	903
Reference - 300 users	2	1593	2445	5206	8855	2001
PR - 300 users	3	1082	1671	3408	8729	1449

• Min response time not affected
• Mean response time cut by 25%
• 25th, 50th, 75th, 95th percentile cut by 25% to 33%
• 99th percentile less impacted

Overall this looks very positive.

Detailed results

Reference - 200 users

PR - 200 users

Reference - 300 users

PR - 300 users

Remarks: the failures on the 300 users scenarios are caused by reaching limits on ssh-forwarding and are not real Kibana failures

[TinaHeiligers]

LGTM

[kibana-ci]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: b958776
• Storybooks Preview
• Cloud Deployment
• Webpack Bundle Analyzer

Test Failures

• [job] [logs] Security Solution Tests / Timeline search and filters Update kqlMode for timeline "before all" hook for "should be able to update timeline kqlMode with filter"

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
core	970	977	+7
data	2803	2802	-1
screenshotMode	14	13	-1
securitySolution	45	46	+1
total			+6

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
kibana	334	335	+1
observability	31	30	-1
total			-0

Unknown metric groups

API count

id	before	after	diff
@kbn/utility-types	27	28	+1
core	2515	2524	+9
data	3415	3414	-1
screenshotMode	33	32	-1
securitySolution	45	46	+1
total			+9

History

• 💚 Build #39833 succeeded b483dcc
• 💚 Build #39728 succeeded c9941df
• 💚 Build #39686 succeeded 301ec95
• 💔 Build #39680 failed a40bcdf
• 💔 Build #39617 failed 4937199
• 💔 Build #39609 failed 861b5ee

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream