Search request resource consumption estimation

[tushar-kharbanda72]

Is your feature request related to a problem? Please describe.
#1042 aims to build back-pressure support for Search requests. This is the next phase of #1180 to make rejections more pro-active than reactive.

Describe the solution you'd like

Improve the framework added as part of (#1042) to also estimate the cost of new search queries, based on their potential resource utilisation. It can be achieved by looking at the past query patterns, with similar query constructs and the actual data on shard. This will help in building a pro-active back-pressure model for search requests, where estimates will be compared against the available resources during the admission decision for finer .

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[ansjcy]

@deshsidd @jainankitk this might be related to the cost estimation initiatives we have.

[deshsidd]

Thanks for bringing this up @ansjcy. @tushar-kharbanda72 We are actively working on a Search Query Cost Estimate proposal and we should have an RFC out shortly. This will be one of the many use-cases for the cost estimation effort. I know this issue and the linked ones were created a couple of years ago but hoping to get some traction. Will reach out to you when we have that out and would love to get your thoughts.