Group shard request per node in the can match phase #78164
Labels
>enhancement
:Search/Search
Search-related issues that do not fall into other categories
Team:Search
Meta label for search team
The can match phase fans out the original search request to all shards.
These individual requests have a cost:
Additionally, the broadcasting of these shard request is done on the network thread without any throttling, so search requests that target a large number of shards can disturb the cluster coordination, leading to nodes leaving the cluster.
Similar to #74648, we should move to a per-node request model for this phase. We can group shard request per node and perform the local operations for multiple shard/index at once. That would reduce the cost of coordination drastically and would leave room on data nodes to perform the operations on a separate thread pool.
The text was updated successfully, but these errors were encountered: