Enhanced Circuit Breaking Approach

[Roaster05]

This PR implements a modified approach to circuit breaking in Elasticsearch. The goal is to enhance the parent circuit breaker mechanism by ensuring it only triggers during aggregation-based search queries. The changes involve identifying specific labels associated with aggregation requests and introducing a new cluster setting to control this modified behavior.

Key Changes

1. Label Identification for Aggregation Requests

• Labels with [agg] Prefix: Certain labels that indicate aggregation-based queries have been prefixed with [agg]. This helps in identifying whether the circuit breaker should consider these labels when deciding to trip.
  • Modified labels include:
    • <reduce_aggs> to [agg]<reduce_aggs>
    • preallocate[...] to [agg]preallocate[...]
    • <reused_arrays> to [agg]<reused_arrays>
    • <agg [name]> to [agg]<agg [name]>
    • allocated_buckets to [agg]allocated_buckets

2. New Cluster Setting

• Dynamic Cluster Setting: A new boolean setting, indices.breaker.total.use_modified_circuit_breaker, has been introduced. This setting allows enabling or disabling the modified circuit breaking approach dynamically.
  • Default value: false
  • Setting is dynamic and scoped to the node.

3. Consumer Method for Setting Updates

• Setting Update Consumer: A method has been implemented to handle updates to the new cluster setting. This ensures that changes to the setting are applied in real-time without requiring a node restart.

4. HierarchyCircuitBreaker Check

• Circuit Breaker Check: In the HierarchyCircuitBreakerService, when checking if the circuit should be broken, the code now also checks the value of the new setting and the label prefix. The circuit will only break if the setting is enabled and the label has the [agg] prefix.

Configuration

• Enable Modified Circuit Breaker

  indices:
    breaker:
      total:
        use_modified_circuit_breaker: true