Feature: Enable request hedging for WebClient #806
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hedging is a mechanism for idempotent requests to automatically start a new request after some time threshold has been reached. For example, automatically firing off a new request when the latency of an existing HTTP call is at the 95th percentile. In some kind of retry mechanism, you'd have to cancel the first request. With hedging, you instead let them all run and simply use the first one that responds. The goal is to trade off some extra network traffic for reduced tail latencies. This is something we built and have been using ourselves. We thought it would be useful to contribute back to the community.
There are two specific implementation issues that I need help from the community here to answer.
First, the implementation here uses
io.micrometer:micrometer.core
to track the histogram of response times (to calculate the percentile latency). spring-cloud-commons doesn't have a required dependency on that library; it's optional throughorg.springframework.boot:spring-boot-start-actuator
. I'm at a loss whether the right thing to do is to factor out this class with the dependency, add the dependency as required, or any other compromise (use a different library; note the dependency is needed to use the class).The second is ordering. This thing adds an
ExchangeFilterFunction
. There's no easy way to control where this gets added in the filter list relative to the filters added by, for example, load balancing. The solutions I came up with to control this require the end user to override bothLoadBalancerWebClientBuilderBeanPostProcessor
and this thing's BeanPostProcessor`.