Request can be affinitized to destinations

[alnikola]

Request can be affinitized to destinations by an affinity key extracted from cookie or custom header. Affinity is active only when load balancing is enabled and it can be configured for each backend independently.
One request can have affinity to a single destination or to a set. It enables to scenarios where a session needs to be affinitized to a pool of load-balanced destinations.

Fixes #45

Session Affinity

Concept

Session affinity is a mechanism to bind (affinitize) a causally related request sequence to the destination handled the first request when the load is balanced among several destinations. It is useful in scenarios where the most requests in a sequence work with the same data and the cost of data access differs for different nodes (destinations) handling requests. The most common example is a transient caching (e.g. in-memory) where the first request fetches data from a slower persistent storage into a fast local cache and the others work only with the cached data thus increasing throughput.

Affinity Key

Request to destination affinity is established via the affinity key identifying the target destination. That key can be stored on different request parts depending on the given session affinity implementation, but each request cannot have more than one such key.

The current design doesn't require for key to uniquely identify the single affinitized destination, it's allowed to establish affinity to a destination group. In such case, the exact destination to handle the given request will be determined by the load balancer.

Establishing a new affinity or resolution of the existed one

Once a request arrives and gets routed to a backend with enabled session affinity, the proxy automatically decides whether a new affinity should be established or an existing one needs to be resolved based on the presence and validity of an affinity key on the request as follows:

1. Request doesn't contain a key. Resolution is skipped and a new affinity will be establish to the destination chosen by the load balancer
2. Affinity key is found on the request and valid. The affinity mechanism tries to find all healthy destinations matching to the key, and if succeeded it passes the request down the pipeline. If multiple affinitized destinations are found, the load balancer is invoked to choose the single target destination, otherwise, if only one destination is found, the load balancer is skipped.
3. Affinity key is invalid or no healthy affinitized destinations found. It's treated as a failure to be handled by a failure policy explained below

If a new affinity was established for the request, the affinity key gets attached to a response where exact key representation and location depends on the implementation. Currently, there are two built-in providers storing the key on a cookie or custom header. Once the response gets delivered to the downstream client, it's the client responsibility to attach the key to all following requests in the same session. Further, when the next request carrying the key arrives to the proxy, it resolves the existing affinity, but affinity key does not get again attached to the response. Thus, only the first response carries the affinity key.

Affinity failure policy

If the affinity key cannot be decoded or no target destination found, it's considered as a failure, and an affinity failure policy is called to handle it. The policy has the full access to HttpContext and can send response to the client by itself. It returns a boolean value indicating whether the request processing can proceed further or must be terminated.

Configuration

Session affinity implemented by services and middleware. Services are added via AddSessionAffinityProvider(). Middleware is added via UseAffinitizedDestinationLookup() and UseRequestAffinitizer(). Note, the first method must be called before adding LoadBalancingMiddleware whereas the second must be called after.

Further, session affinity is configured per backend as in the example below:

    "Backends": {
      "backend1": {
        "LoadBalancing": {
          "Mode": "Random"
        },
        "SessionAffinity": {
          "Enabled": "true",
          "Mode": "Cookie",
          "AffinityFailurePolicy": "Redistribute"
        },
        "Metadata": {
          "CustomHealth": "false"
        },
        "Destinations": {
          "backend1/destination1": {
            "Address": "https://localhost:10000/"
          },
          "backend1/destination2": {
            "Address": "http://localhost:10010/"
          }
        }
      }

[alnikola]

PR is created to get an early feedback on the approach and not completed yet. The following is still in progress:

1. Affinity key protection
2. Affinitization failure modes
3. Tests
4. Logging

[Tratcher]

Please file an issue for the cross-destination affinity provider we discussed.

[Tratcher]

I noticed an issue with setting the affinity cookie before calling HttpProxy. HttpProxy copies back response headers using TryAdd

reverse-proxy/src/ReverseProxy/Service/Proxy/HttpProxy.cs

Line 453 in e0ff67b

destination.TryAdd(header.Key, new StringValues(header.Value.ToArray()));

This means if we add a Set-Cookie response header for affinity then any Set-Cookie response headers from the destination won't be sent back to the client.

I think HttpProxy is using the wrong API regardless. It should be using Set/Indexer or Append. Set would make sense for replacing things like the default server header. Append would make sense for our cookie scenario. Try changing it to Append for now and we'll see what happens.

[Tratcher]

Making good progress. I didn't review everything, but I wanted to send you the feedback I have so far. I'll be off tomorrow.

[Tratcher]

Failure modes look like the next section to work on?

[alnikola]

Failure modes look like the next section to work on?

@Tratcher I implemented handling of missing affinitized destinations with PickRandom and ReturnError strategies. That failure type looks like the only one we want to currently handle in session affinity logic.

[Tratcher]

Incremental feedback, mostly style and flow control concerns.

[alnikola]

It needs to be yet decided whether we really need a destination lookup table in the default implementation.

[alnikola]

PR is ready for review.

There is still one open question above. However, even if we decide to implement it by default, not sure if it should be part of this PR since it already quite big.

[alnikola]

It was decided to separately implement a destination lookup table as a concrete application of a generic component-specific data storage discussed in #229.

[Tratcher]

This is getting much more polished. Good job.

[Tratcher]

Polish

[Tratcher]

Design: What if the failure handler needs to remove a some affinity relationship state? E.g. if the affinity is tracked in some other table? It doesn't have access to the provider's implementation details. Would we add a cleanup method to the provider interface and then pass that provider in here?

Or do we always rely on AffinitizeRequest to overwrite any old state?

[alnikola]

Yes, it might be needed in the future to implement cross-backend affinity failure handling, but I believe we shouldn't mix it in this PR.

Regarding the contract, I'd prefer to pass a more specialized interface (e.g. IAffinityStateCleaner) not the full ISeesionAffinityProvder to the failure policy to avoid too strong coupling.

[Tratcher]

Very nice!

Next can do a PR for a one page doc on how to use it?

• What it's for
• How to turn it on in config
• Which modes are available and how they work
• What happens when affinity fails

No need to cover custom providers yet.