Question: How can I share info between servers?

[Nikoms]

Hello,

I'd like to use cockatiel but my app is running on multiple servers. Is there a way to share a state (for circuit breaker example) between different servers? If an API is down, I would like all of them to stop sending traffic. I think everything is in-memory here, but I'd like to know if there is a way to "plug" another implementation like redis.

If it's not implemented, is it because it's a bad idea? 😅

Thank you :)

[ValeryShvyndzikau]

Hello @connor4312, @novemberborn,
I have almost the same question, I'm considering Cockatiel for the AWS serverless architecture and I need to share Circuit Breaker state between Lambda invocations. It can be DynamoDB as a storage. For example, there is a such capability Opossum library provides and can be achieved in this way:
const breakerState = breaker.toJSON();
const breaker = new CircuitBreaker({state: state});

[connor4312]

I think the needs for hydrating state, as you say in serverless, are different from sharing state.

Hydrating state is easy and I think I'll implement that for this issue.

Sharing state is very hard. You need a data store that can read and write pretty fast since we'd need to do a read pretty often. You also need to be able to do distributed locking to prevent multiple servers doing half-open tries at once. And this is an external data store, so you need failure handling for that datastore too and end up with a chicken and egg problem.

This would mostly be useful for the case of large numbers of servers sending a small number of requests, which generally falls in the realm of serverless for which I think hydration is a better solution.

[ghost91-]

@connor4312 1d00074 implements (de)serializing the state of the CircuitBreakerPolicy class, but wouldn't we also need to do that for the state of the IBreaker implementations?

My first idea would be to add hydrate and toJSON methods or similar to the IBreaker interface that the implementations need to implement. And they would be called by the CircuitBreakerPolicy at the appropriate places, allowing the CircuitBreakerPolicy to embed the breaker state in its own serialized state.

Alternatively, we could also just use the same approach that is used for CircuitBreakerPolicy also for IBreaker implementatiion (add a toJSON method, allow passing in an initialState).

[connor4312]

Good catch, fixed in 3.2.1. Opted to define a gettable/settable state on the breakers to avoid consumers having to do extra work to pass in initialState