Support multiple shared connections for Redis standalone mode in LettuceConnectionFactory

[yangty89]

Hi @mp911de, after having introduced an enhancement for Lettuce with your help, I'm currently looking at a potential performance improvement for Lettuce connection usages managed by Spring :) The general idea is to support multiple shared connections for Redis standalone mode.

By default, LettuceConnectionFactory maintains a single shared connection and will inject it into a newly created LettuceConnection instance each time the factory's getConnection method is called. It's a pretty graceful design as it gets rid of the complexity of maintaining a connection pool, and interacts with Redis in a pipelining way when serving multiple threads, which ensures a high level of performance.

Alternatively, under certain circumstances, one can also create and use a connection pool by using the native Lettuce API. A connection pool may bring about positive impacts on performance by taking advantage of the parallel processing capacity of the underlying multi-core processor. However, due to the thread confinement enforced by the connection pool, the pipelining feature mentioned above would not be well exploited.

I'm therefore thinking about a possibly new pattern of connection usage, which may take advantage of both the pipelining feature of Lettuce and the parallel processing capacity of modern processors, and get the most of both worlds. It may look like something showed below:

To verify the idea, I manually built a list of connections, and shared them between business threads. I then conducted some benchmarking of throughput(ops/s) using RedisClientBenchmark#syncSet method, comparing it with the single-shared-connection pattern and the connection pooling, and even with Jedis. The results show that the multiple-shared-connections pattern outperforms the others when it keeps a good balance between the use of the pipelining feature and the parallel processing capacity, by aligning the number of connections with the number of processors (8 in my case).

I think therefore that it might be worth considering supporting the multiple-shared-connections pattern in Spring for Redis standalone mode, as it might bring about a notable performance improvement under heavy load. Specifically, we could perhaps consider maintaining a list of shared connections in LettuceConnectionFactory. Each time the factory's getConnection method is called, a shared connection could be fetched from the list in a round-robin manner. However, we should carefully create these shared connections so that they are attached to different netty EventLoops, and thus benefit from the system's parallel processing capacity. I would like to know your opinion on this, thanks!

[mp911de]

What is the difference to configuring LettuceConnectionFactory with a connection pool? The shared connection approach is in place even when using connection pooling.

For pipelining were require dedicated connections as the result of pipelining is a single return object containing all results of the pipelining batch. A single thread that receives the pipelining result will be surprised seeing other result elements that aren't expected.

So from that perspective, pipelining is also a matter of isolation.

[yangty89]

Thanks for your reply :) I agree that for the traditional pipelining behavior where a single thread emits multiple commands and then waits for corresponding responses, a dedicated connection should be used instead of a shared connection. It is exactly what LettuceConnection does by calling its getOrCreateDedicatedConnection method.

However, in the issue I'm in fact trying discussing another kind of pipelining behavior, where a single shared connection is used to serve commands (which are non-pipelining, non-transactional and non-blocking, such as set and get) of multiple concurrent threads (illustrated by the first picture). I think it is also described in Lettuce Wiki: "Lettuce is designed to operate in a pipelining way. Multiple threads can share one connection. While one Thread may process one command, the other Thread can send a new command."

The multiple-shared-connections pattern (illustrated by the third picture) I would like to propose shares the same pipelining behavior mentioned above with the single-shared-connection pattern currently used in LettuceConnectionFactory. However, the former could further exploit the parallel processing capacity of the underlying multi-core CPU.

As for LettuceConnectionFactory, the implementation of multiple-shared-connections pattern may look like something below. It might look similar to the connection pooling approach (illustrated by the second picture), however, the connection of the latter is restricted by the thread confinement enforced by the connection pool, and thus could only serve one business thread at a time.

public class LettuceConnectionFactory implements RedisConnectionFactory, ReactiveRedisConnectionFactory,
		InitializingBean, DisposableBean, SmartLifecycle {

	private final int sharedConnectionNumber; // number of shared connections
	// a list of shared connections instead of a single shared connection
	private List<SharedConnection<byte[]>> connections = new ArrayList<>(sharedConnectionNumber);
	private int sharedConnectionIndex = 0; // round-robin index

	private SharedConnection<byte[]> getOrCreateSharedConnection() {

		return doInLock(() -> {

			if (connections.size() < sharedConnectionNumber) {
				// create a new shared connection, put it into the list, and return it.
				SharedConnection<byte[]> connection = new SharedConnection<>(this.connectionProvider);
				connections.add(connection);
				return connection;
			}

			// fetch a connection in a round-robin manner and return it 
			return this.connections.get(sharedConnectionIndex++ % sharedConnectionNumber);
		});
	}

}

In my opinion, the multiple-shared-connections pattern could take advantage of both pipelining behavior and the underlying system's parallel processing capacity, and therefore may yield a better performance result, as showed in the fourth picture. Please let me know if it's clearer and makes sense to you, thanks!

[mp911de]

Thank you for the explanation, it makes more clear what you're aiming for. I marked the ticket for team attention. Given the complexity I doubt that we will implement the feature the way you envisioned it. However, we could make getOrCreateSharedConnection and getOrCreateSharedReactiveConnection methods protected so that you can implement such a feature yourself.

[yangty89]

@mp911de Thanks for reply :) I agree that the feature may be somewhat hard to implement... I think the complexity lies partly in that we should take care of spreading evenly the shared connections between different EventLoop instances, so as to well exploit the parallelism of the underlying machine. However, the shared connections and the dedicated connections currently share the same EventExecutorChooser instance for selecting an EventLoop to bind with, which might interfere with the objective mentioned.

I'll try communicating with Netty's maintainers to see if it's possible to work out a way that may support sophisticated chooser management. In our case, employing different EventExecutorChooser instances for shared connections and dedicated connections could help solve the problem mentioned above. If this is feasible, I would like to, if possible, try to implement the feature basing on this idea, in an effective and concise way as possible. I'll inform you if I manage to make progress on it :)