Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lettuce created more than 1 native redis connection to redis cluster node #3127

Open
jmf0526 opened this issue Jan 15, 2025 · 0 comments
Open
Labels
for: team-attention An issue we need to discuss as a team to make progress status: waiting-for-triage

Comments

@jmf0526
Copy link

jmf0526 commented Jan 15, 2025

Current Behavior

In my env there are 40 redis cluster, each cluster has 20 master nodes and 20 replica nodes, so in total there are 1600 redis nodes. When my redis client application start up, there are 800+ redis connection on client side (observed by netstat), roughly 1 connection per redis master; but when the application workload is heavy , the connection number goes up to 7000+, on average there are 9 connection per one master node, and those connections never close.
There are multiple application instance on one server, I fear that if connection number increases indefinitly, tcp port could get exasted on the application server.

Java client code

The following code is used to create one redisTemplate for each redis cluster, since there are 40 cluster, so 40 redisTemplate are created in total.

ClusterTopologyRefreshoptions clusterTopologyRefreshoptions = clusterTopologyRefreshoptions.builder()
.enableAllAdaptiveRefreshTriggers()
.enablePeriodicRefresh(Duration.ofSeconds(300)).build();
                                                                                  
Clusterclientoptions clusterclientoptions = clusterclientoptions.builder().topologyRefreshoptions(ClusterTopologyRefreshoptions).build();

LettuceclientConfiguration lettuceclientconfiguration = LettucePoolingclientConfiguration.builder()
.readFrom(ReadFrom.MASTER_PREFERRED)
.clientOptions(clusterClient0ptions)
.clientResources(DefaultclientResources.builder().commandLatencyRecorder(CommandLatencyRecorder.disabled()).build())
.build();
//20master 20replica
Redisclusterconfiguration redisclusterconfiguration = createRedisclusterconfiguration();
Lettuceconnectionfactory lettuceconnectionfactory = new Lettuceconectionfactory(redisclusterconfiguration, lettuceclientconfiguration);

StringRedisTemplate redisTemplate = new StringRedisTemplate();
redisTemplate.setConnectionFactory(lettuceconnectionfactory );
redisTemplate.afterPropertiesSet();
return redisTemplate;

Expected behavior/code

Based on this comment (#860 (comment)) the tcp connection per cluster is (2 * number of nodes) + 1, but this is not true in my ev, with 40X40 nodes, the connection number should be 40*((2 * 40) +1))=3240, but in my env there are 7000+ connections.

My Question

How many native Redis connections does Lettuce create to connect to a Redis cluster node? From what I understand, Lettuce shares native Redis connections between threads. However, in my environment, I observe that up to 10+ connections can be created per cluster node.

Is there something wrong in my code?
Or is this behavior configurable?

Environment

  • Lettuce version(s): 6.1.10.RELEASE
  • Spring data redis version: 2.3.2.RELEASE
  • Redis version: 6.2.7
@tishun tishun added for: team-attention An issue we need to discuss as a team to make progress status: waiting-for-triage labels Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
for: team-attention An issue we need to discuss as a team to make progress status: waiting-for-triage
Projects
None yet
Development

No branches or pull requests

2 participants