Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving performance of RedisAdvancedClusterAsyncCommandsImpl::mget #2042

Closed
JGulbronson opened this issue Mar 11, 2022 · 1 comment
Closed
Labels
type: feature A new feature
Milestone

Comments

@JGulbronson
Copy link
Contributor

Not so much a feature request as a performance improvement request (which I'm happy to submit a PR for).

We're using Lettuce (cluster) at Faire, and recently encountered a performance issue when making MGETs with a large number (10k+) of long strings (30 characters or more), with a large shared prefix.

We noticed that we had extremely high CPU usage after the MGETs completed, while still inside the Lettuce code. We eventually traced it to this line: https://github.com/lettuce-io/lettuce-core/blob/4110f2820766c4967951639aa2b6bdd9d50466be/src/main/java/io/lettuce/core/cluster/RedisAdvancedClusterAsyncCommandsImpl.java#L307

The issue is that indexOf has very poor performance for this scenario, as checking string equality is expensive. My proposal is that we create a Map<String, Int> upfront for each value in partitioned, which will be O(n). Then we use that instead of indexOf, and do the position lookups in constant time. This should improve performance at the cost of higher memory usage, but it will get GC'd immediately after so it feels like a worthwhile tradeoff.

Happy to discuss further!

@mp911de
Copy link
Collaborator

mp911de commented Mar 11, 2022

Sounds good. Feel free to submit a pull request.

@mp911de mp911de added the type: feature A new feature label Mar 11, 2022
@mp911de mp911de added this to the 6.1.7 milestone Mar 16, 2022
mp911de pushed a commit that referenced this issue Mar 16, 2022
mp911de added a commit that referenced this issue Mar 16, 2022
Initialize collections with sizes.

Original pull request: #2044.
mp911de pushed a commit that referenced this issue Mar 16, 2022
mp911de added a commit that referenced this issue Mar 16, 2022
Initialize collections with sizes.

Original pull request: #2044.
@mp911de mp911de closed this as completed Mar 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: feature A new feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants