Make the second layer hashmap more memory efficient #11
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
goffrie
reviewed
Oct 1, 2021
| use std::{collections::HashMap, hash::Hash, mem}; | ||
|
|
||
| #[derive(Clone, Debug, Eq, PartialEq)] | ||
| pub enum SecondLayerMap<K, V> |
There was a problem hiding this comment.
It would be nice to have a doc comment explaining for the reader what this is, and why it's there.
There was a problem hiding this comment.
I went ahead and added docs for that whole file. Let me know if everything looks good to you
|
Thanks for the fast review! I just pushed the changes for your suggestions |
|
Good stuff! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The bulk of
IndexedSignatureis taken up by the internal hashmap. The second layer of this hashmap almost always has 1 entry (when I was testing manually with over 7 million blocks, there were still only one value within +99.7% of the second layer hashmaps).These changes optimize the second layer hashmap for the common case of a single entry where a
std::mem::size_of::<SecondLayerMap<&[u8], u32>>()shows 24 bytes whilestd::mem::size_of<HashMap<&[u8], u32>>()shows 48 bytes on my machine.This should also make
.get()a bit faster with a single entry as well, but it was never a significant amount of time when I've been profiling, so I saw a less than 2% improvement on a couple benchmarks with no regressions.