-
Notifications
You must be signed in to change notification settings - Fork 185
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changing baked data to use zerotrie #5064
Conversation
Have you measured impact on binary size and performance? |
While I think we should explore this, there are some factors at play that might make ZeroTrie less compelling for baked data than for blob data:
My suspicion is that despite these factors, ZeroTrie will still be faster and smaller overall, although to a lesser degree than it was in blob1 vs blob2. I would however like to see some figures. |
9305cee
to
32f6a00
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
^ Pending metrics.
Do you have metrics? |
Yes in the diffs |
This comment was marked as outdated.
This comment was marked as outdated.
You should review #5161 first, as it says in the description. |
utils/zerotrie/src/zerotrie.rs
Outdated
@@ -628,24 +629,28 @@ impl_zerotrie_subtype!( | |||
ZeroTrieSimpleAscii, | |||
String, | |||
reader::get_iter_ascii_or_panic, | |||
core::iter::Map<reader::ZeroTrieIterator<'_>, fn((Vec<u8>, usize)) -> (String, usize)>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: I'm in support of changing the impl Iterator
to a concrete type, but I don't want to restrict the concrete type to being a specific map composition like this. Please introduce a new type such as ZeroTrieAsciiIterator
or something like that so we can change the internals in the future. Safest would be to have a different type for all four concrete impls.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And these four types redirect all 76 iterator methods to the types they wrap?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair point, we can clean this up later before zerotrie 1.0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've made a #[doc(hidden)]
type alias, should be non-clickable in the docs
2368556
to
ec8a5f7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Praise: Impressive size reductions, and this is a fair comparison when converting TinyStr to ZeroTrie since those are both reference-less structures.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hash map instability was creeping in somewhere?
No description provided.