perf: Do not build glob matchers repeatedly when include-exclude feature is enabled #244
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ure is enabled I've noticed in my profiles that if include-exclude feature is enabled, the call to ::iter() function is dominated by path filtering. This commit fixes that by explicitly passing in a premade Globset for a given object. Note that we're using std::sync::OnceLock to lazily store away premade Globsets, which bumps MSRV to 1.70. An alternative would be to use once_cell as an explicit dependency. For my use case, in debug builds this takes down the matching time from 650ms to 6ms. I know you shouldn't benchmark debug builds, but rust-embed::iter is on my app's initialization path and that's how I noticed it in the first place. I'd expect release timings to be somewhat similar
|
Nice! LGTM. Thanks @osiewicz. |
osiewicz
added a commit
to zed-industries/zed
that referenced
this pull request
May 11, 2024
RustEmbed repeatedly compiled regexes for handling of 'include='/'exclude' statements in a hot loop, which caused each call to Assets::iter() to take 600ms. Since it is being called twice on our startup path, that alone contributed over a second to startup time in debug builds. I've filed a PR with them pyrossh/rust-embed#244 which brings down the time for a single iter() call to 6ms.
osiewicz
added a commit
to zed-industries/zed
that referenced
this pull request
May 11, 2024
RustEmbed repeatedly compiled regexes for handling of 'include='/'exclude' statements in a hot loop, which caused each call to Assets::iter() to take 600ms. Since it is being called twice on our startup path, that alone contributed over a second to startup time in debug builds. I've filed a PR with them pyrossh/rust-embed#244 which brings down the time for a single iter() call to 6ms. Release Notes: - N/A
osiewicz
added a commit
to RemcoSmitsDev/zed
that referenced
this pull request
May 18, 2024
RustEmbed repeatedly compiled regexes for handling of 'include='/'exclude' statements in a hot loop, which caused each call to Assets::iter() to take 600ms. Since it is being called twice on our startup path, that alone contributed over a second to startup time in debug builds. I've filed a PR with them pyrossh/rust-embed#244 which brings down the time for a single iter() call to 6ms. Release Notes: - N/A
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've noticed in my profiles that if include-exclude feature is enabled, the call to ::iter() function is dominated by path filtering. This commit fixes that by explicitly passing in a premade Globset for a given object.
Note that we're using std::sync::OnceLock to lazily store away premade Globsets, which bumps MSRV to 1.70. An alternative would be to use once_cell as an explicit dependency.
For my use case, in debug builds this takes down the matching time from 650ms to 6ms. I know you shouldn't benchmark debug builds, but rust-embed::iter is on my app's initialization path and that's how I noticed it in the first place. I'd expect release timings to be somewhat similar