perf: improve package cache usage #12512
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
and separate cache for optimizer
bluwy
added
p4-important
|
|
patak-dev
reviewed
Mar 21, 2023
Comment on lines
60
to
62
| // use separate package cache for optimizer as it caches paths around node_modules | ||
| // and it's unlikely for the core Vite process to traverse into node_modules again | ||
| const packageCache: PackageCache = new Map() |
There was a problem hiding this comment.
Good idea! It would still traverse for excluded deps but that is completely orthogonal to the paths cached here.
and split cache for optimizer esm and cjs resolvers
|
/ecosystem-ci run |
|
📝 Ran ecosystem CI: Open
|
patak-dev
approved these changes
Mar 21, 2023
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR implements a smarter package caching algorithm.
Why and how
When we're given
/User/foo/bar/node_modules/my-lib/deep/deep/deep/file.js, andfile.jswants to getmy-dep/package.json. When we found a match, we use/User/.../file.jsas the cache key for thePackageData.However, we also know that if
/User/foo/bar/node_modules/my-lib/deep/another-file.jshappens to want to getmy-dep/package.jsontoo, it would get the samePackageDatatoo, because we had traversed the same directories to get there.This PR implements this, where the directories traversed while handling
/User/foo/bar/node_modules/my-lib/deep/deep/deep/file.jsare also cached, so thatanother-filecan fast-path to thePackageData.Additional context
This is a huge change. I've broken up to multiple commits so it's easier to follow along, though it seems to be too many commits. Or you can review the diff directly too 💪
It looks like
packageCacheis used for two cases before, this PR preserves this oddness since it's easier to work with it, for now.More logs and notes on perf at https://gist.github.com/bluwy/53d11b29394c7e9e399f777fae099efc
What is the purpose of this pull request?
Before submitting the PR, please make sure you do the following
fixes #123).