Memory leak and high cpu when using symlinks

[RuneKR]

Describe the bug
Memory leak and high cpu when using symlinks

To Reproduce
I am using storybook v3.4.11. I can provide an example repo on request.

I have a monorepo setup:

• node_modules (root of symlink)
• packages/node_modules/my-scope/
  • packagea
    • node_modules (symlink)
  • packageb
    • node_modules (symlink)

Expected behavior
When i use symlinks the CPU goes to 99% and uses more than 2 GB of memory! Removing the symlinks and running start-storybook will result in normal behavior of 30 % cpu and 500 MB of memory.

System:

• OS: Windows 10 and also Mac OS

[RuneKR]

I suspect it might be related to my require.context:

require.context('../packages/node_modules/@my-scope/', true, /^(?!.*(?:node_modules)).*\.story.tsx$/)

since i will have a node_modules folder under @my-scope/{package}

[igor-dv]

Can we help in some way? It's not directly related to Storybook, but if you have some reproduction I will try to take a look.

[RuneKR]

Hi,

Awesome to hear. I will get something setup 22th of this month!

It doesn't look like to be caused by the fact that the modules are symlinked but rather that node_modules gets scanned even though they are a part of the regex ignore

[RuneKR]

Sorry for the delay - https://github.com/RuneKR/bad-performance

I did come up with a solution, but i am keen to hear your thoughts!

[igor-dv]

Will check this soon

[RuneKR]

Thank you!

[stale]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[stale]

Hey there, it's me again! I am going close this issue to help our maintainers focus on the current development roadmap instead. If the issue mentioned is still a concern, please open a new ticket and mention this old one. Cheers and thanks for using Storybook!

[remorses]

I am facing the same issue when using pnpm, pnpm uses symlinks for every dependency

I think storybook recompiles the same package more than once because it is using the symlinked path to identify a module, it should instead use the real resolved path

[remorses]

Or maybe the issue is require.context

[shilman]

@remorses there are a few possibly related PRs that just got merged into 6.1. does this fix the problem? https://github.com/storybookjs/storybook/pulls?q=is%3Apr+author%3Amerceyz+is%3Aclosed

[RuneKR]

I found my issue to be related to webpack watch and babel transpile. I had to ignore the node module folders in the symlinked paths manually:

// watch configuration
config.watchOptions.ignored = ['./node_modules', './packages/**/node_modules']

// babel configuration for each symlinked path
config.module.rules[0].exclude.push('./packages/my-path/node_modules')
config.module.rules[0].exclude.push('./packages/my-other-path/node_modules')

[remorses]

I resolved my issue by making config.resolve.symlinks true in the storybook webpack config

Compiled modules went from 20k to 6k