Preserve module identity for symlinks

[gaearon]

Summary

We bumped into a problem trying to use Yarn Workspaces with Jest in the React repo.
It is the same issue as described in #3830.

In our case, we had packages/* symlinked through node_modules.

We noticed that importing the same file with the path root/packages/react/module from a test in root/packages/react/folder/__tests__ produced a different instantiation depending on how it is imported:

• Through the react/folder/module symlink.
• Or by the relative ../module path.

This is because the old resolving mechanism established resolved paths as:

• react/folder/module => repo/node_modules/react/folder/module.js
• ../module => repo/packages/react/folder/module.js

So Jest treated them as two different modules. This created very subtle and hard-to-debug errors (such as duplicate caches in arbitrary module branches depending on the initial imports that "forked" them).

With this fix, the resolving mechanism resolves both cases to repo/packages/react/folder/module.js.

It also fixes a different problem we experienced with Jest/Yarn together not compiling modules.

Test plan

Tests pass in React repo with relative paths, whereas they would fail before.

The reproducing case in https://github.com/tmeasday/jest-symlink-repro used to print twice:

Now it prints once, as expected in its README:

I added a new integration test and verified it fails on master.

I also added some logging to the test runner for this integration test because when it failed, there used to be no indication at all as to the reasons why it was failing. Removed per feedback.

Locally, I see a test failure in packages/jest-worker/src/__tests__/child.test.js but it also happens in master so is unrelated.

[gaearon]

Apparently the build on master is failing 😞

Should I send another PR to disable the failing test in the meantime?
Since then we don't know if this one works or not.

[thymikee]

@gaearon if it's jest-worker test then no need to disable it, because it won't fail on CI (which is run in band). It's a known problem we have.

[thymikee]

So basically we're hitting fs one more time, wonder how it affects perfomance.

[gaearon]

I know it was slow (nodejs/node#2680) and was optimized in nodejs/node@b488b19.

Perhaps counter-intuitively, running on React codebase (with warm caches) makes this PR slightly faster. Maybe there's some indirection in accessing the symlinks later that resolving them removes?

before the change (5 tries)
44.67 real       123.24 user         8.76 sys
44.61 real       123.07 user         8.76 sys
44.58 real       123.44 user         8.77 sys
44.62 real       122.45 user         8.72 sys
46.73 real       124.52 user         8.93 sys

after the change (5 tries)
44.08 real       121.83 user         8.14 sys
43.63 real       121.55 user         8.15 sys
44.01 real       121.95 user         8.23 sys
43.85 real       122.70 user         8.29 sys
44.33 real       122.54 user         8.16 sys

before the change again (5 tries)
44.43 real       122.47 user         8.66 sys
45.18 real       123.94 user         8.91 sys
44.95 real       123.20 user         8.84 sys
44.34 real       123.15 user         8.76 sys
44.50 real       122.88 user         8.72 sys

[thymikee]

Thanks for caring about this!

[gaearon]

It could be that it's faster for us (since we use symlinks) but slower on projects that don't use symlinks at all. ¯\(ツ)/¯

[thymikee]

This is a leftover, right? I know these integration tests don't give great error feedback, but it gets better while running inside specific directory (when debugging what's wrong)

[gaearon]

That was not obvious at all. Contributing document doesn't mention this :-(
I spent half an hour guessing why the test doesn't work without feedback.

As I wrote above:

I also added some logging to the test runner for this integration test because when it failed, there used to be no indication at all as to the reasons why it was failing.

I can remove though.

[SimenB]

Can you open up a separate issue for this? If nothing else it should be documented, but maybe we can make it better

[SimenB]

Failing test on master is tracked in #4745

[gaearon]

I also verified that it fixes a different (but related) Yarn/Jest integration problem for us.

After migrating to Yarn Workspaces, our packages was no longer being compiled because we imported them through symlinks. But with this fix, our Babel transform picks them up.

@cpojer told me he’s aware this was an issue. @BYK said GraphQL team also bumped into this.

In my testing, this PR solves it.

[cpojer]

Thanks so much @gaearon for making this happen. This definitely seems like the right fix for this issue.

I published 21.3.0-beta.3 so you can start using this fix.

[gaearon]

Sweet. Thanks. 💃

[gaearon]

Unrelated, but I noticed that in 21.3.0-beta.3 even caught errors in tests like

    fit('should throw on children for void elements', () => {
      const container = document.createElement('div');
      let caughtErr;
      try {
        ReactDOM.render(<input>children</input>, container); // throws
      } catch (err) {
        caughtErr = err;
      }
      expect(caughtErr).not.toBe(undefined);
    });

end up being reported as failures. This isn't the case in 21.2.1.

I know I could use toThrow but that doesn't let me transform message (need to massage it) before asserting.

Is this intentional or a bug? Worth making an issue?

[milesj]

Thanks so much for this. I've been plagued by it since workspaces was released!

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.