Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scoped mocks failing with pnpm but working with npm #302

Closed
darcyrush opened this issue May 3, 2024 · 18 comments · Fixed by #315
Closed

Scoped mocks failing with pnpm but working with npm #302

darcyrush opened this issue May 3, 2024 · 18 comments · Fixed by #315

Comments

@darcyrush
Copy link

Ok, this is a was a weird issue to track down, but I have found an issue with either esmock, pnpm/npm or tsimp. I realize that's a lot of options for failure, but I thought I would start here since esmock reliably reproduces the issue.

Basically when mocking a scoped package installed via pnpm, the mock doesn't take effect. When the package is installed via npm the mock takes effect as expected.

Here is a minimal reproduction repository; https://github.com/darcyrush/esmock-scoped-package

# Works
rm -fr node_modules/ && npm i && npm run test

# Fails
rm -fr node_modules/ && pnpm i && npm run test

I haven't tested yarn as I'm not familiar with it, but I imagine it will work as it uses the same install logic as npm. (pnpm uses symlinking of packages)

Node 21.7.3
Ubuntu 22.04 Linux 5.15.0-105-generic x86_64
@iambumblehead
Copy link
Owner

Thanks for opening the issue. I don't use pnpm myself and the person who added pnpm support hasn't been around lately @koshic

The main file using pnpm resolver is here https://github.com/iambumblehead/esmock/blob/main/src/pnpResolver.js

@iambumblehead
Copy link
Owner

iambumblehead commented May 3, 2024

the demo you made looks great and would probably be a good thing to copy into the esmock tests. It needs this, however,

diff --git a/package.json b/package.json
index bcd50f7..1cc3c44 100644
--- a/package.json
+++ b/package.json
@@ -5,7 +5,7 @@
     "test": "TSIMP_PROJECT=./test/tsconfig.json node --import tsimp/import --test-reporter spec --test 'test/example.test.ts'"
   },
   "dependencies": {
-    "@nestjs/core": "^10.3.8"
+    "@nestjs/core": "^10.3.8",
     "@nestjs/platform-express": "^10.3.8"
   },
   "devDependencies": {

There are many esmock tests, but currently no tests for pnpm specifically

@iambumblehead
Copy link
Owner

failed test output here looks like this btw

> test
> TSIMP_PROJECT=./test/tsconfig.json node --import tsimp/import --test-reporter spec --test 'test/example.test.ts'

[Nest] 48166  - 2024/05/03 7:56:54     LOG [NestFactory] Starting Nest application...
[Nest] 48166  - 2024/05/03 7:56:54   ERROR [ExceptionHandler] metatype is not a constructor
TypeError: metatype is not a constructor
    at Injector.instantiateClass (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/injector.js:365:19)
    at callback (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/injector.js:65:45)
    at async Injector.resolveConstructorParams (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/injector.js:144:24)
    at async Injector.loadInstance (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/injector.js:70:13)
    at async Injector.loadProvider (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/injector.js:97:9)
    at async /home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/instance-loader.js:56:13
    at async Promise.all (index 0)
    at async InstanceLoader.createInstancesOfProviders (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/instance-loader.js:55:9)
    at async /home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/instance-loader.js:40:13
    at async Promise.all (index 1)
✖ test/example.test.ts (2044.283154ms)
  'test failed'

@iambumblehead
Copy link
Owner

@darcyrush both conditions at this pnpResolver.js line fail

const pnpapi = process.versions.pnp && (await import('pnpapi')).default

process.versions.pnp is undefined and pnpapi is not found when it is imported. Any ideas how to resolve?

@koshic
Copy link
Collaborator

koshic commented May 3, 2024

@iambumblehead "pnp" (yarn feature, no node_modules on the disk) != "pnpm" (package manager), so undefined is expected.

@iambumblehead
Copy link
Owner

iambumblehead commented May 3, 2024

'nothing to add here, but any thoughts @darcyrush

(I don't know enough about these tools to offer any assistance or input)

@iambumblehead
Copy link
Owner

iambumblehead commented May 3, 2024

adding console.log to esmockModule.js shows the following (differing) paths are resolved for npm vs pnpm

pnpm

{
  moduleId: '../src/example.js',
  fileURL: 'file:///home/bumble/software/esmock-scoped-package/src/example.ts'
}
{
  moduleId: '@nestjs/core',
  fileURL: 'file:///home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/index.js'
}

npm

{
  moduleId: '../src/example.js',
  fileURL: 'file:///home/bumble/software/esmock-scoped-package/src/example.ts'
}
{
  moduleId: '@nestjs/core',
  fileURL: 'file:///home/bumble/software/esmock-scoped-package/node_modules/@nestjs/core/index.js'
}

changing the resolver removing realpath conversion; tests still fail with the same error TypeError: metatype is not a constructor

pnpm

{
  moduleId: '../src/example.js',
  fileURL: 'file:///home/bumble/software/esmock-scoped-package/src/example.ts'
}
{
  moduleId: '@nestjs/core',
  fileURL: 'file:///home/bumble/software/esmock-scoped-package/node_modules/@nestjs/core/index.js'
}

@iambumblehead
Copy link
Owner

Per the finding in the previous comment, the error occurs regardless of how esmock resolves the modules so my hunch is that a similar symlink/truepath thing affects tsimp which possibly causes the issue.

@darcyrush
Copy link
Author

failed test output here looks like this btw

> test
> TSIMP_PROJECT=./test/tsconfig.json node --import tsimp/import --test-reporter spec --test 'test/example.test.ts'

[Nest] 48166  - 2024/05/03 7:56:54     LOG [NestFactory] Starting Nest application...
[Nest] 48166  - 2024/05/03 7:56:54   ERROR [ExceptionHandler] metatype is not a constructor
TypeError: metatype is not a constructor
    at Injector.instantiateClass (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/injector.js:365:19)
    at callback (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/injector.js:65:45)
    at async Injector.resolveConstructorParams (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/injector.js:144:24)
    at async Injector.loadInstance (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/injector.js:70:13)
    at async Injector.loadProvider (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/injector.js:97:9)
    at async /home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/instance-loader.js:56:13
    at async Promise.all (index 0)
    at async InstanceLoader.createInstancesOfProviders (/home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/instance-loader.js:55:9)
    at async /home/bumble/software/esmock-scoped-package/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/@nestjs/core/injector/instance-loader.js:40:13
    at async Promise.all (index 1)
✖ test/example.test.ts (2044.283154ms)
  'test failed'

Yes, this is output for the test failing due to the mock not taking effect. Apologies, I should have stated that in the issue. The test and test case implementation itself is nonsensical, but I couldn't find a more simple name-spaced package to use as a better example.

Thank you for taking the time to look into this. Before opening an issue with tsimp, I wanted to see if other TS runners were affected so I added both tsx and ts-node. It appears they are all impacted by this, which makes me think this may still be related to esmock.

I updated the minimal reproduction repository to help streamline the investigation.

@kylekirkby
Copy link

Hey all 👋

Did anyone find a fix for this by any chance?

Just hit this myself...

@iambumblehead
Copy link
Owner

It would be great if esmock would handle these pnpm situations. I haven't used pnpm and so there has not been an organic reason for me personally to resoilve this. esmock wraps the growing complexities of esm module resolution, loader behaviour, etc. and so, even for myself, fixing this issue is likely difficult.

@iambumblehead
Copy link
Owner

installing pnpm means downloading a binary and giving it access to your system (not nice)

@insanity54
Copy link

insanity54 commented Oct 17, 2024

edit removed aggression (I apologize, I almost got hit by a car today and I misdirected my anger here)

I understand your concerns about giving pnpm binary access to your system. However, it's worth noting that many of us already give similar access to tools like node and npm because they help us accomplish important tasks. It's natural to be cautious, but I believe it's important to try something before forming strong opinions about it.

I write this in defense of pnpm. In a competition between npm vs. pnpm, pnpm wins in almost every metric. Install times, cache size, monorepo support, etc.

@iambumblehead
Copy link
Owner

I think this resolves the issue

diff --git a/src/esmockLoader.js b/src/esmockLoader.js
index 0b16977..51c3f11 100644
--- a/src/esmockLoader.js
+++ b/src/esmockLoader.js
@@ -25,9 +25,13 @@ const isnotfoundRe = /isfound=false/
 const iscommonjsmoduleRe = /^(commonjs|module)$/
 const isstrict3 = /strict=3/
 const hashbangRe = /^(#![^\n]*\n)/
+
+const moduleIdEsc = str =>
+  str.indexOf('+') >= 0 ? str.replace(/(?!\\)\+/g, '\\+') : str
+
 // returned regexp will match embedded moduleid w/ treeid
 const moduleIdReCreate = (moduleid, treeid) => new RegExp(
-  `.*(${moduleid}(\\?${treeid}(?:(?!#-#).)*)).*`)
+  `.*(${moduleIdEsc(moduleid)}(\\?${treeid}(?:(?!#-#).)*)).*`)
 
 // node v12.0-v18.x, global
 const mockKeys = global.mockKeys = (global.mockKeys || {})

@insanity54 thanks for chiming in, my apology for not being more careful with my message.

@iambumblehead
Copy link
Owner

planning to copy @darcyrush's test case into the esmock pipeline before merging the PR and publishing

@iambumblehead
Copy link
Owner

I won't be around to finish this for at least a few hours...

@iambumblehead
Copy link
Owner

planning to merge and publish soon

@iambumblehead
Copy link
Owner

anyone participating or reading this thread, please report back and share your experience using pnpm with esmock version 2.6.8+

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants