🐛 BUG: Discrepancy in Node Module Compatibility Between wrangler deploy and vitest-pool-workers Testing

[zach-is-my-name]

Which Cloudflare product(s) does this pertain to?

Workers Vitest Integration

What version(s) of the tool(s) are you using?

Wrangler 3.83, "vitest": "^2.1.5", "@cloudflare/vitest-pool-workers": "^0.5.28",

What version of Node are you using?

v20.13.1

What operating system and version are you using?

Mac Sonoma 15.3

Describe the Bug

Description:

I'm experiencing an issue where my Cloudflare Worker with Durable Objects works correctly when deployed using wrangler deploy, but fails during testing with vitest-pool-workers due to Node module compatibility problems.

Problem Summary:

Deployment Works as Expected:

• When I deploy my Worker using wrangler deploy, it successfully handles Node module dependencies through compatibility transformations.
• This suggests that wrangler correctly replaces or polyfills Node-specific modules to make them compatible with the Workers environment.

Testing Fails Due to Node Dependencies:

• When running tests using vitest and @cloudflare/vitest-pool-workers, the same Worker code fails to execute.
• The testing environment does not seem to perform the same compatibility module replacements as wrangler deploy.
• As a result, imports of Node modules cause errors, and the Durable Objects cannot be tested effectively.

Details:

Node Dependencies in Use:

import { https } from 'node:https';
import { Wallet, keccak256 } from 'ethers';

• My Worker code imports https from 'node:https' and uses Wallet and keccak256 from the ethers library.
• These dependencies are essential for the functionality of my Durable Objects.

Testing Setup:

• I'm using vitest along with @cloudflare/vitest-pool-workers to test my Worker.
• My vitest.config.ts is configured using defineWorkersProject from @cloudflare/vitest-pool-workers/config.

Error Messages During Testing:

When running tests, I encounter errors like:

Error: Cannot find module 'node:https'

This indicates that the Node modules are not being resolved or replaced in the testing environment.

Expected Behavior:

• The testing environment provided by vitest-pool-workers should handle Node module compatibility in the same way as wrangler deploy.
• This would allow tests to run successfully, ensuring that the Durable Objects function correctly in both development and production environments.

Questions and Request for Assistance:

1. Is there a way to configure vitest-pool-workers to perform the same Node module compatibility transformations as wrangler deploy?

2. Are there recommended practices for testing Workers that depend on Node modules using vitest-pool-workers?

3. Could this be a limitation or missing feature in vitest-pool-workers that needs to be addressed?

Steps to Reproduce:

1. Create a Worker with Node Dependencies:

   • Write a Cloudflare Worker script that imports Node modules like node:https and uses ethers.

2. Deploy the Worker:

   • Use wrangler deploy to deploy the Worker.
   • Confirm that it works as expected in the production environment.

3. Set Up Testing Environment:

   • Install vitest and @cloudflare/vitest-pool-workers.

   • Configure vitest.config.ts using defineWorkersProject:

     import { defineWorkersProject } from '@cloudflare/vitest-pool-workers/config';
     import path from 'path';
     
     export default defineWorkersProject({
       test: {
         poolOptions: {
           workers: {
             main: './dist/index.js',
             wrangler: {
               configPath: path.resolve(__dirname, './wrangler.toml'),
             },
           },
         },
       },
     });

4. Run Tests:

   • Execute vitest to run the tests.
   • Observe the errors related to missing Node modules.

Additional Information:

• Environment Details:

  • vitest version: [Please specify]
  • @cloudflare/vitest-pool-workers version: [Please specify]
  • wrangler version: [Please specify]
  • Node.js version: [Please specify]

• Attempts to Resolve:

  • I tried adjusting the vitest configuration, including using defineConfig and specifying pool: '@cloudflare/vitest-pool-workers', but the issue persists.
  • It seems that the compatibility layer provided by wrangler during deployment is not available or not activated during testing.

Conclusion:

I believe there may be a discrepancy between how wrangler deploy and vitest-pool-workers handle Node module compatibility. I would appreciate any guidance on resolving this issue or insights into whether this is a known limitation.

Thank you for your assistance!

Please provide a link to a minimal reproduction

https://github.com/zach-is-my-name/reproduction-vitest-pool-compat-module-error

Please provide any relevant error logs

Click to expand

[~/Projects/charli/apps/session-time-tracker]$ bun run test $ vitest run --config tests/vitest.config.ts

RUN v2.1.5 /Users/zm/Projects/charli/apps/session-time-tracker

Using vars defined in .dev.vars
[vpw:inf] Starting isolated runtimes for tests/vitest.config.ts...
❯ tests/sessionTimer.test.ts (0)
❯ SessionTimer Durable Object (0)
❯ tests/websocketEndpoint.test.ts (3)
❯ POST /websocket (3)
× should process peer:joined websocket event
· should return error for invalid signature
· should return error for unsupported event type
workerd/server/server.c++:3059: error: Fallback service failed to fetch module; payload = ; spec = /?specifier=node%3Ahttps&referrer=%2FUsers%2Fzm%2FProjects%2Fcharli%2Fapps%2Fsession-time-tracker%2Fnode_modules%2Fethers%2Flib.esm%2Futils%2Fgeturl.js&rawSpecifier=node%3Ahttps
workerd/server/server.c++:3059: error: Fallback service failed to fetch module; payload = ; spec = /?specifier=node%3Ahttps&referrer=%2FUsers%2Fzm%2FProjects%2Fcharli%2Fapps%2Fsession-time-tracker%2Fnode_modules%2Fethers%2Flib.esm%2Futils%2Fgeturl.js&rawSpecifier=node%3Ahttps
workerd/server/server.c++:3059: error: Fallback service failed to fetch module; payload = ; spec = /?specifier=node%3Ahttps&referrer=%2FUsers%2Fzm%2FProjects%2Fcharli%2Fapps%2Fsession-time-tracker%2Fnode_modules%2Fethers%2Flib.esm%2Futils%2Fgeturl.js&rawSpecifier=node%3Ahttps
workerd/server/server.c++:3059: error: Fallback service failed to fetch module; payload = ; spec = /?specifier=node%3Ahttps&referrer=%2FUsers%2Fzm%2FProjects%2Fcharli%2Fapps%2Fsession-time-tracker%2Fnode_modules%2Fethers%2Flib.esm%2Futils%2Fgeturl.js&rawSpecifier=node%3Ahttps
workerd/server/server.c++:3059: error: Fallback service failed to fetch module; payload = ; spec = /?specifier=node%3Ahttps&referrer=%2FUsers%2Fzm%2FProjects%2Fcharli%2Fapps%2Fsession-time-tracker%2Fnode_modules%2Fethers%2Flib.esm%2Futils%2Fgeturl.js&rawSpecifier=node%3Ahttps
workerd/server/server.c++:3059: error: Fallback service failed to fetch module; payload = ; spec = /?specifier=node%3Ahttps&referrer=%2FUsers%2Fzm%2FProjects%2Fcharli%2Fapps%2Fsession-time-tracker%2Fnode_modules%2Fethers%2Flib.esm%2
❯ tests/connectionManager.test.ts (0)
❯ tests/initEndpoint.test.ts (0)
❯ tests/sessionTimer.test.ts (0)
❯ SessionTimer Durable Object (0)
❯ tests/webSocketManager.test.ts (0)
❯ tests/webhookEndpoint.test.ts (0)
❯ tests/websocketEndpoint.test.ts (3)
❯ POST /websocket (3)
× should process peer:joined websocket event
× should return error for invalid signature
× should return error for unsupported event type

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Suites 6 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

FAIL tests/connectionManager.test.ts [ tests/connectionManager.test.ts ]
FAIL tests/initEndpoint.test.ts [ tests/initEndpoint.test.ts ]
FAIL tests/webSocketManager.test.ts [ tests/webSocketManager.test.ts ]
FAIL tests/webhookEndpoint.test.ts [ tests/webhookEndpoint.test.ts ]
Error: No such module "node:https".
imported from "Users/zm/Projects/charli/apps/session-time-tracker/node_modules/ethers/lib.esm/utils/geturl.js"
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/9]⎯

FAIL tests/sessionTimer.test.ts [ tests/sessionTimer.test.ts ]
Error: No test found in suite tests/sessionTimer.test.ts
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/9]⎯

FAIL tests/sessionTimer.test.ts > SessionTimer Durable Object
Error: No test found in suite SessionTimer Durable Object
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/9]⎯

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Tests 3 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

FAIL tests/websocketEndpoint.test.ts > POST /websocket > should process peer:joined websocket event
AssertionError: expected 404 to be 200 // Object.is equality

• Expected

• Received

• 200

• 404

❯ tests/websocketEndpoint.test.ts:51:29
49| });
50|
51| expect(response.status).toBe(200);
| ^
52| const text = await response.text();
53| expect(text).toBe('Webhook processed successfully');

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[4/9]⎯

FAIL tests/websocketEndpoint.test.ts > POST /websocket > should return error for invalid signature
AssertionError: expected 404 to be 401 // Object.is equality

• Expected

• Received

• 401

• 404

❯ tests/websocketEndpoint.test.ts:78:29
76| });
77|
78| expect(response.status).toBe(401);
| ^
79| const data = await response.json() as { status: string; message: string };
80|

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[5/9]⎯

FAIL tests/websocketEndpoint.test.ts > POST /websocket > should return error for unsupported event type
AssertionError: expected 404 to be 400 // Object.is equality

• Expected

• Received

• 400

• 404

❯ tests/websocketEndpoint.test.ts:98:29
96| });
97|
98| expect(response.status).toBe(400);
| ^
99| const data = await response.json() as { status: string; message: string };
100|

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[6/9]⎯

Test Files 6 failed (6)
Tests 3 failed (3)
Start at 18:38:11
Duration 918ms (transform 107ms, setup 131ms, collect 51ms, tests 52ms, environment 0ms, prepare 935ms)

[vpw:dbg] Shutting down runtimes...

[emily-shen]

Hiya, yeah this is a known issue unfortunately :( The vitest integration provides its own polyfills to get vitest to work, and they're not exactly the same as the ones that wrangler provides with the nodejs_compat flags (either version). That's not ideal, and we're looking into a solution for this but its a bit of a tricky one.

For reference, these are the polyfills that vitest-pool-workers is adding
https://github.com/cloudflare/workers-sdk/tree/main/packages/vitest-pool-workers/src/worker/lib

[zach-is-my-name]

Thanks! My workaround is to create a sister testing repo of my production Workers repo without those polyfilled deps.

If there's a better workaround, please advise.

Otherwise I'm glad to know it's not due to improper configuration.

Feel free to close and track as desired.

[emily-shen]

I don't think there's a better work around currently :/ we've been looking into adding these polyfills to vitest, probs via an unenv plugin.

For reference in case anyone wanders across this - if an api is red in the workerd column, but green in wrangler, it won't be provided to vitest currently and its not you :(
https://workers-nodejs-compat-matrix.pages.dev/

[emily-shen]

Ah whoops sorry I was totally mistaken - there is a way to get around this @zach-is-my-name

if you use wrangler to build + add polyfills, you can then point vitest at the output which will be the same as what wrangler deploys.
So run npx wrangler deploy --dry-run --outdir dist
Then set [main](https://developers.cloudflare.com/workers/testing/vitest-integration/configuration/#workerspooloptions) in your vitest config to your build output.

[zach-is-my-name]

Thanks @emily-shen.
Differently than the reproduction? https://github.com/zach-is-my-name/reproduction-vitest-pool-compat-module-error

[~/tmp/reproduction-vitest-pool-compat-module-error]$ ls dist
README.md    index.js     index.js.map
[~/tmp/reproduction-vitest-pool-compat-module-error]$ head wrangler.toml
#:schema node_modules/wrangler/config-schema.json
name = "hello-world"
main = "src/index.ts"
compatibility_date = "2024-11-12"
compatibility_flags = ["nodejs_compat_v2"]

[~/tmp/reproduction-vitest-pool-compat-module-error]$ cat test/vitest.config.ts
import {  defineWorkersProject } from "@cloudflare/vitest-pool-workers/config";
import path from 'path';

export default defineWorkersProject({
        test: {
                poolOptions: {
                        workers: {
                                main: "../dist/index.js",
        wrangler: { configPath: path.resolve(__dirname, '../wrangler.toml') },
                        },
                },
        },
});

[~/tmp/reproduction-vitest-pool-compat-module-error]$ bun run test
$ vitest run --config test/vitest.config.ts

 RUN  v2.0.5 /Users/zm/tmp/reproduction-vitest-pool-compat-module-error

[vpw:inf] Starting isolated runtimes for test/vitest.config.ts...
[mf:wrn] The latest compatibility date supported by the installed Cloudflare Workers Runtime is "2024-11-06",
but you've requested "2024-11-12". Falling back to "2024-11-06"...
workerd/server/server.c++:3112: error: Fallback service failed to fetch module; exception = workerd/server/workerd-api.c++:416: failed: expected featureFlags.getNodeJsCompat(); The nodejs_compat compatibility flag is required to use the nodeJsCompatModule type.
stack: 102a1050f 1029dd94f; spec = /?specifier=%2FUsers%2Fzm%2Ftmp%2Freproduction-vitest-pool-compat-module-error%2Fnode_modules%2F%40cloudflare%2Fvitest-pool-workers%2Fdist%2Fworker%2Flib%2Fcloudflare%2Fmock-agent.cjs%3Fmf_vitest_no_cjs_esm_shim&referrer=%2FUsers%2Fzm%2Ftmp%2Freproduction-vitest-pool-compat-module-error%2Fnode_modules%2F%40cloudflare%2Fvitest-pool-workers%2Fdist%2Fworker%2Flib%2Fcloudflare%2Fmock-agent.cjs&rawSpecifier=.%2Fmock-agent.cjs%3Fmf_vitest_no_cjs_esm_shim
service core:user:vitest-pool-workers-runner-: Uncaught Error: No such module "Users/zm/tmp/reproduction-vitest-pool-compat-module-error/node_modules/@cloudflare/vitest-pool-workers/dist/worker/lib/cloudflare/mock-agent.cjs?mf_vitest_no_cjs_esm_shim".
  imported from "Users/zm/tmp/reproduction-vitest-pool-compat-module-error/node_modules/@cloudflare/vitest-pool-workers/dist/worker/lib/cloudflare/mock-agent.cjs"

[emily-shen]

Argh sorry my bad 🤦 . In your integration test you're still importing from src/index so its not using the polyfills wrangler adds at build time. If I change that in your repo it seems to work for me. (although https.get is only mocked by the polyfills, so I assume that's just for your repro).
Hope that works for you!

[zach-is-my-name]

Thanks @emily-shen! I'll check it out. Please allow a few more days I'm working on resolving other Failed to pop isolated storage stack frame in tests/.... errors in the tests with the dependencies removed before I can look at this again. Edit: Still busy with other urgent things. Hopefully I can confirm all this within the next 24 hours...

[zach-is-my-name]

I'm going to close this as I don't have the bandwidth to follow-up.

[tizmagik]

I just hit this issue as well today. @emily-shen is the current recommendation still to do a full build before running tests? That would significantly slow down our testing feedback loop while running them locally. Are there any other options? Please advise, thank you!

[zach-is-my-name]

Does this fix it @tizmagik ?

https://developers.cloudflare.com/workers/testing/vitest-integration/known-issues/#module-resolution
from
#7810