Stryker is not working with Jest 28

[pmcelhaney]

Summary

Stryker is not working with Jest 28. It's trying to apply a mixin to a JestEnvironment class which is undefined. It works fine with Jest 27.

yarn run v1.22.18
$ stryker run
�[32m15:22:11 (44356) INFO ConfigReader�[39m Using stryker.conf.json
�[32m15:22:11 (44356) INFO BroadcastReporter�[39m Detected that current console does not support the "progress" reporter, downgrading to "progress-append-only" reporter
�[33m15:22:11 (44356) WARN InputFileResolver�[39m No files marked to be mutated, Stryker will perform a dry-run without actually mutating anything. You can configure the `mutate` property in your config file (or use `--mutate` via command line).
�[32m15:22:11 (44356) INFO Instrumenter�[39m Instrumented 0 source file(s) with 0 mutant(s)
�[32m15:22:11 (44356) INFO ConcurrencyTokenProvider�[39m Creating 9 test runner process(es).
�[32m15:22:11 (44356) INFO DryRunExecutor�[39m Starting initial test run (jest test runner with "perTest" coverage analysis). This may take a while.
�[91m15:22:12 (44356) ERROR DryRunExecutor�[39m One or more tests resulted in an error:
	undefinedClass extends value #<Object> is not a constructor or null TypeError: Class extends value #<Object> is not a constructor or null
    at mixinJestEnvironment (/code/reproductions/stryker-jest-28/node_modules/@stryker-mutator/jest-runner/dist/src/jest-plugins/mixin-jest-environment.js:25:46)
    at new jestEnvironmentGeneric (/code/reproductions/stryker-jest-28/node_modules/@stryker-mutator/jest-runner/dist/src/jest-plugins/jest-environment-generic.js:7:91)
    at runTestInternal (/code/reproductions/stryker-jest-28/node_modules/jest-runner/build/runTest.js:277:23)
    at async runTest (/code/reproductions/stryker-jest-28/node_modules/jest-runner/build/runTest.js:499:34)
    at async TestRunner.runTests (/code/reproductions/stryker-jest-28/node_modules/jest-runner/build/index.js:147:12)
    at async TestScheduler.scheduleTests (/code/reproductions/stryker-jest-28/node_modules/@jest/core/build/TestScheduler.js:317:13)
    at async runJest (/code/reproductions/stryker-jest-28/node_modules/@jest/core/build/runJest.js:407:19)
    at async _run10000 (/code/reproductions/stryker-jest-28/node_modules/@jest/core/build/cli/index.js:338:7)
    at async Object.runCLI (/code/reproductions/stryker-jest-28/node_modules/@jest/core/build/cli/index.js:190:3)
    at async JestGreaterThan25TestAdapter.run (/code/reproductions/stryker-jest-28/node_modules/@stryker-mutator/jest-runner/dist/src/jest-test-adapters/jest-greater-than-25-adapter.js:9:24)
�[91m15:22:12 (44356) ERROR Stryker�[39m Unexpected error occurred while running Stryker Error: Something went wrong in the initial test run
    at DryRunExecutor.validateResultCompleted (/code/reproductions/stryker-jest-28/node_modules/@stryker-mutator/core/dist/src/process/3-dry-run-executor.js:61:15)
    at DryRunExecutor.timeDryRun (/code/reproductions/stryker-jest-28/node_modules/@stryker-mutator/core/dist/src/process/3-dry-run-executor.js:78:14)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async /code/reproductions/stryker-jest-28/node_modules/@stryker-mutator/core/dist/src/concurrent/pool.js:62:28
�[32m15:22:12 (44356) INFO Stryker�[39m This might be a known problem with a solution documented in our troubleshooting guide.
�[32m15:22:12 (44356) INFO Stryker�[39m You can find it at https://stryker-mutator.io/docs/stryker-js/troubleshooting/
�[32m15:22:12 (44356) INFO Stryker�[39m Still having trouble figuring out what went wrong? Try `npx stryker run --fileLogLevel trace --logLevel debug` to get some more info.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Please see this repo with the minimum configuration needed to reproduce the problem.

https://github.com/pmcelhaney/StrykerAndJest28

https://replit.com/@pmcelhaney/StrykerAndJest28#README.md

[covertbert]

+1 Don't suppose there's any estimate of when this issue will be looked into?

[HonzaMac]

@covertbert Feel free to fix it, it is open-source project.

[pmcelhaney]

I think I've found the problem.

stryker-js/packages/jest-runner/src/messaging.ts

Line 17 in 55fbe8a

this.jestEnvironment = 'jest-circus/runner';

In the current release (6.0.1), that line references "jest-runner/circus", which doesn't exist. Either way I think this is the wrong class.

It's supposed to reference a test environment, like "test-environment-node", not a test runner. If I change that line to "test-environment-node" in the current release it almost works.

The other part is the requireResolve() needs to to have .default appended to the return statement.

I figured this out by adding a jest.config.js in the ./stryker-tmp/sandbox#######/ directory so that I can run Jest the same way that Stryker does. After making these changes, I was able to get Jest to run.

[nicojs]

Thanks, everyone!

I was pretty stunned by this, seeing as jest 28 has been out for 10 days now. We use dependabot for a reason, why didn't we get a PR from them? Apparently, our dependabot is wrongly configured since February, see #3500 🤦‍♂️

Anyway, I'll try to look into it asap. There seems to be some changed stuff for TestEnvironment: https://jestjs.io/docs/upgrading-to-jest28#test-environment

@pmcelhaney
You are correct in that the field is initialized wrongly. However, the field is always overridden here:

stryker-js/packages/jest-runner/src/jest-plugins/with-coverage-analysis.ts

Line 86 in 6c8550e

state.jestEnvironment = nameEnvironment(originalJestEnvironment);

so that doesn't seem to be the problem.

[pmcelhaney]

Thanks! I tested the current release, which appears to be a bit behind main, so that may why I was (apparently) not hitting that override function.

If that's not the issue, I wonder if the second part of what I found is implicated: requireResolve() seems to get tripped up on some combination of ESM, default export, and transpilation.

[covertbert]

@nicojs Dependabot deciding to just not update certain dependencies was one of many reason we moved to Renovate at my company. Would highly recommend checking it out! It gives you more control, and probably more importantly, more visibility of what's happening with your dependencies.

[pmcelhaney]

@nicojs I've updated my test repo to show exactly what I changed to get Jest working in the sandbox directory.

The code never hits the overrideEnvironment() function so I'm probably missing something in the jest.config.js in the sandbox directory. Anyway, I hope this helps!

[nicojs]

Fixed and released in v6.0.2

Thanks for the help everyone🎉

@pmcelhaney overrideEnvironment is definitely hit. How are you testing it? Are you debugging the code? In that case, don't forget that StrykerJS is running test runners always in a child process. Debugging is possible, but a bit more complex.

[pmcelhaney]

Awesome, thanks so much!

You can see my troubleshooting methodology in that repo I shared. I'd love to know what I missed so that I can improve my mental model of how it all fits together.