ts-jest version 26.4.2 broke coverage

[ygortgaleno]

🐛 Bug Report

Upgrading ts-jest to 26.4.2 broke the coverage files using default coverage provider(babel).

jest.config.js

module.exports = {
  roots: ['<rootDir>/src'],
  collectCoverageFrom: [
    '<rootDir>/src/**/*.ts',
    '!<rootDir>/src/**/protocols/**',
    '!<rootDir>/src/**/*.spec.ts',
  ],
  coverageDirectory: 'coverage',
  testEnvironment: 'node',
  transform: {
    '.+\\.ts$': 'ts-jest',
  },
  moduleNameMapper: {
    '^@domain/(.*)$': '<rootDir>/src/domain/$1',
  },
};

Downgrading ts-jest to 26.4.1 solves the issue.

To Reproduce

Run jest with coverage option with default coverage provider.

npx jest --coverage

or

yarn jest --coverage

Expected behavior

Show files coverage.

Debug log:

> jest --coverage

 PASS  src/domain/use_cases/Tools/CreateTools.spec.ts
 PASS  src/domain/entities/Tool.spec.ts
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
----------|---------|----------|---------|---------|-------------------
All files |       0 |        0 |       0 |       0 |                   
----------|---------|----------|---------|---------|-------------------

Test Suites: 2 passed, 2 total
Tests:       10 passed, 10 total
Snapshots:   0 total
Time:        1.554 s

System:
    OS: Ubuntu 20
    Node version: 12.18.1(LTS)

Npm packages:
    jest: 26.6.1
    ts-jest: 26.4.2
    typescript: 4.0.3
    babel(optional):

[ahnpnl]

Hi, would you please provide a repo to reproduce the issue ?

[ygortgaleno]

Sure.
Take a look in branch ts-jest-coverage-broken.

https://github.com/ygortgaleno/bossabox-backend/tree/ts-jest-coverage-broken

[guyellis]

@ahnpnl - I have a project where the change from 26.4.1 to 26.4.2 breaks the tests because 26.4.2 is not "finding" ambient types. i.e. I have types declared in modules without export and those are (obviously) ambient types and 26.4.2 is not picking them up.

This PR has the errors: https://github.com/guyellis/plant/pull/4180

I think that this is the same problem that the OP has mentioned. If it's different I'm happy to open a different issue.

[ahnpnl]

@ygortgaleno I have checked out your project but I see coverage works for me on the branch ts-jest-coverage-broken.

@guyellis I think your issue is a different one, but I know what causes the issue.

[guyellis]

@ahnpnl - would you like me to open a new issue or will you take it from here?

[ahnpnl]

yes, I think it's better to have a new issue. Your issue is about ambient types.

[guyellis]

Tracking Ambient Type issue here.

[ygortgaleno]

Here when I run with ts-jest 26.4.2

But I added an action for target branch and coverage was ok, I think has something wrong with my environment, so I will close this issue, thanks for help.

[ghiscoding]

@ahnpnl @ygortgaleno
I have the exact same problem (it shows All Files 0 same as provided print screen), Jest Coverage is broken with latest version of "ts-jest": "^26.4.2". I was about to open a new issue on Jest GitHub but it took me an hour to figure out that it was actually ts-jest that broke it. Going back to version 26.4.1 fixes it (even with latest versions of Jest).

For a repro, you can refer to 1 of my open source lib Slickgrid-Universal
Here's my package.json and here's my jest.config.js

So this issue is valid and should be reopened, it is a regression on my side (I actually have 2 open source libs that encountered the exact same problem).

[ahnpnl]

hi @ghiscoding , I've checked out your repo but I don't know what are the steps to reproduce. Would you please give some instructions ?

[ghiscoding]

@ahnpnl you can run, change package.json to use latest ts-jest and then run the Jest tests with yarn test or npm test which calls npx jest --runInBand --coverage --config ./test/jest.config.js and then you should see that the Coverage is empty (or nearly empty) while I'm expecting .

-----------------------------------------|---------|----------|---------|---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
File                                     | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                                                                                                                                                                                                                                                                                                
-----------------------------------------|---------|----------|---------|---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
All files                                |     100 |    86.68 |     100 |     100 |                                                                                                                                                                                                                                                                                                                  
 common/src                              |     100 |      100 |     100 |     100 |      
... bunch of files
...
Test Suites: 174 passed, 174 total
Tests:       1 skipped, 2891 passed, 2892 total

Does that help?

[ahnpnl]

I actually had to run build first to be able to run tests. I tried with 26.4.3 (prepare to release) and I got this

I guess this proves that 26.4.3 will solve the issue ?

[ghiscoding]

I'm working on a branch (not pushed) which I have updated to the following

  "devDependencies": {
    "@types/jest": "^26.0.15",
    "jest": "^26.6.1",
    "jest-cli": "^26.6.1",
    "jest-environment-jsdom": "^26.6.1",
    "jest-extended": "^0.11.5",
    "jest-junit": "^12.0.0",
    "ts-jest": "^26.4.2"

and I don't see anything showing up in the Coverage, while going back to 26.4.1 does show up.

[ahnpnl]

would you please push that branch ?

[ghiscoding]

Ok sure, here's the PR of the new branch, do you want me to merge it? It's missing the unit tests, but I can add them later if you want it merged. CircleCI is running now, I'll get the result in few minutes, I just went back to 26.4.2 and I see 2 files (instead of a hundred)

I'll report back once CircleCI is over

CircleCI shows all coverage... wth???
I don't understand, it shows all coverage on CircleCI

[ahnpnl]

I cannot reproduce either. My local shows exactly like your CI

[ghiscoding]

I have 2 other small changes in webpack config, let me undo them and see if that makes a difference.... nope still the same I don't see the full result. I don't understand... let me delete my node_modules and re-install, that sometimes fixes these kind of issues

[ghiscoding]

@ahnpnl
I deleted the node_modules, deleted the jest-coverage report folder and re-ran the Jest tests but still it's incomplete. It now shows 1 more file (now I have 2/4 files that I've changed in the PR). Anything else I can try to troubleshoot this problem? I could go back to previous version to fix it temporary but I'd like to be at the latest as much as possible (even more so on a patch version). I have the same problems on 2 different computers (1 computer with this lib and another computer with another lib I have which give me the same kind of nearly empty coverage).

However I see 1 difference in the yarn.lock file, I can see that @jest/create-cache-key-function@^26.5.0 got added in ts-jest: 26.4.2, is that expected?

"@jest/create-cache-key-function@^26.5.0":
  version "26.5.0"
  resolved "https://registry.yarnpkg.com/@jest/create-cache-key-function/-/create-cache-key-function-26.5.0.tgz#1d07947adc51ea17766d9f0ccf5a8d6ea94c47dc"
  integrity sha512-DJ+pEBUIqarrbv1W/C39f9YH0rJ4wsXZ/VC6JafJPlHW2HOucKceeaqTOQj9MEDQZjySxMLkOq5mfXZXNZcmWw==

ts-jest@^26.4.2:
  version "26.4.2"
  resolved "https://registry.yarnpkg.com/ts-jest/-/ts-jest-26.4.2.tgz#00b6c970bee202ceef7c6e6e9805c4837b22dab8"
  integrity sha512-0+MynTTzzbuy5rGjzsCKjxHJk5gY906c/FSaqQ3081+G7dp2Yygfa9hVlbrtNNcztffh1mC6Rs9jb/yHpcjsoQ==
  dependencies:
    "@jest/create-cache-key-function" "^26.5.0"
...

Going back to 26.4.1 always fixes it while going back to ^26.4.2 always breaks the coverage on my local. Even though you can't reproduce, I can consistently reproduce the problem on my side, so I'm not sure what's wrong.

I'm on Windows 10 with NodeJS v12.19.0 if that makes any difference.

[ahnpnl]

However I see 1 difference in the yarn.lock file, I can see that @jest/create-cache-key-function@^26.5.0 got added in ts-jest: 26.4.2, is that expected?

"@jest/create-cache-key-function@^26.5.0":
  version "26.5.0"
  resolved "https://registry.yarnpkg.com/@jest/create-cache-key-function/-/create-cache-key-function-26.5.0.tgz#1d07947adc51ea17766d9f0ccf5a8d6ea94c47dc"
  integrity sha512-DJ+pEBUIqarrbv1W/C39f9YH0rJ4wsXZ/VC6JafJPlHW2HOucKceeaqTOQj9MEDQZjySxMLkOq5mfXZXNZcmWw==

ts-jest@^26.4.2:
  version "26.4.2"
  resolved "https://registry.yarnpkg.com/ts-jest/-/ts-jest-26.4.2.tgz#00b6c970bee202ceef7c6e6e9805c4837b22dab8"
  integrity sha512-0+MynTTzzbuy5rGjzsCKjxHJk5gY906c/FSaqQ3081+G7dp2Yygfa9hVlbrtNNcztffh1mC6Rs9jb/yHpcjsoQ==
  dependencies:
    "@jest/create-cache-key-function" "^26.5.0"
...

yes that is expected.

I just published 26.4.3-beta.0 at tag next. You can also try with this next tag.

[ahnpnl]

I am using Mac. Is your issue about missing coverage of files or the coverage is not correct ? Maybe try to clear jest cache and rerun ?

[ghiscoding]

@ahnpnl
I did the Jest clear cache with npx jest --clearCache and now it works, and then I wanted to see what happened if I run it twice and it provides full test coverage both times. I find that strange that I never had to do that before the new version but now it seems that I need to clear Jest before running a full test coverage (I don't mind adding it to my script, I'm already running runInBand and I don't think clearing Jest would slow it down that much, I just wonder why I have to clear it now when I've never had to do that before).

I think installed 26.4.3-beta.0 as you suggested, ran the full test coverage and it works too. So I'm not sure if it's just because of that Jest clear cache that everything started to work? but I only cleared once and now it works every time I run the coverage... never thought of doing a clear cache (rarely ever have to do that).

Is your issue about missing coverage of files or the coverage is not correct ?

I'm expecting a full test coverage report of all files, I typically run this every time I'm adding new features so that I can see which lines are missing unit tests. However the latest version was only show 1 or 2 files of coverage, not the entire hundred files which was totally unexpected.

So it looks like I'm back in business, even if I find that strange since I had the same issues on 2 different computers with 2 different projects.

Also I must say, thanks a ton for your quick response, it helps a lot.

ohh I just noticed you already pushed a new version 26.4.3 that was fast 😮🚀 ...and it works too, it looks like you could close the issue, again thanks a ton