check-coverage doesn’t work correctly

[lo1tuma]

I’m using istanbul 1.0.0-alpha2 with a setup similar to sample-babel-node.
My npm test script looks like this:

NODE_ENV=test babel-node node_modules/istanbul/lib/cli.js cover _mocha test/unit -- --opts test/unit/mocha.opts && babel-node node_modules/istanbul/lib/cli.js check-coverage

If I run this script I get different results for the cover and check-coverage command:

cover command:

=============================== Coverage summary ===============================
Statements   : 100% ( 3802/3802 )
Branches     : 99.89% ( 945/946 )
Functions    : 100% ( 691/691 )
Lines        : 100% ( 3769/3769 )
================================================================================

check-coverage command:

Error: ERROR: Coverage for statements (90.41%) does not meet global threshold (100%)
ERROR: Coverage for branches (63.77%) does not meet global threshold (100%)
ERROR: Coverage for functions (99.27%) does not meet global threshold (100%)

As you can see the coverage for e.g. branches is one time 99.89% and the other time it is only 63.77%.
I think the problem is that the check-coverage command reads the coverage result from coverage.raw.json which is the coverageMap before the source map was applied.
I’ve changed this code on my local istanbul installation to write the content of transformed.map instead of finalCoverage to the coverage.raw.json file. This solved the problem for me.

[gotwarlost]

Could you try the new version I just pushed. check-coverage was erroneously computing coverage from the raw (pre-source map) files as opposed to the post processed files.

Reopen if the problem still exists.

[lo1tuma]

Hi @gotwarlost, I’ve tried istanbul-api version 1.0.0-alpha.10, but the problem still exists:

$ npm ls | grep istanbul
├─┬ [email protected]
├─┬ [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ ├─┬ [email protected]
│ ├─┬ [email protected]
│ ├── [email protected]
│ ├── [email protected]

Coverage summary:

=============================== Coverage summary ===============================
Statements   : 100% ( 207/207 )
Branches     : 100% ( 50/50 )
Functions    : 100% ( 81/81 )
Lines        : 100% ( 207/207 )
================================================================================

check-coverage output:

> istanbul check-coverage --statement 100 --branch 100 --function 100 --lines 100


/Users/foo/workspace/myproject/node_modules/istanbul/lib/cli.js:38
        throw ex; // turn it into an uncaught exception
        ^
ERROR: Coverage for statements (98.04%) does not meet global threshold (100%)
ERROR: Coverage for branches (85.85%) does not meet global threshold (100%)

[gotwarlost]

Dammit, will take a look. Thanks for testing. If this project is public on github, could you give me a link. Otherwise, I'll deal with reproducing it.

[lo1tuma]

Unfortunately the project is not opensource, but I’ve create a reproducible example based on sample-mocha-compilers.

[lo1tuma]

FWIW this change solved the issue for me, but I'm not sure if this is the correct way to solve the problem.

[gotwarlost]

Thanks for the repro case. I will look into in shortly.

[panzerdp]

+1 Having the same problem.

[kommander]

+1 Any updates yet? Love istanbul, want to go on using it.

[brunops]

+1 Same problem
related issue gotwarlost/istanbul#631

[brunops]

The difference between cover and check-coverage seems to be something around using source maps or not.

cover
https://github.com/istanbuljs/istanbul-api/blob/master/lib/run-cover.js#L167

check-coverage
https://github.com/istanbuljs/istanbul-api/blob/master/lib/run-check-coverage.js#L83

Not sure how to fix this though, any help?

[cjewiss]

+1 Still a problem on 1.1.0-alpha.1 .Any updates yet? Thanks

[brunops]

@cjewiss gotwarlost/istanbul#631 (comment)

[cjewiss]

Awesome. All works nicely now using nyc. Thanks