lcov reporter crashes when a js file is changed and rerun during coverage with es2015 and stage-1 presets

[jedwards1211]

See https://github.com/istanbuljs/nyc/issues/425 for more detail on what actually happens.

For some reason I've only been able to reproduce with babel-preset-es2015 and babel-preset-stage-1 active in my .babelrc.

Minimum repro repository: https://github.com/jedwards1211/istanbul-lcov-issue

git clone [email protected]:jedwards1211/istanbul-lcov-issue
cd istanbul-lcov-issue
npm install
npm test

This is an intermittent issue, possibly just due to ipc timing in this test, which I'm seeing about 80% of the time, so rerun npm test a few times if you don't see it.

Here's what the bug occuring looks like:

$ npm test

> istanbul-lcov-issue@ test /Users/andy/istanbul-lcov-issue
> NODE_ENV=coverage nyc --reporter=lcov --reporter=text mocha ./test/index.js



  lcov reporter
{ hello: 'world' }
{ hello: 'world' }
{ hello: 'world' }
    ✓ works for file that changes during test (4303ms)


  1 passing (4s)

/Users/andy/istanbul-lcov-issue/node_modules/nyc/node_modules/istanbul-reports/lib/lcovonly/index.js:32
        writer.println('FN:' + [meta.decl.start.line, meta.name].join(','));
                                    ^

TypeError: Cannot read property 'decl' of undefined
    at /Users/andy/istanbul-lcov-issue/node_modules/nyc/node_modules/istanbul-reports/lib/lcovonly/index.js:32:37
    at Array.forEach (native)
    at LcovOnlyReport.onDetail (/Users/andy/istanbul-lcov-issue/node_modules/nyc/node_modules/istanbul-reports/lib/lcovonly/index.js:30:28)
    at LcovReport.(anonymous function) [as onDetail] (/Users/andy/istanbul-lcov-issue/node_modules/nyc/node_modules/istanbul-reports/lib/lcov/index.js:21:24)
    at Visitor.(anonymous function) [as onDetail] (/Users/andy/istanbul-lcov-issue/node_modules/nyc/node_modules/istanbul-lib-report/lib/tree.js:34:30)
    at ReportNode.Node.visit (/Users/andy/istanbul-lcov-issue/node_modules/nyc/node_modules/istanbul-lib-report/lib/tree.js:123:17)
    at /Users/andy/istanbul-lcov-issue/node_modules/nyc/node_modules/istanbul-lib-report/lib/tree.js:116:23
    at Array.forEach (native)
    at visitChildren (/Users/andy/istanbul-lcov-issue/node_modules/nyc/node_modules/istanbul-lib-report/lib/tree.js:115:32)
    at ReportNode.Node.visit (/Users/andy/istanbul-lcov-issue/node_modules/nyc/node_modules/istanbul-lib-report/lib/tree.js:126:5)
npm ERR! Test failed.  See above for more details.

[mlrawlings]

I'm seeing the same issue with the lcov report. You can see here (scroll to bottom).

The html report also has a few files that have a similar Error as their content rather than the actual file contents:

@bcoe Is this an issue with the reporter(s) or are invalid .json coverage files being generated (due to the file contents changing) that are then being read by the reporter?

I know in our case, we have the same runtime files loaded on the server and then also in phantomjs after going through a module bundler. We're not using sourcemaps, so I didn't expect things to work, but gave it a try anyways. Initially it looked like things worked and the line hits in all the files that actually produce reports look correct. What's confusing is why most files are fine, but some aren't.

[jedwards1211]

@mlrawlings are you using babel-plugin-istanbul? If so I wouldn't expect there to be a problem, because the line numbers, function indexes, etc. generated by istanbul are based upon the source, not the transpiled code. Or maybe your bundler is affecting the code before transpilation?

[mlrawlings]

@jedwards1211 I'm not using babel and the code is definitely getting modified before going to istanbul, which is why I didn't expect it to work, but it does... for most of the files.

This is what the wrapper looks like:

$_mod.define(/* FILEPATH */, function(require, module, exports, __filename, __dirname) { /* CODE */
});

I just tested doing coverage for only the files that go through the

[jedwards1211]

@mlrawlings Okay, well just FYI, if it's possible to use babel, it would probably work for you. I have a project with some modules that get run on the server with babel-node and on the client via webpack, and I've been able to merge the coverage from the two successfully.

[jedwards1211]

@mlrawlings basically if you can instrument the code before bundling it, it should work.

[mlrawlings]

@jedwards1211 I think I have my issues figured out (it was working in some cases because the line numbers were unchanged and only the first line had a character shift), I have another problem where the bundler is picking up a file based on the browser field in package.json but using the original filename, but that's not nyc/istanbul's fault.

In your case, I'm curious what you expect to happen. How are you supposed to get accurate coverage of a file that is changing?

[jedwards1211]

@mlrawlings it's definitely questionable whether it's a valid case to support, though perhaps the reporter could at least do something more reasonable than crashing. On the other hand, it shouldn't be too difficult to merge coverage after adding a new function to a file. But I probably could have tested hot reloading without adding an additional function or otherwise changing the code structure in the file.