--reporter-option(s) flags in .opts files are ignored when running with _mocha

[SupernaviX]

EDIT 2019-03-14: I had originally misdiagnosed the issue, and thought it was related to .opt files. The actual issue is with the binary used.
EDIT 2019-03-15: The actual issue is with both .opt files and the mocha binary.

Prerequisites

• Checked that your issue hasn't already been filed by cross-referencing issues with the faq label
• Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
• 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
• Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with: node node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend that you not install Mocha globally.

Description

When I upgrade to the latest version of mocha, my settings --reporter-option and --reporter-options flags are not being respected by the built-in xunit reporter, or by custom reporters. After some debugging, it looks like the --reporter-option flag is ignored when

• I set it in an options file, AND
• I use the _mocha command to run tests.

The --reporter-option flag works when I pass it to mocha or _mocha directly, and options files work with mocha. It's only when I use options files and the _mocha command together that the flag is ignored.

The --reporter-options flag has identical behavior in all cases.

Steps to Reproduce

• Create a project with at least one test
• Add a .mocha.jenkins file with the following content:

--reporter xunit
--reporter-option output=spec/reports/unit-test-report.xml

• Run

npx _mocha --opts .mocha.jenkins

Expected behavior:

Results are saved in "spec/reports/unit-test-report.xml"

Actual behavior:

Results are printed on the console.

Reproduces how often:

Always

Versions

• The output of mocha --version and node node_modules/.bin/mocha --version: 6.0.2
• The output of node --version: v8.12.0
• Your operating system
  • name and version: macOS-10.13.3 (High Sierra)
  • architecture (32 or 64-bit): 64-bit
• Your shell (e.g., bash, zsh, PowerShell, cmd): bash
• Any third-party Mocha-related modules (and their versions):
  • Also reproducible with mocha-jenkins-reporter-0.4.1
• Any code transpiler (e.g., TypeScript, CoffeeScript, Babel) being used (and its version):
  • Typescript-3.3 if relevant

Additional Information

[juergba]

I don't know much about --reporter-option, so please apologize ... just in case.

{ 'reporter-option': [ 'output=spec/reports/unit-test-report.xml' ],
  'reporter-options': [ 'output=spec/reports/unit-test-report.xml' ],
  O: [ 'output=spec/reports/unit-test-report.xml' ],

I guess this part shows the content after Mocha finished its option parsing, right? So the option file was found and the flags are read successfully.

reporterOptions: undefined }
Where does this line come from? Is this the result after the reporter has processed the options?
Could we say: options are parsed correctly, but interpreted incorrectly by the reporter?
Is the reporter expecting an object, but gets an array?

You are using --opts for your .mocha.jenkins file. The docu states:

--opts Path to mocha.opts

Did your way of using --opts work with Moch v5?

[juergba]

I made some short tests with:

--reporter  xunit
--reporter-option   output=d:/temp.dat

It worked correctly both ways, via CLI and also via --opts .mocha.jenkins.
I played around with spaces, tabs ... but could not reproduce the bug so far.

I tested against our current master, maybe the problem is already fixed, will need more testing.

BTW: I tested with node.exe mocha, not via npm / npx. Maybe you could try also via node.exe

[SupernaviX]

Thanks for looking into this!

Thanks to your last suggestion, I figured out what the actual issue is! It turns out that my project was using _mocha instead of mocha to run the tests.

The error and repro steps I reported are wrong. I had copypasted the command with an opts file from my package.json, but wrote the command with flags from scratch, and I didn't notice that the former was using _mocha and the latter was just using mocha. From now on I'll attach a minimal repro whenever I file a bug 😅

I'm not sure if this is even a bug now, since the _mocha binary is undocumented and doesn't look like it's meant for external consumption. Maybe it would be worth calling out as a breaking change in mocha v6, but I don't know why we were even using it in the first place.

[plroebuck]

Was surprised at your reported issue, as running XUnit reporter with same arguments is one of our integration tests.

I'm not sure if this is even a bug now, since the _mocha binary is undocumented and doesn't
look like it's meant for external consumption. Maybe it would be worth calling out as a breaking
change in mocha v6, but I don't know why we were even using it in the first place.

Simplistic explanation why there are two binaries is that "/bin/mocha" relaunches itself as "/bin/_mocha" with Node cmdline options applied -- if you're not using V8 options or debugger, there's nothing gained. Currently, there's a PR in work to not bother relaunching if no Node flags exist, so "/bin/mocha" is the binary that should be used.

That said, your setup should have worked against "/bin/_mocha". Could you correct whatever is incorrect in issue description above so we can look into this further? Exact macOS version?

P.S. Ditch global "mocha" installation. Use "bash" alias if direct launching of local install desired.

$ cd /path/to/project
$ alias mocha=$(pwd)/node_modules/.bin/mocha
$ alias _mocha=$(pwd)/node_modules/.bin/_mocha

[juergba]

That said, your setup should have worked against "/bin/_mocha". Could you correct whatever is incorrect in issue description above so we can look into this further? Exact macOS version?

In my setup it does not work with "/bin/_mocha" and IMO this is ok, so no bug.
The config files are not read in "/bin/_mocha", they are prepared in "/bin/mocha" and have to be passed to "/bin/_mocha".

Just compare the call in /bin/_mocha require('../lib/cli').main();

with PR #3827

} else {
  require('../lib/cli/cli').main(unparse(mochaArgs, {alias: aliases}));
}

[SupernaviX]

I corrected the description in case other people run into this.

I'm inclined to agree that this isn't a bug, but if other people are using _mocha for whatever reason it still might be worth listing this as a breaking change.

[juergba]

Just to make sure: "/bin/_mocha" does work with CLI options, but not with config/option files. So I'm not sure wether your corrected description is correct now. I don't know wether this behavior has changed since v5.2.

I remove the label "unconfirmed-bug", please close this issue if you don't have any further question.

[plroebuck]

Hold on. The question at large is whether this was intentional. IMO "/bin/_mocha" should definitely work, unless @boneskull meant to do so; nothing was mentioned in the release notes for Mocha-6.0.

[juergba]

Some of my conclusions were incorrect. 😎
Both binaries mocha and _mocha should and do read the config files.

Unfortunately I have not yet succeeded in debugging a child process (mocha ->_mocha) with VSCode.
But when running with _mocha directly, the mocha instance shows four reporter-option:

• O: ["output=d:/temp.dat"]
• reporter-option: ["output=d:/temp.dat"]
• reporter-options: ["output=d:/temp.dat"]
• reporterOptions: undefined

The value of the fourth one is certainly wrong and we also don't use camelcase options.

[juergba]

The bug is caused by this line in "cli/cli.js": .parse(argv, loadOptions(argv));

When run with mocha, argv already contains the reporter-option. Its coerce function ("cli/run.js") matches ["output=d:/temp.dat"] to {output: "d:/temp.dat"} which is correctly read by the XUnit reporter.

When run with _mocha, reporter-option is only returned by the cb-function. Its coerce function does not run and XUnit reporter can't get the required output location.

[brettz9]

I don't know if this is the same, I am not able to get reporter-options working in either _mocha and mocha when the config is within mocha on package.json or in .mochrarc.js

[brettz9]

require is not working either as an array or set of strings. But other properties are working.