slimer.exit() should be able to set exit status

[jcoglan]

Currently (version 0.7) calling slimer.exit(1) makes the process exit with status 0. This makes Slimer unsuitable for use as a testing platform since CI servers cannot tell whether the tests failed without relying on a special output format like TAP or XML.

It should be possible to write test scripts with Slimer and use the exit status to indicate failure.

[laurentj]

This is a known issue. As indicated in API_COMPAT, there is no way in the mozilla platform to set the exit status. So the exit() parameter is ignored for the moment.

[laurentj]

A workaround for your script is to display (console.log) an error message and a rule in the CI server could check if the output contains this message or not.

[jcoglan]

I don't know a lot about how this stuff works, but the SpiderMonkey shell supports quit(1) which exits with 1. Can you explain a bit more so I understand why you can't do a similar thing?

For now, I can make my test tools emit TAP and pipe into something that knows what that means, so this isn't a show-stopper.

[laurentj]

I don't use the spidermonkey shell. SpiderMonkey is only a javascript engine. It does not contain a rendering engine, it does not provide a DOM API, an API to manipulate files, an API to do networking etc. SpiderMonkey is only a tiny part of a browser, a tiny part of Gecko. SlimerJS is built on top the Mozilla platform (Gecko/XulRunner), like Firefox.

In future version, probably SlimerJS will embed directly the Mozilla platform, so we could patch it to provide missing features.

[laurentj]

Possible solution for a patch on gecko for a future version of SlimerJS:

1. add a property to set an exit code, into the interface nsIAppStartup
2. implement this property into nsAppStartup
3. returns this property from XREMain::XRE_mainRun

It should be enough.

[laurentj]

A WIP patch does exist https://bugzilla.mozilla.org/show_bug.cgi?id=894697

[abozhilov]

I think it could be achieve it without changes in the Gecko. Since slimerjs is launched via shell script, you can use nsIEnvironment https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIEnvironment to set var for the current process in the phantom/slimer.exit() and then exit from the shell with the value of that var.

The above solution will not work, because the xulrunner is run in its own shell.

For now the solution which would work is to write status file in the profile dir and get the exit status from that file after xulrunner has finished.

[laurentj]

@abozhilov yes I thought about a solution like that, but I found it is was an ugly hack.

However, since we can't wait after Mozilla and many projects depends on it, I'm ok to implement this solution.

[abozhilov]

#278

[laurentj]

So now we have a workaround for the exit code, thanks to @abozhilov .

[nathanboktae]

Can we close this issue now that #278 has been merged?