an error in a beforeEach hook causes the test run to terminate prematurely

[jfirebaugh]

Testcase: https://gist.github.com/3711350

As you can see from the output, only one example was executed. I would have expected the first example to fail, and then the second two to execute and pass.

[tj]

yeah mocha currently assumes that if you have an issue in before hooks then subsequent things will fail, though we could limit the scope of that assumption to only those test-cases nested within the same describe()

[billmag]

Is this slated for a fix at any point?

[tj]

yup, just low priority ATM

[dansitu]

Is anybody currently working on a fix for this? I'll write one otherwise, but I don't want to duplicate effort.

Do you think the new limited scope should be enabled by default, or by a flag?

[tj]

default should be fine since the problem is likely self-contained within that block of nested describe()s, if it's a database connection error or similar and other suites use that db they'll just fail as well no big deal

[dansitu]

Sounds good; I'll give it a crack when I get to work on Monday :)

On Nov 17, 2012, at 11:51 AM, TJ Holowaychuk [email protected] wrote:

default should be fine since the problem is likely self-contained within that block of nested describe()s, if it's a database connection error or similar and other suites use that db they'll just fail as well no big deal

—
Reply to this email directly or view it on GitHub.

[tj]

great! thanks

[bitwiseman]

Any update on this? I though #270 was talking about this problem - it describes the same scenario but is less specific about expected result.

I've got a good ways toward handling all four types of hooks and have tests to cover the work I've done so far. https://github.com/bitwiseman/mocha/tree/hooks-recovery .

This does not fully address timeouts in async hooks or uncaught, but is still a huge step forward. This does a bit more the previously proposed fix - things have changed in six months.

I disagree with the assumption that if a hook fails once it will fail every time as a justification for bailing on the test run. Even if a number of tests will fail, it is better to keep going and get results.

[tj]

optimizing for failure is itself not a great practice, bailing from the failed suite is definitely ideal IMO, maybe not from the entire process but definitely the suite

[jlipps]

I'm also getting bit by this behavior. I would assume that a beforeAll type hook, if it failed, would cause the entire suite to fail. But in a beforeEach, I'd expect it to just fail the particular test it precedes.

The case I'm working with is iOS Instruments testing, where the Instruments process itself (which I contact to start a new session using beforeEach) hangs every so often. In this case, if I were to keep running beforeEach, subsequent tests would likely not fail for the same reason.

I want them to keep running so that I can at least have all tests attempt to run. With things how they are now, I can never make it through an entire run of > 100 tests because Instruments is so flakey it will crash at least once in the first 10-20 tests.

In this case it's clearly not a Mocha issue, but I think allowing for a bit of flexibility on how beforeEach errors are handled would be helpful. If you're running unit tests, you're right, you probably never want to keep going if a beforeEach fails. But when one beforeEach fails, on say the 10th time it's run instead of the first, you know it's worked before and might work again.

[aespinosa]

We also encounter these when we're using mocha for integration tests for deployed webapps. I agree with @jlipps that beforeEach() should be attempted again if your dealing with let's say a flaky api endpoint.

@jlipps do you mind sharing, how are you currently getting around this mocha limitation in your apps?

[wallyqs]

+1 for the fix/workaround for this. 👿

[jlipps]

fix/workaround described in comments for #1014

[travisjeffery]

fixed by #1043

[Apollon0718]

Uncaught TypeError: vue_router__WEBPACK_IMPORTED_MODULE_2__.default.beforeEach is not a function
what does this mean?
code is as follow
Router.beforeEach((to, from, next) => {
// redirect to login page if not logged in and trying to access a restricted page
const publicPages = ['/pages/login','/pages/register'];
const authRequired = !publicPages.includes(to.path);
const loggedIn = localStorage.getItem('auth_user');

if (authRequired && !loggedIn) {
return next('/login');
}

next();
})

[shridharkalagi]

i'm still seeing the same issue? any update on this?