Failing async before() (bdd) or suiteSetup() (tdd) reports against the previous suite

[dhendo]

If you have an asynchronous section in a before() or suiteSetup() that fails, the error is reported against the previous test suite:

TDD ui case

var should = require('should');

testSetup = function(done) {
    setTimeout(function() {
        // Either a failed assertion
        should.exist(somethingthatdoesntexist);
        // Or a thrown error
        //throw new Error("Something went wrong here... and ");
    }, 10);
};

suite('The First test suite', function() {
    test('has one test', function(done) {
        done();
    });

    test('has a second test', function(done) {
        done();
    });

});

// Now fail inside an async section in the test setup
suite('The Second suite', function() {
    suiteSetup(testSetup);
    test('has one test', function(done) {
        done();
    });
});

Output

  The First test suite
    ? has one test  
    ? has a second test  

  The Second suite
    0) has a second test

  ? 1 of 3 tests failed:

  0) The First test suite has a second test:
     ReferenceError: somethingthatdoesntexist is not defined

BDD ui test case

var should = require('should');

testSetup = function(done) {
    setTimeout(function() {
        // Either a failed assertion
        should.exist(somethingthatdoesntexist);
        // Or a thrown error
        //throw new Error("Something went wronghere... and ");
    }, 10);
};

describe('The First test suite', function() {
    describe('has one test', function() {
        it('is great', function(done) {
            done();
        });
    });

    describe('has a second test', function() {
        it('is great', function(done) {
            done();
        });
    });

});


// Now fail inside an async section in the test setup
describe('The Second suite', function() {
    before(testSetup);
    describe('has one test', function() {
        it('is great', function(done) {
            done();
        });
    });
});

Expected results

" 0) The First test suite has a second test:
     ReferenceError: somethingthatdoesntexist is not defined"

This is incorrect - and should report that the failure was in the beforeAll hook:

  0) The Second suite "before all" hook:
     ReferenceError: somethingthatdoesntexist is not defined

As it does if the error is thrown outside the setTimeout function.

[tj]

appears fine to me:

describe('first', function(){
  it('should something', function(done){
    done()
  })
})

describe('second', function(){
  before(function(done){
    process.nextTick(function(){
      throw new Error('failed')
    });
  })

  it('should do something', function(done){
    done()
  })
})

outputs:

..

  ✖ 1 of 2 tests failed:

  0) second "before all" hook:
     Error: failed
      at Array.0 (/Users/tj/Projects/mocha/test.js:11:13)
      at EventEmitter._tickCallback (node.js:192:40)

[tj]

did i miss something?

[dhendo]

Your code sample fails for me:

  ..

  ✖ 1 of 2 tests failed:

  0) first should something:
     Error: failed
      at Array.0 (/home/davidh/dev/TriggeredMessagingV1/node/mochatest/test/test.js:10:13)
      at EventEmitter._tickCallback (node.js:192:40)

Using mocha 0.3.2, node 0.6.5 on Ubuntu

[tj]

which version of mocha?

[dhendo]

0.3.2

[tj]

weiiird... very weird, I get the expected results for node 0.4.x and 0.6.x

[dhendo]

Very strange. I've managed to try it on a few more machines, and it seems to work fine.