Test case passes even though subcase failed

[marack]

Description

It seems that the overall result of a test case is dependent only on the result of the final subcase to run. This means that a test case can pass even though one of the subcases has failed.

Steps to reproduce

The following code results in a failing unit test suite as expected (okay):

#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
#include "doctest.h"
TEST_CASE("test")
{
  SUBCASE("one") { CHECK(true); }
  SUBCASE("two") { CHECK(false); }
}

...
[doctest] test cases:      1 |      0 passed |      1 failed |      0 skipped
[doctest] assertions:      2 |      1 passed |      1 failed |
[doctest] Status: FAILURE!

However if we swap which subcase is failing we get a passing unit test suite (not okay):

#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
#include "doctest.h"
TEST_CASE("test")
{
  SUBCASE("one") { CHECK(false); }
  SUBCASE("two") { CHECK(true); }
}

...
[doctest] test cases:      1 |      1 passed |      0 failed |      0 skipped
[doctest] assertions:      2 |      1 passed |      1 failed |
[doctest] Status: SUCCESS!

Extra information

• doctest version: v2.0.0, master, dev
• Operating System: Fedora 28
• Compiler+version: gcc 8.1.1

[onqtam]

Damn this seems like a major bug - thanks for reporting it!

I might have caused it when refactoring the internals between versions 1.2.8, 1.2.9 and 2.0.0 - so perhaps you can try with 1.2.8

Unfortunately I will have access to a computer after a month so until then I cant do anything.

[andijcr]

noticed the same bug. Seems a regression, i tested the 2.0.0, 1.2.8 and 1.2.9, and it appears only in the 2.0.0 release

[onqtam]

Hi!

I just fixed the bug and will release version 2.0.1 today. However I won't be able to attend to any other issues in the near month since I'm overwhelmed by work. Since this was an actual bug I felt it would be irresponsible of me to leave it unaddressed.