[enhancement] Add support for "stand-alone assertions".

[aghast]

Description

The features that make doctest attractive for unit tests also make it attractive for other uses: it is cheap to include, lightweight, and low-cost in terms of compilation speed and performance impact.

I'd like to see a version of doctest that supports using the existing macros and code to run "stand alone" assertions. That is, I want to be able to use all the various assertion macros in my mainline code:

double sqrt(double x, double epsilon)
{
    FAST_CHECK_GE(x, 0.0);
    FAST_CHECK_LE(epsilon, 0.01);
    // ...
}

In order to do this, the reporting has to change - I'm not quite clear how. I made one attempt at just "faking" a test case in my own main() and then calling my program() function. This unfortunately mixes the output of my program with the output from doctest, so it's not quite a solution. (I wanted to make sure that all the regular data structures that doctest maintains internally were active while it was running.)

Keeping in mind that the idea is to use doctest for both in-source unit testing and stand-alone assertions, I suspect the solution will either involve a new set of macros, or will involve checking the "running in test" status prior to dispatching an error report. The existing macro-based solution for logging will obviously only support all-or-nothing changes, rather than "test mode" and "run mode".

[onqtam]

That's interesting - I'll think about it, but it probably won't happen for version 1.3...
And sorry for the late reply :)

[onqtam]

Hi! I'm actually thinking about this for the upcoming 1.3 release (hopefully within a month)...

But such inline asserts can come in many forms:

• https://github.com/gpakosz/PPK_ASSERT - quite extensible
• https://github.com/foonathan/debug_assert - also powerful
• https://github.com/boostorg/assert/blob/develop/doc/assert.adoc
• https://github.com/banditcpp/snowhouse
• https://github.com/bloomberg/bde/blob/master/groups/bsl/bsls/bsls_assert.h
• https://github.com/Softwariness/ThrowAssert
• http://cnicholson.net/2009/02/stupid-c-tricks-adventures-in-assert/
• https://www.softwariness.com/articles/assertions-in-cpp/
• and many others...

what set of features would you like to see implemented..? for example: user defined callback functions...?

what would make sense for doctest?

[aghast]

First, my "vision" for this request is simply that the doctest unit test assertions represent a constraint DSL. Given that I want to impose constraints elsewhere in my code (preconditions, invariants, etc.) it makes sense to use the same language so as to reduce confusion. So the priority then is for the "test" statements and the "assert" statements to have exactly the same look and to have exactly the same meaning.

With that said, what makes sense for doctest is what currently makes sense for doctest: simple, lightweight, fast. Anything that violates one of those, is wrong.

Ideally, the same macros would be used, because that's simple. They would just magically figure out whether they were in a test or not. Otherwise, if somehow that is not possible, the minimum different macros should be available: FAST_ASSERT_LE(x, y) or some such.

If something more complex is needed, it should be an optional extra configuration (like implementing parameter parsing in my own main() function is "optional" and "extra"). So if I want the "assertion failed" to call a user defined function, it should be either a separate (optional) #define or an extra configuration statement or something.

The essence of assertions is that they (1) can be configured on or off; and (2) they terminate the program on failure.

Point (1) seems like a no-brainer. There should be some option -DDOCTEST_NO_ASSERT or whatever, that compiles support for assertions right out.

Point (2) is less certain. I have seen some assert libraries that support "levels" much the way the logging does, with "FATAL" assertions and "DEBUG" assertions. I would suggest leaving that for a "version 2", since I'm sure there are some tricky bits (such as the inevitable "how can I replace your log library with my own log library") that would need to be dealt with.

I don't have any secret desire for controlling how the assertions are handled, because in my own uses cases an assertion failures means the program is borked and I don't care how long it takes any more, just give me a good report. Obviously, other people will have different views on that. But I favor the attributes mentioned above over some complex beast that can report failures using TCP over barbed wire.

[onqtam]

just implemented this in the dev branch! The fast asserts are optimized for speed - all other asserts will work as well but I would use them sparingly in production code. So now you can compile all your production code once and have doctest asserts in it - either for production builds or test builds!

custom handlers should look something like this:

void handler(const doctest::AssertData& ad) {
    ((void)ad.m_file); // access data about the assert - this is called only on failure
    ((void)ad.m_line);
    ((void)ad.m_expr);
}

do this on a doctest::Context somewhere:

context.setAsDefaultForAssertsOutOfTestCases();
context.setAssertHandler(handler);

if no handler is set (but a context is set as a default one - otherwise will crash) - then std::abort() is called when an assert fails.

documentation and an example will follow soon! Should also do it for the new reporter system...

I may also change the names before release. Let me know what you think!

EDIT:

https://github.com/onqtam/doctest/blob/dev/doc/markdown/assertions.md#using-asserts-out-of-a-testing-context

[nuzrub]

This was very helpful to me, thanks :) I hope this will be added to the documentation at somepoint (or maybe it already has but I just couldn't find it).

[onqtam]

@nuzrub here it is in the documentation along with an example :)

Perhaps the documentation isn't structured very well - people often don't find something...