When you make Cataclysm from source, an executable tests/cata_test is built
from test cases found in the tests/ directory. These tests are written in the
Catch2 framework.
Run tests/cata_test --help to see the available command-line options and/or
consult the Catch2 tutorial
for a more thorough introduction.
When creating tests, ensure that all objects used (directly or indirectly) are fully reset before testing. Several tests have been rendered flaky by properties of randomly generated objects or interactions between tests via global objects (often the player object). As a general guideline, test cases should be standalone (one test should not rely on the output of another).
When generating objects with json definitions, use REQUIRE statements to assert the properties of the objects that the test needs. This protects the test from shifting json definitions by making it apparent what about the object changed to cause the test to break.
You can choose several ways to organize and express your tests, but the basic
unit is a TEST_CASE. Each test .cpp file should define at least one test
case, with a name, and an optional (but strongly encouraged) list of tags:
TEST_CASE( "sweet junk food", "[food][junk][sweet]" )
{
// ...
}Within the TEST_CASE, the Catch2 framework allows a number of different
macros for logically grouping related parts of the test together. One approach
that encourages a high level of readability is the
BDD
(behavior-driven-development) style using GIVEN, WHEN, and THEN sections.
Here's an outline of what a test might look like using those:
TEST_CASE( "sweet junk food", "[food][junk][sweet]" )
{
GIVEN( "character has a sweet tooth" ) {
WHEN( "they eat some junk food" ) {
THEN( "they get a morale bonus from its sweetness" ) {
}
}
}
}Thinking in these terms may help you understand the logical progression from
setting up the test and initializing the test data (usually expressed by the
GIVEN part), performing some operation that generates a result you want to
test (often contained in the WHEN part), and verifying this result meets your
expectations (the THEN part, naturally).
Filling in the above with actual test code might look like this:
TEST_CASE( "sweet junk food", "[food][junk][sweet]" )
{
avatar dummy;
dummy.clear_morale();
GIVEN( "character has a sweet tooth" ) {
dummy.toggle_trait( trait_PROJUNK );
WHEN( "they eat some junk food" ) {
item necco( "neccowafers" );
dummy.eat( necco );
THEN( "they get a morale bonus from its sweetness" ) {
CHECK( dummy.has_morale( MORALE_SWEETTOOTH ) >= 5 );
}
}
}
}Let's look at each part in turn to see what's going on. First, we declare an
avatar, representing the character or player. This test is going to check the
player's morale, so we clear it to ensure a clean slate:
avatar dummy;
dummy.clear_morale();Inside the GIVEN, we want some code that implements what the GIVEN is
saying - that the character has a sweet tooth. In the game's code, this is
represented with the PROJUNK trait, so we can set that using toggle_trait:
GIVEN( "character has a sweet tooth" ) {
dummy.toggle_trait( trait_PROJUNK );Now, notice we are nested inside the GIVEN - for the rest of the scope of
that GIVEN, the dummy will have this trait. For this simple test it will
only affect a couple more lines, but when your tests become larger and more
complex (which they will), you will need to be aware of these nested scopes and
how you can use them to avoid cross-pollution between your tests.
Anyway, now that our dummy has a sweet tooth, we want them to eat something
sweet, so we can spawn the neccowafers item and tell them to eat some:
WHEN( "they eat some junk food" ) {
dummy.eat( item( "neccowafers" ) );The function(s) you invoke at this point are often the focus of your testing;
the goal is to exercise some pathway through those function(s) in such a way
that your code will be reached, and thus covered by the test. The eat
function is used as an example here, but that is quite a high-level, complex
function itself, with many behaviors and sub-behaviors. Since this test case is
only interested in the morale effect, a better test would invoke a lower-level
function that eat invokes, such as modify_morale.
Our dummy has eaten the neccowafers, but did it do anything? Because they
have a sweet tooth, they should get a specific morale bonus known as
MORALE_SWEETTOOTH, and it should be at least 5 in magnitude:
THEN( "they get a morale bonus from its sweetness" ) {
CHECK( dummy.has_morale( MORALE_SWEETTOOTH ) >= 5 );
}This CHECK macro takes a boolean expression, failing the test if the
expression is false. Likewise, you can use CHECK_FALSE, which will fail if
the expression is true.
While the CHECK and CHECK_FALSE macros make assertions about the truth or
falsity of expressions, they still allow the test to continue, even when they
fail. This lets you do several CHECKs, and be informed if one or more of
them do not meet your expectations.
Another kind of assertion is the REQUIRE (and its counterpart
REQUIRE_FALSE). Unlike the CHECK assertions, REQUIRE will not continue if
it fails - this assertion is considered essential for the test to continue.
A REQUIRE is useful when you wish to double-check your assumptions after
making some change to the system state. For example, here are a couple of
REQUIREs added to the sweet-tooth test, to ensure our dummy really has the
desired trait, and that the neccowafers really are junk food:
GIVEN( "character has a sweet tooth" ) {
dummy.toggle_trait( trait_PROJUNK );
REQUIRE( dummy.has_trait( trait_PROJUNK ) );
WHEN( "they eat some junk food" ) {
item necco( "neccowafers" );
REQUIRE( necco.has_flag( "ALLERGEN_JUNK" ) );
dummy.eat( necco );
THEN( "they get a morale bonus from its sweetness" ) {
CHECK( dummy.has_morale( MORALE_SWEETTOOTH ) >= 5 );
}
}
}We use REQUIRE here, because there is no reason to continue the test if these
fail. If our assumptions are wrong, nothing that follows is valid. Clearly, if
toggle_trait failed to give the character the PROJUNK trait, or if the
neccowafers turn out not to be made of sugar after all, then our test of the
morale bonus is meaningless.
You can think of REQUIRE as being a prerequisite for the test, while CHECK
is looking at the results of the test.