Example Standard Output Exit Code And Files

Assume you want to test your new hello-world program, which is an executable file.

You can do this by writing a test case file, located in the same directory as your hello-world executable:

[act]

hello-world

[assert]

stdout equals <<EOF
Hello, World!
EOF

exit-code == 0

If the file hello-world.case contains this text, you can execute the test:

> exactly hello-world.case
PASS

Maybe you prefer to have the expected output in a separate file:

[act]

hello-world

[assert]

stdout equals -contents-of hello-world-output.txt

exit-code == 0

If hello-world-output.txt is found in the same directory as the test case file, and contains the expected output, then the test will pass:

> exactly hello-world.case
PASS

Maybe you are not interested in the whole output, but just that it contains the word 'Hello':

[act]

hello-world

[assert]

stdout any line : contents ~ 'Hello'

The output should also satisfy the stronger assertion that every line contains 'Hello':

[act]

hello-world

[assert]

stdout every line : contents matches 'Hello'

The contents of stderr, and also arbitrary files may be checked the same way as stdout.

This version of hello-word has an option for storing the output in a file:

[act]

hello-world -o output.txt

[assert]

contents output.txt :
         every line : contents matches 'Hello'

stderr is-empty

It might be a good idea to explicitly check that the expected file has been created:

[act]

hello-world -o output.txt

[assert]

exists output.txt : type file

contents output.txt :
         every line : contents matches 'Hello'

Sometimes the output from a program needs to be transformed before you can apply your assertion:

[act]

hello-world

[assert]

stdout -transformed-by ( char-case -to-upper | replace '^HELLO, ' '' )
       every line : contents matches ^WORLD!$

A transformation is only applied for the purpose of the assertion, it does not modify the transformed file. So both of the following assertions on stdout will pass:

[act]

hello-world

[assert]

stdout -transformed-by ( char-case -to-upper | replace '^HELLO, ' '' )
       every line : contents matches ^WORLD!$

stdout equals <<EOF
Hello, World!
EOF

If you need to do multiple assertions on some transformation of a file, it might be convenient to store the result of the transformation and then do the assertions on the stored result.

Exactly does not have a concept of variables (unfortunately), so the transformation needs to be stored in a new file:

[act]

hello-world

[before-assert]

file transformed-output.txt = -contents-of
                              -rel-result stdout
                              -transformed-by ( char-case -to-upper | replace '^HELLO, ' '' )

[assert]

contents transformed-output.txt :
         every line : contents matches ^WORLD!$

contents transformed-output.txt :
         num-lines == 1

In the example above, the special file stdout is transformed. It is located in the special "result" directory, which is referenced using the option -rel-result - the path is relative the "result" directory.

And since a test case is executed in a temporary sandbox directory, the temporary file transformed-output.txt will be automatically removed and will not pollute your source folder.

To be able to reuse a transformer, or just to make the test more readable, you may define a "symbol" for it:

[act]

hello-world

[assert]

def text-transformer MY_TRANSFORMER = char-case -to-upper | replace '^HELLO, ' ''

stdout -transformed-by MY_TRANSFORMER
       every line : contents matches ^WORLD!$

The same can be done with "line matchers":

[act]

hello-world

[assert]

def text-transformer MY_TRANSFORMER  = char-case -to-upper | replace '^HELLO, ' ''

def line-matcher     MY_LINE_MATCHER = contents matches ^WORLD!$

stdout -transformed-by MY_TRANSFORMER
       every line : MY_LINE_MATCHER

To make the assertions clearer, you can put the symbol definitions in a different phase. The order of the phases in the test case file is irrelevant, so [assert] may appear before [before-assert], even though the execution order is the opposite.

Also the "act" phase is the default phase, so it does not need to be declared if the action to check appears at the beginning of the file.

hello-world -o output.txt

[before-assert]

def text-transformer MY_TRANSFORMER  = char-case -to-upper | replace '^HELLO, ' ''

def line-matcher     MY_LINE_MATCHER = contents matches ^WORLD!$

[assert]

contents output.txt :
         -transformed-by MY_TRANSFORMER
         every line : MY_LINE_MATCHER

stderr is-empty