The BevoTest framework is a unit testing framework. It executes test procedures on test items. Each test procedure's input and expected result is specified by a test case. An execution of a test records test execution results in a test log.
BevoTest handles test cases that expect returned values or expect particular exceptions to be thrown. It is robust to test items that throw unexpected exceptions or errors, that don't timely terminate (infinite looping or very slow test items), and that attempt security violations.
The BevoTest framework conforms to IEEE Std 829, IEEE Standard for Software Test Documentation.
BevoTest is licensed under the Apache License, Version 2.0.
A source code JAR and a class file JAR are available. See the latest release for the files to download.
The framework is documented on this page and in BevoTest JavaDoc.
There are four steps to using BevoTest:
- Create a
Test: This is just a named container for test cases. - Specify test cases: Create some instances of
TestReturnsorTestThrows. Merely creating one of these with the correct arguments adds the test case to the test. - Run test: Create a
TestLogand hand it to theTest.runmethod. - Report results: Use a test reporter on the test log. There is a
PlaintextTestReportersupplied with BevoTest, or supply your own.
Here is a file with examples of BevoTest being used: ExampleTest.java
To specify a test case expecting a particular return value:
new BevoTest.TestReturns<String, Integer>(ts, String.class, "Length of String", 14, 2000L) {
@Override public void executeTest() {
// Set up:
final String testItem = "Test test test";
// Execute:
starting(testItem);
returned(testItem.length());
// Any tear down here
}
};The arguments to the TestReturns constructor are two type arguments: the type being tested and the expected value type; and five value arguments: the Test, the Class value of the type being tested, the test case description, expected return value, and timeout in milliseconds. In the body of the test procedure, there are three phases: 1) set up of the test case, 2) execution of the item being tested, and 3) tear down (clean up). The calls to starting and returned divide the phases of the test procedure.
If the test case, instead of expecting a returned value, expects a particular type of exception to be thrown, construct a TestThrows instead of a TestReturns.
If the method being tested is a static method (or a constructor invocation, or a static initializer, or the variable initializer of a static variable), where there is no test item instance to pass to starting, call startingStatic instead, passing the Class under test.
If the test case may need to be skipped under certain conditions, override shouldSkip().
There is more detail in the BevoTest JavaDoc.
Test results are recorded in a test log. The log keeps some environmental information, a test start and stop time, and a collection of test log entries. Specifically, the environmental description includes: JVM version, heap size, OS, class path, and working directory.
Test results entries in the log keep a reference to the test case, the type of the item tested, the actual returned value or any exception thrown, the time the test took to run, a test status, and a test evaluation (pass/fail). Specifically, the statuses are: Enqueued, Running setup, Running processing, Running teardown, Skipped, Complete normal, Complete abnormal, and Timed out.
To convert a test log into a plain text test report, use the PlaintextTestReporter class. This class has a few options to change the output format, and you can write your own reporter if you like.
The TestLog class provides hooks for a status UI: subclass and override the three notify... methods. The log and entries are thread safe.
The framework also supplies a security policy utility class that simplifies adding permissions to the policy, without the hassle of a security policy configuration file.
There is more detail in the BevoTest JavaDoc.
Background: BevoTest handles test case executions that exceed the time limit by using certain Java functions that require Java Virtual Machine "safe point" checks. Unfortunately, these checks are not inserted into loops that are considered too insignificant by the Java Virtual Machine to warrant the overhead of safe point checks. These is mainly loops with int counters. If the code under test exceeds the timeout in one of these loops, BevoTest's attempts to terminate the test will not work. Fortunately, we now have a fix.
Update to Java SE 8 Update 91 or later, and add the -XX:+UseCountedLoopSafepoints VM argument to your java command.