Skip to content

Developer Notes

Jump to bottom
Alberto Goffi edited this page Oct 28, 2016 · 71 revisions

Building Toradocu

To compile Toradocu, run the command: ./gradlew shadowJar This will create the file build/libs/toradocu-1.0-all.jar. Notice that building Toradocu requires Java JDK 1.8+.

Faster build times

It takes a long time to compile Toradocu with the command ./gradlew shadowJar, largely because of the time to create the massive final .jar file. Once you have made the shadowJar file, if you edit the Toradocu source code, it is much faster to run ./gradlew jar to create file build/libs/toradocu-1.0.jar.

Using this method, you need to slightly change the way that you run Toradocu: java -cp toradocu-1.0.jar:toradocu-1.0-all.jar org.toradocu.Toradocu ARGS

You should use this technique if:

  • you have already made the shadowJar in the past,
  • you have changed Toradocu's source code, and
  • you have not changed Toradocu's external dependencies. Whenever you change Toradocu's external dependencies, you need to remake the shadowJar.

Precision/recall test suite

Running tests

To run a single precision/recall test: ./gradlew test --tests org.toradocu.<test suite name>. For example: ./gradlew test --tests org.toradocu.PrecisionRecallGuava19

To run all the precision/recall tests, use the command: ./gradlew test --tests 'org.toradocu.PrecisionRecall*'

Precision and recall values are printed to standard output during a precision/recall test. A complete test report with the standard output and standard error produced by each executed test case can be found in build/reports/tests/test/index.html.

The precision/recall test suites are always run during ./gradlew build. This is primarily to find regression errors during continuous integration.

If the tests fail

If the tests fail, and you don't know exactly why, then you might want to know how your changes to Toradocu are affecting the output. You can do so by generating output from the original version of Toradocu, save it, and compare that to the current output.

To see the goal and actual output for each test where Toradocu does not produce the goal output:

./gradlew test --tests 'org.toradocu.PrecisionRecall*' | grep "condition\. Comment"

Adding new tests

Setting up resources

Adding new test cases requires both the source code and a .jar file containing compiled .class files for the system to test.

A way to install these is to update the {download,extract}{Sources,Binaries} Gradle tasks which will be run before tests. (Alternately, you can manually place the .jar file or a directory containing binaries of the class to test in src/test/resources/bin/, and place the source files of the class to test in src/test/resources/src/.)

Creating goal output files

The precision/recall test suite works by comparing the actual and goal output from Toradocu's condition translator.

First, create an empty directory src/test/resources/goal-output/SYSTEM-NAME/, such as src/test/resources/goal-output/plume-lib-1.1.0/.

The condition translator output is formatted as JSON. You will need to create goal output files in the directory you created above. These files can be created using a command like the following:

java -jar build/libs/toradocu-1.0-all.jar \
	--class-dir src/test/resources/bin/plume.jar \
	--source-dir src/test/resources/src/plume-lib-1.1.0/java/src \
	--target-class plume.ArraysMDE \
	--condition-translator-output src/test/resources/goal-output/plume-lib-1.1.0/plume.ArraysMDE_goal.json

Then, edit this file to indicate goal output. To make this editing task easier, when the --condition-translator-output option is used, Toradocu also prints (to standard output) the line numbers of all lines that may have to be modified within the actual output file in order to form the goal output file.

Creating the test case source file

Create a new class in directory toradocu/src/test/java/org/toradocu that extends AbstractPrecisionRecallTestSuite.

Within the constructor for this class, call the superclass's constructor, passing in the path to the sources, the path to the binaries, and the path to the goal output files. See the Javadoc for AbstractPrecisionRecallTestSuite for more details on the constructor parameters.

Then, for each class that has an goal output file, create a JUnit test case, such as:

@Test
public void testConcurrentHashMultiset() throws Exception {
  test("com.google.common.collect.ConcurrentHashMultiset", 0.818, 0.692);
}

Summary of precision/recall tests

To view a summary of the most recently run precision/recall tests, run python3 summarize_precision_recall_tests.py. Note that Python 3 is required to run the script. This will output a summary to standard output which can then be used to update the precision_recall_tests.xlsx spreadsheet. This spreadsheet stores a history of previous precision/recall test results and can be used to track Toradocu's performance over time.

Code style

The source code of this project follows the Google Java Style Guide. The style of the code is automatically checked by a Git pre-commit hook. Also, the hook prevents any commits of files containing trailing spaces. The hook is automatically installed as soon as any Gradle task is executed.

If a commit fails due to improper formatting in a source file, you can format your code according to the style guide using the following command:

python run-google-java-format.py [file1 file2 ...]
Clone this wiki locally