PEtab test suite

The PEtab test suite is a collection of test models for the PEtab parameter estimation data format. It is intended to be used to verify and quantify PEtab-support by developers of tools for model simulation and parameter estimation.

Downloading and installing the test suite

The PEtab test suite can be downloaded from GitHub via

git clone https://github.com/petab-dev/petab_test_suite

The test suite comes with all necessary files pregenerated.

Python library

The PEtab test suite comes with a Python package named petabtests in the homonymous subdirectory. It contains Python functions for generating the tests and evaluating results. This can be installed via

cd petab_test_suite
pip3 install -e .

Using the test suite

Files

The petabtests/cases subdirectory contains different test suites for different PEtab versions and model formats. Each test suite is a collection of enumerated tests. Each test consists of a single PEtab problem defined in XXXX/_XXXX.yaml file and the expected result (chi2 value, log-likelihood, simulation table reference, and tolerances) in XXXX/_XXXX_solution.yaml. XXXX/XXX.md contains a short description of the respective test problem that is not relevant for the execution of the test itself.

Evaluate results

To evaluate how a tool performs on a given test problem, three metrics are employed: Simulations, chi2 value and log-likelihood. A tool can be said to cover a test problem if any of those values matches the ground truth values up to some tolerance.

The Python package provides convenience functions for evaluation in petabtests/evaluate.py.

-> Overview of passed test cases for different tools supporting PEtab

Contributing

Contributions to the PEtab test suite are welcome.

Adding a new test case

To add a new test case, create a new subdirectory in the respective test suite directory under petabtests/cases/. The subdirectory name should be a four-digit number, starting with the next available number.

Most files in the test suite are generated automatically. Those start with an underscore, except for the README.md files. The only files that need to be created manually are the XXXX/XXXX.py files that contain the script to generate the test problem and solution files. Their content should be self-explanatory For adding a new test case, copy XXXX/XXXX.py from an existing test case and adjust it to the new test case.

All remaining files are generated by the petabtests_create script which will be available on your $PATH after installing the provided Python library (see above).