The following guide indicates how to run the tool against an implementation of one of the NMOS APIs.
| Description | Screenshot |
|---|---|
| Open a browser and go to the address of the server you deployed. |  |
| Select a test suite from the dropdown. |  |
| Provide the IP or hostname and port of the relevant API under test, along with the preferred API version. |  |
| Optionally, select a subset of the tests to run. This should default to 'all'. |  |
| Once all of the field have been completed, press 'Run'. The result of the tests will be shown after a few seconds. |  |
The result of each test case will be one of the following:
| Test Result | Reason |
|---|---|
 |
Successful test case. |
 |
Required feature of the specification has been found to be implemented incorrectly. |
 |
Not a failure, but the API being tested is responding or configured in a way which is not recommended in most cases. |
 |
Test is disabled due to test suite configuration; change the config or test manually. |
 |
Test was not run due to prior responses from the API, which may be OK, or indicate a fault. |
 |
Recommended/optional feature of the specifications has been found to be not implemented. |
 |
Test suite does not currently test this feature, so it must be tested manually. |
 |
Test is not applicable, e.g. due to the version of the specification being tested. |
The testing tool has a number of general configuration settings that affect its operation, defined in nmostesting/Config.py.
To alter these, copy nmostesting/UserConfig.example.py to nmostesting/UserConfig.py and make your changes there.
MAX_TEST_ITERATIONS can be set to restrict the maximum number of resources or test points that time-consuming tests run against.
The default of 0 means unlimited (all available resources or test points) for a really thorough test!
The following pages provide information on additional instructions for testing specific features:
The testing tool supports two modes of operation that support usage in larger scale or continuous integration tests.
Some of the tests contained within this tool perform a number of steps which may not be obvious without viewing the source code. To help with debugging, descriptions of such behaviour is covered by the advanced testing pages.