Skip to content

Commit

Permalink
Add more documentation for creating and running tests (ni#21)
Browse files Browse the repository at this point in the history
* Add more documentation for creating and running tests

* More information local test configuration

* Edits based on ni#21 (comment)
  • Loading branch information
csjall authored Nov 19, 2018
1 parent bdc2c31 commit 1214983
Showing 1 changed file with 64 additions and 6 deletions.
70 changes: 64 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,80 @@

The **niveristand-custom-device-testing-tools** repository provides a common set of tools to automate testing NI VeriStand custom devices, including both unit and system tests. The intended audience includes custom device developers and integrators.

To get started with testing a custom device, create a VI Tester test case which derives from `VeriStandTestCase`. Example test cases will be available in supported custom devices in the future.


## LabVIEW Version

The source for this repository is written in LabVIEW 2015, but should be forward compatible.
The source for this repository is written in [LabVIEW 2015](http://www.ni.com/en-us/shop/labview.html), but should be forward compatible.


## Dependencies

The following top-level dependencies are required to build and use the repository:

- [LabVIEW Command Line Interface](http://www.ni.com/download/labview-command-line-interface-18.0/7545/en/)
- [NI VeriStand](http://www.ni.com/veristand/), matching the version of LabVIEW used to run the tests
- VI Tester JUnit XML Test Results >= 2.0.1.26
- [NI VeriStand](http://www.ni.com/veristand/) >= 2015
- [LabVIEW Command Line Interface](http://www.ni.com/download/labview-command-line-interface-18.0/7545/en/) >= 2018
- [VI Tester JUnit XML Test Results](vipm://jki_lib_vi_tester_junit_xml_results?repo_url=http://www.jkisoft.com/packages) >= 2.0.1.16


## Usage

The testing tools use [VI Tester](https://github.com/JKISoftware/JKI-VI-Tester/wiki) to execute automated tests. VI Tester makes it easy to write unit tests. The VeriStand extensions in the **niveristand-custom-device-testing-tools** repository make it possible to write system tests that deploy VeriStand system definition files as part of the test setup.


### Creating New Tests

To create a new unit test, create a VI Tester test case which derives from `TestCase`. A test case may define `TestCase.setUp` and `TestCase.tearDown` methods which run before and after each test. A test may call any of the `TestCase.pass` or `TestCase.fail` methods to assert preconditions and generate test failures if invalid.

To create a new system test, create a VI Tester test case which derives from `VeriStandTestCase`. A test case may define `VeriStandTestCase.setUp` which calls `VeriStandTestCase.OpenVeriStandConnection` to launch VeriStand and deploy a VeriStand system definition. In this situation, a test must define `VeriStandTestCase.tearDown` which calls `VeriStandTestCase.CloseVeriStandConnection` to undeploy the VeriStand system definition.

See the VI Tester documentation for more information on [creating new tests](https://github.com/JKISoftware/JKI-VI-Tester/wiki/Creating-New-Tests).


### Running Existing Tests

Test may be run from the LabVIEW editor or the command line. The LabVIEW editor is recommended for interactive test development and debugging. The command line is recommended for automated build and test systems.

To run existing tests from the LabVIEW editor, launch the VI Tester user interface by selecting `Tools > VI Tester > Test VIs...` from the LabVIEW menu. VI Tester will enumerate all the test cases in the project. Click the `Run all tests` or `Run selected test` button to run tests.

See the VI Tester documentation for more information on [running existing tests](https://github.com/JKISoftware/JKI-VI-Tester/wiki/Running-Tests-for-a-Project).

To run existing tests from the command line, run the VI Tester operation using `LabVIEWCLI.exe` and specify the path to one or more test cases:

```
LabVIEWCLI.exe
-OperationName RunVITester
-TestPath <Test case path>
[-ReportPath <XML report path>]
[-AdditionalOperationDirectory <RunVITester operation path>]
[-LabVIEWPath <LabVIEW executable path>]
```

To run the test cases for the **niveristand-custom-device-testing-tools** repository, use the following command line argument (assuming the **niveristand-custom-device-testing-tools** repository is cloned to the folder `C:\Git\VeriStand\niveristand-custom-device-testing-tools`):

```
LabVIEWCLI.exe
-OperationName RunVITester
-TestPath "C:\Git\VeriStand\niveristand-custom-device-testing-tools\VeriStandTestUtilities\Tests\Unit\VeriStandTestUtilitiesTests.lvclass"
-TestPath "C:\Git\VeriStand\niveristand-custom-device-testing-tools\VeriStandTestCase\Tests\Unit\VeriStandTestCaseTests.lvclass"
-AdditionalOperationDirectory "C:\Git\VeriStand\niveristand-custom-device-testing-tools\RunVITester"
```

Specify absolute file paths for the path parameters. Specify the optional LabVIEWPath parameter to launch a specific version of LabVIEW.


### Local Test Configuration

System definition files are commited with ephemeral configuration such as IP addresses. Instead of modifying the system definition files or test VIs on each test machine, a user can update these properties with a local configuration file. The local configuration file contains a list of overrides and optional test properties. The overrides are applied and the merged system definition file is deployed.

```
[Overrides]
"Targets/Controller/Custom Devices/SLSC/NI SLSC-12001/user.CD.Chassis IP Address" = "10.2.104.19"
[Properties]
"Loopback Resource Name" = "rio://10.2.104.35/RIO0"
```

By default the testing tools will load the local configuration file, named `config.ini`, which is saved next to system definition file. Alternatively a user can specify one or more local configuration files as an optional parameter to `VeriStandTestCase.OpenVeriStandConnection`.

## Git History & Rebasing Policy
Branch rebasing and other history modifications will be listed here, with several notable exceptions:
Expand Down

0 comments on commit 1214983

Please sign in to comment.