Dev/performance

[csjall]

[x] This contribution adheres to CONTRIBUTING.md.

What does this Pull Request accomplish?

Add system definition benchmark test. Test will fail if target loop rates are not achieved. Refactored some of the system definition file manipulation to enable modifying the target loop rate.

Why should this Pull Request be merged?

This is a general purpose benchmark test utility that can be used by multiple custom devices.

What testing has been done?

Added a test for new utility method. All tests are green.

[niveristand-diff-bot]

Bleep bloop!

LabVIEW Diff Robot here with some diffs served up hot for your pull request.

Notice something funny? Help fix me on my GitHub repo.

VeriStandTestCase.lvclass--4dELq9_SetMultipleChannelValues.vi.png:

VeriStandTestCase.lvclass--sIMIBa_OpenVeriStandConnection.vi.png:

VeriStandTestCase.lvclass--uHZ8lb_OpenSystemDefinitionFile.vi.png:

VeriStandTestUtilities.lvlib--m9Wqwb_QuerySystemDefinitionFile.vi.png:

[rtzoeller]

• @csjall you added a test that makes sure benchmarking passes for a trivial system definition with a low target rate and a high tolerance. Can we also add one that makes sure benchmarking fails when we request an impossibly-high target rate?

[rtzoeller]

• Two of the VIs have the same icon (GetSystemChannelValues.vi and GetSystemChannelValuesForDuration.vi). Can we differentiate them somehow?

[rtzoeller]

• In GetSystemChannelValuesForDuration.vi you are searching a 1D array to check for duplicates. Given that the system time is monotonically increasing, it is sufficient to compare with only the last element of the array.

[rtzoeller]

• Should GetSystemChannelNames.vi be called GetSystemChannelFullPaths.vi instead? The use case appears to be "given a known set of names, get the associated full paths".

[csjall]

The benchmark tests are super flaky on Windows host. Most of the times it passes, but sometimes it fails with HP count != 0. We should run on RT target or get rid of these tests altogether.

[rtzoeller]

I am fine with requiring some sort of RT host to run these tests, assuming we don't require anything else beyond VeriStand. We could allow the user to specify the target OS in an overrides file, which would mitigate pain associated with requiring an RT target for the testing tools.