Enable multiple synchronized bitfiles in the RT engine

[buckd]

• This contribution adheres to CONTRIBUTING.md.

What does this Pull Request accomplish?

Allows a single APU to be split across multiple FPGAs so users requiring higher channel count and/or more FPGA fabric can still use the custom device.

Why should this Pull Request be merged?

There are several users that have requested this capability. It will allow NI to close more opportunities and make customers more successful.

What testing has been done?

Cursory testing ensuring FPGAs remain synchronized through use of the custom device and monitoring the newly added synchronization channels. The behavior of the channels mirrors my benchmarks outside of the custom device, just testing synchronization of the FPGAs using LabVIEW.

More testing still needs to be done, including validating I/O signal values and synchronization and automated tests still need to be written. I am currently working on this, but want to get the review going.

Note: There are other files that need to be updated due to typedef changes, but I'm not including them here because it would make this review even more unmanageable. It doesn't affect functionality, but build and load times will be longer because each of those VIs will need to recompile. I will update those with a separate pull request after this goes in.

[rtzoeller]

@buckd do we want to bump the version here? From my conversation with @csjall, I was under the impression we would bump the patch version for now (we did already, 1.3.3 -> 1.3.4), and bump the feature version (i.e. to 1.4.0) when we pulled it out from behind a feature toggle.

[buckd]

We can do it that way. I must have missed that discussion (and the fact it was already bumped).

[rtzoeller]

• Should the input and output references use an in place element structure here?
• Can you explain to me why this warning might occur?
  
• Is FPGA run method.vi unused? You did not modify it, but it is adjacent to VIs you were touching.

[rtzoeller]

• Remove the pop-up behavior of the compiled data on deployment

[buckd]

• Should the input and output references use an in place element structure here?

They could, but it's not necessary. This was copied from the single FPGA VI. The unbundle is only used to get the type for describing the refnum and the bundle then goes into the rest of the RT data. Since this is during initialization, it's not time critical.

• Can you explain to me why this warning might occur?
  

Let's say the Engine Simulation Toolkit IP isn't the only thing running on the FPGA. Maybe there's an SLSC backend or some other custom IP compiled to the same bitfile in order to utilize a single FPGA board to accomplish multiple things. During deployment, something else could run the bitfile before the Engine Simulation custom device gets to this code.

• Is FPGA run method.vi unused? You did not modify it, but it is adjacent to VIs you were touching.

This appears to be unused. Removed from the library and repo.

[csjall]

• Add a system test that deploys multi-FPGA system definition and check synchronization. Look at the EDS System tests for an example.

[rtzoeller]

• To enable testing, @rtzoeller needs to make the multi-FPGA feature toggle take the system definition feature toggle flag into consideration, i.e. make the custom device use the multi-fpga version regardless of the ini file's setting if it is already configured for multi-fpga.

[buckd]

@csjall Test added. Results below.
Note there is no config override of the IP Address due to this issue.

[csjall]

Since the test hardware is only supported under LabVIEW 2018 or later, we should skip the test if we are running in earlier versions. Be sure to skip deployment and undeployment as well. Add a skip message for more information.

[csjall]

Since the test hardware is only supported under LabVIEW 2018 or later, we should skip the test if we are running in earlier versions. Be sure to skip deployment and undeployment as well. Add a skip message for more information.

Go ahead and submit changes as is. Create a separate tech debt issue to skip this test on earlier year versions.

[buckd]

Since the test hardware is only supported under LabVIEW 2018 or later, we should skip the test if we are running in earlier versions. Be sure to skip deployment and undeployment as well. Add a skip message for more information.

Issue filed.