sw sample when multiple FPGAs exist in the system

[Yang-YiFan]

Hi,
I'm trying to write an accelerator in a system with 2 PAC cards. However, I'm not sure how to handle the sw part. I'm following the 03 linked list tutorial. From my understanding, I should instantiate two OPAE_SVC_WRAPPER objects representing two FPGAs (AFUs) in the system with different UUID. So I have to synthesize the same AFU twice with different UUID in hw/rtl/linked_list_cci.json. Then I can

OPAE_SVC_WRAPPER fpga1(AFU_ACCEL_UUID1);
OPAE_SVC_WRAPPER fpga2(AFU_ACCEL_UUID2);

and play with two FPGAs.
Does the flow I described above correct? If not, could you please direct me to the correct path and offer some code samples?
Thanks!

[michael-adler]

Are the AFUs intended to be identical? If so, you can use the same UUID on both and load both FPGAs using the same image. In the simple examples, like 01_hello_world, the software will always find the same FPGA and would fail if it is busy. That is only because fpgaEnumerate is called with only a single entry. The 03 example is more sophisticated, though it uses C++. In findAndOpenAccel (https://github.com/OPAE/intel-fpga-bbb/blob/master/samples/tutorial/03_ccip/base/sw/opae_svc_wrapper.cpp#L115) it loops through a list of returned accelerator tokens and keeps trying until opening one succeeds. This code would successfully open 2 accelerators. The C++ is just calling the OPAE SDK C library, so you can use either language.

The enumeration allows you to explicitly name the PCIe ID of a card, too, so there are many ways you can tackle the problem of finding a specific PAC card when multiple are running the same AFU UUID.

[Yang-YiFan]

Yes, I want to use the same image. So you are suggesting

OPAE_SVC_WRAPPER fpga1(AFU_ACCEL_UUID);
OPAE_SVC_WRAPPER fpga2(AFU_ACCEL_UUID);

with the same UUID will work?

[michael-adler]

Yes, it should, though I've never tried that using the example code.

[Yang-YiFan]

Great! I'll try that out and update you if it works. Thanks!
btw I suggest adding a few descriptions about multiple FPGAs (PAC) support in the document are added somewhere. Someone may encounter a similar issue.

[Yang-YiFan]

I've tried the method you mentioned but got segmentation fault. The code is pretty straight forward (using the 03 example):

vector<OPAE_SVC_WRAPPER> fpga;
fpga.push_back(OPAE_SVC_WRAPPER(AFU_ACCEL_UUID));
assert(fpga[0].isOk());
auto buf_handle = fpga[0].allocBuffer(getpagesize());
return 0;

ASE will issue SIGSEGV when I try to call allocBuffer. The reason why I use a vector is that there is no default constructor for class OPAE_SVC_WRAPPER, so I cannot simply create an array of it. Could you please suggest some possible solutions about this?
Thanks!

[michael-adler]

The code in samples is just a starting point. You can copy and restructure it however you like, including changing the way constructors are used.

Unfortunately, you will find that ASE does not currently support two connections from the same process. FPGAs should work, but not ASE. I think there are only a few problems to fix. The first problem is that ASE is found using an environment variable. The second is that there are some static variables inside ASE to remove. I will put those on the list of problems we have to fix.