Enhance HITL (hardware in the loop) test

[HiFiPhile]

Related area

HITL test

Hardware specification

DCD only for the moment

Is your feature request related to a problem?

I'm going to add more boards into the loop, currently planned:

• LPCXpresso LPC54608 OM13092 (HS port)
• STM32F746DISCO (both HS & FS port)
• NUCLEO-L412KB (like now)

Meanwhile maybe a good idea to wait #1970 to avoid mixing boards.

I'm also open to sponsorship if anyone wants other boards to be added.

PS: Made a tiny hub as I can't find one :)

Describe the solution you'd like

Add more boards into HITL setup.

I have checked existing issues, dicussion and documentation

• I confirm I have checked existing issues, dicussion and documentation.

[hathach]

thank you, that is great to hear, I can implement #1970 soon enough, your current list of board is great, it already cover 3 different usb ips. I have an Pi4 with pico rp2040 running tests and plan to add more boards (since I have several devkit laying around), since these aren't computing heavy as compiling IAR :).

We will probably need to spend more time to write more actual hardware tesing script :) . Only tested cdc, msc, dfu so far. Haven't tried testing with audio.

PS: what's up with the hub breakout, wouldn't it look nicer with consumer hub :)

[HiFiPhile]

We will probably need to spend more time to write more actual hardware testing script

Yes I'm also thinking about it.

what's up with the hub breakout, wouldn't it look nicer with consumer hub :)

For space saving, with male plug directly soldered on the hub I don't need to deal with cable mess ;)

[tannewt]

Here are a couple hardware project for HITL. Thanks @esden for the pointers! https://github.com/probe-rs/hive https://github.com/esden/redcuttle

[hathach]

Here are a couple hardware project for HITL. Thanks @esden for the pointers! https://github.com/probe-rs/hive https://github.com/esden/redcuttle

thanks Scott, it looks very neat, but probably overkilled I guess. I would prefer to just stick the dev board in, most of them has on-board debugger anyway for flashing. It is of course just me :D

[hathach]

#2204 implement unique ID for several families including: rp2040 and various stm32 (including l4 and f7)

[HiFiPhile]

I've connected the boards.

3 debuggers are flashed to Jlink:

SN	Board	Device
770935966	STM32F746DISCO	STM32F746NG
774470029	NUCLEO-L412KB	STM32L412KB
727600775	OM13092	LPC54608J512

Both STM32L4 & STM32F7 works, while only HS port works on OM13092. (added missing -DBOARD_TUD_RHPORT=$(PORT))

Tried changing 54628 to 54608 in board.mk and I don't spot any USB related difference...

[hathach]

thank you 👍 , just saw them show up in the lsusb (ssh shell). I will try to update ci test script later on. Since you are often working with audio, maybe we should spend more time to write some script to capture the mic input and compare with knowngood etc... but that can always wait :)

[HiFiPhile]

Since you are often working with audio, maybe we should spend more time to write some script to capture the mic input and compare with knowngood etc... but that can always wait :)

It would be nice but compare audio is a little tricky due to PC resampling etc., I'll try to figure something out.

[HiFiPhile]

I've added a LPCXpresso4367 to replace 54608, it passes all tests with EA4357 BSP. I think it's good to have chipidea_hs tested.

PS1:
Can we modify Make to build both FS & HS option for stm32f746 ? Since HS use external phy I think it worth to test both.

PS2:
Potential null pointer dereference building host example for ea4357.

/home/tusb/tinyusb/src/portable/ehci/ehci.c: In function 'hcd_edpt_xfer.constprop':
/home/tusb/tinyusb/src/portable/ehci/ehci.c:459:5: error: potential null pointer dereference [-Werror=null-dereference]
  459 |     TU_VERIFY(!qhd->qtd_overlay.halted);
      |     ^
compilation terminated due to -Wfatal-errors.

qhd_get_from_addr() could return NULL.

[hathach]

ah thanks, I also have LPCXpresso43s67, I can help adding bsp and testing this out.

Can we modify Make to build both FS & HS option for stm32f746 ? Since HS use external phy I think it worth to test both.

Yeah, sure, I would love to, we will need

• add addtional json field for compile argument e.g PORT=0 (may need to duplicate f476 in json)
• update test scrip to build to build with extra compile argument and run test on f746

You don't have to do this, give me a bit of time, I will try to see what would be the best way to implement this.

Potential null pointer dereference building host example for ea4357.

weid, I don't seems to see that warnings, I will check it out later.

[hathach]

#2217 add lpc43s67 since it is what I got. I guess it is probably run fine with your non-s board. If it is not the case we can change the jlink device to non-s verion. Also add uuid support lpc43. Let me know if those are now working with your hil setup there :)

[josuah]

A type-C switcher allows to programmatically connect two devices pin-by-pin, rather than using a HUB).
Some USBC0 to USBC1, or some USBC0 to USBC2, either at a time.
https://www.digikey.com/en/blog/introducing-the-ultimate-usb-type-c-switch
https://www.codethink.co.uk/articles/2019/codethink-creates-custom-usb-3-switch-to-support-customers-cicd-pipeline-requirements/

This can help to automate host-mode operation:

1. Workstation <-> DUT: Load a firmware to the DUT (in device mode)
2. Keyboard <-> DUT: Test the DUT keyboard support (in host mode) and write the test results to flash
3. Workstation <-> DUT: Get the results from the flash over USB (in device mode)

[HiFiPhile]

@josuah

Thanks for your interest in HIL test.

A port switcher is not the priority, as currently all boards has a debugger attached who can read test results.

Currently all boards in the loop are bought by me or hathach, without a sponsorship we can't test all kinds of configurations.

[josuah]

Thinking again, if wanting to leave all boards connected at all time for an automated test script, a switcher of only the D+/D- and +5V lines would be a good compromise.

A port switcher is not the priority, as currently all boards has a debugger attached who can read test results.

Noted! This sounds like a better solution indeed.

For practical setup, regardless of the solution used, here is something that scales well and is cheap and quick to do:

A bit like a thick wallet of PCBs. Or an office binder made for PCBs.
Of course, everyone will have their own method and preference according to their material at hand and preferences!

Without a sponsorship we can't test all kinds of configurations.

Several projects might want a HITL test rig. As discussed on 1BitSquared Discord, there are several test project slowly getting built.

• https://github.com/esden/redcuttle
• https://github.com/probe-rs/hive
• https://renode.readthedocs.io/en/latest/introduction/supported-boards.html
• that project without a name discussed on the chat 1, 2
• [...]

It looks like a solution is already engineered-out at TinyUSB.

[HiFiPhile]

I think I have never showed how it's done:)

One 1 to 7 hub PCB and 3 boards separated by paper inside a cardboard box :

[hathach]

@josuah thank you for your suggestion, I only have plan to put each dev board to a plastic bag and try to push it under my desk as many as I could.

@HiFiPhile your setup is still much better than mine, just an RPI4 + bunches of dev board in plastic bags packed under my PC desk.

PS: IAR build failed due to expired license, I have contact them to renew our opensource license.

[hathach]

I have added quite a few board recently and upgrade the self-host to pi5. Cable is a mess but it is good enough, so I guess we could close this issue now.

[HiFiPhile]

Wow that's a lot !

[hathach]

Yeah, it is like 7 or 8 boards. Though some like metro m7 1011 does not play when with others when doing parallel testing (maybe power issue or something). I also got of usb bus funny problems but get them more stable now. I will try to add a few more, currently occupied all 3x7 usb hub.

I starts to do local testing with these locally while developing since it is rather convenient:)