Unit Testing not working on ESP32 (test port not automatically recognized) #4076

maxgerhardt · 2021-10-04T13:53:14Z

What kind of issue is this?

PlatformIO Core.
If you’ve found a bug, please provide an information below.

You can erase any parts of this template not applicable to your Issue.

Configuration

Operating system: Win10 x64

PlatformIO Version (platformio --version): version 5.2.1b4

Description of problem

Unit testing for an ESP32 hangs at the "Testing..." phase.

Steps to Reproduce

Create new blank ESP32 project
Add test/simple_test.cpp from below
Use the project task "Advanced -> Test"

Actual Results

Writing at 0x0003fafb... (100 %)
Wrote 210992 bytes (108452 compressed) at 0x00010000 in 3.1 seconds (effective 536.3 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

(hangs forever)

Expected Results

Successfull unit test result.

If problems with PlatformIO Build System:

The content of platformio.ini:

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino

Source file to reproduce issue:

test\simple_test.cpp

#include <Arduino.h>
#include <unity.h>

void test_led_builtin_pin_number(void) {
    TEST_ASSERT_EQUAL(13, 13);
}

void setup() {
    delay(3000);
    UNITY_BEGIN();
    RUN_TEST(test_led_builtin_pin_number);
    UNITY_END(); // stop unit testing
}

void loop() { }

Additional info

The board is a standard https://www.amazon.de/ESP-32S-ESP32-Development-Bluetooth-Antennenmodul/dp/B071JR9WS9
pio test locks up so that it can't even killed by Ctrl+C, must be killed in the taskmanager
Increasing the delay() does not help
Explicitly setting test_speed = 115200 does not help
Modifying setup() to Serial.begin(115200); Serial.print("\n\n"); does not help (separates bootloader output from test output)
Setting DTR / RTS states with pio test --monitor-dtr <0/1> --monitor-rts <0/1> does not help
Pressing the reset button on the board does not help
Looking at the serial output after the testing has been aborted at the waiting phase gives

The text was updated successfully, but these errors were encountered:

valeros · 2021-10-05T10:44:07Z

Hi @maxgerhardt ! I guess it works fine, except that the esp32dev board doesn't have any hwids in its manifest so PIO couldn't find a proper serial port and hence waits for a timeout to exceed. If you wait a bit longer it should print the Error: Please specify test_port for environment or use global --test-port option message. Could you please try to manually specify the test_port option in your platformio.ini?

Although, during some attempts to reproduce the issue I indeed bumped into situations when the test command sporadically became unresponsive even to the Ctrl+C signal, but it seems like a more generic problem that has nothing to do with ESP32 specifically.

maxgerhardt · 2021-10-05T10:47:36Z

That was indeed the problem.

Uploading...
Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

ets Jun  8 2016 00:22:57
rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:10124
load:0x40080400,len:5828
entry 0x400806a8
test\simple_test.cpp:12:test_led_builtin_pin_number     [PASSED]
-----------------------
1 Tests 0 Failures 0 Ignored
===================== [PASSED] Took 30.94 seconds =====================
Test    Environment    Status    Duration
------  -------------  --------  ------------
*       esp32dev       PASSED    00:00:30.943
===================== 1 succeeded in 00:00:30.943 =====================

That was quite baffling because I thought the detection of the test_port would be the same as the automatic detection of the upload_port, which worked flawlessly for me. I only have my motherboard's COM port and the ESP32 dev board as serial ports.

ankostis · 2022-04-30T08:02:22Z

@maxgerhardtI would edit the title to reflect the content of this report, as a courtesy to future bug-reporters.

maxgerhardt · 2022-04-30T11:05:36Z

I'm not sure what title would be better? The problem of unit testing not working was resolved by setting test_port.

ankostis · 2022-04-30T11:09:28Z

Maybe "test_port automatic detection fails"?

maxgerhardt · 2022-04-30T11:14:51Z

People who will experience the problem will first think of "Unit testing not working" instead of knowing the solution that test port detection fails though :/

ankostis · 2022-04-30T12:03:16Z

The message i'm conveying is that if all kinds of issues have generic titles "foo don't work", "bar crashes", it's difficult to orient when scanning through them. Eg when looking at search results, the results are matched by GitHub based on the contents of the issues (not just the titles), but it's only the titles that's present in the result-list, and these better help humans to remember which one is which, and avoid revisiting the same issue again and again.

Certainly it's not that important all this discussion, still you could extend the current title with some identifying info, if you agree with the above perspective. Thank you for your time.

maxgerhardt · 2022-04-30T12:23:13Z

Updated

HareshPrajapati · 2022-05-13T12:19:05Z

Hi @maxgerhardt ,
Thank you for solving problem , but setting test_port in the platformio.ini file solved problem only for unit test not for google test !
Currently I m doing Google test using 'esp32dev' board , some how test run in the board but it hangs on one task I mean I try to test serval unit tests and google tests in same project and when google test executed other tests not executing after that.
surprisingly , All tests on native environment works perfectly one by one but not working in esp32dev board

I m using google/googletest@^1.11.0 library.

This is my configuration for both env (See below image ):

This is the problem (See below image):

ivankravets · 2022-05-13T12:26:15Z

Please remove googletest from lib_deps
Switch to the latest PlatformIO Core 6.0 (pio upgrade --dev)
Set test_framework = googletest
See new docs https://docs.platformio.org/en/latest/advanced/unit-testing/frameworks/googletest.html

Does it work for you?

HareshPrajapati · 2022-05-14T04:28:55Z

I tried it !
I think After upgradation of PlatformIO Core 6.0.0rc2 I get more troubles !!!
This time I removed all unit test examples and try to build google test with your suggestions but still it hangs on google test

ivankravets · 2022-05-14T07:29:15Z

Have you completed 4) from the list above? Please read again the Get Started section. There is an example of how to implement the main or setup/loop entry points.

Does it work now?

HareshPrajapati · 2022-05-14T09:10:27Z

It Works !!!! @ivankravets

I just changed my test file names to test_main.cpp and Set port in platformIO.ini file.
You need to set all files names as a test_main.cpp in folders where you want your test code
you still need to set test_port in platformIO.ini file.

I suggest to try this example for getting started.
I mean if you have a two code for test than your structure should look like this :

above steps are needed after latest platformIO update (6.0.0)
my old configuration of platformIO runs with any type of file names in test folder

One more thing ..

Don't forget to put below functions in unity test codes otherwise you will get error.

void setUp(void) {
    // set stuff up here
}

void tearDown(void) {
    // clean stuff up here
}

ivankravets · 2022-05-14T09:20:41Z

@HareshPrajapati

You don't need to specify the test port manually. PlatformIO detects it automatically. Could you share the output of pio device list command?
You don't need to declare setUp/tearDown. They are intended for the Unity testing framework.

HareshPrajapati · 2022-05-14T09:40:00Z

@ivankravets

this is my output for pio device list :

F:\1_Hello_World\UnitTest>pio device list
COM4
----
Hardware ID: USB VID:PID=10C4:EA60 SER=0001 LOCATION=1-1.2
Description: Silicon Labs CP210x USB to UART Bridge (COM4)

COM1
----
Hardware ID: ACPI\PNP0501\1
Description: Communications Port (COM1)

COM5
----
Hardware ID: PCI\VEN_8086&DEV_1E3D&SUBSYS_339A103C&REV_04\3&11583659&0&B3
Description: Intel(R) Active Management Technology - SOL (COM5)

You are right ,but some how it throwing errors while I was building unity tests.

ivankravets · 2022-05-14T09:52:15Z

I think we can improve 1). Could file an enhancement request at https://github.com/platformio/platformio-core/issues ?

HareshPrajapati · 2022-05-14T10:37:54Z

@ivankravets
Created 4263

ivankravets · 2022-05-14T20:31:56Z

Hi guys,

Could I ask you to re-test the latest pio upgrade --dev? It has significant improvements regarding automatic serial port detection of test_port and monito_port.

HareshPrajapati · 2022-05-16T03:58:30Z

@ivankravets

Works perfectly 💯 🥇

kapanmc · 2024-06-28T14:29:08Z

@ivankravets may I also ask for your kind help for similar issue

I'm building a project on ESP32-S3 and lately we decided to add tests into the codes and ran into the exact same problem as @maxgerhardt
My terminal was stuck at
If you don't see any output for the first 10 secs, please reset board (press reset button)
line forever.

Then after reading this thread I added
test_port = /dev/cu.usbserial-2130
into my .ini file and it's not stuck anymore yet it doesn't run the tests either!

What would be the problem and how may I get it solved if anyone can help?

Below is my platformio.ini lines;

[platformio]
src_dir = src
boards_dir = .

[env:esp32-s3-devkitc-1-myboard]
platform = espressif32
board = esp32-s3-devkitc-1-myboard
framework = arduino
monitor_speed = 115200
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32#2.0.3
build_flags = 
	-D LV_LVGL_H_INCLUDE_SIMPLE
	-I./include
lib_deps = 
	lovyan03/LovyanGFX@^1.1.12
	tamctec/TAMC_GT911@^1.0.2
	lvgl/[email protected]
	fbiego/ESP32Time@^2.0.6
	throwtheswitch/Unity@^2.5.2
board_build.partitions = huge_app.csv

test_build_src = true  
test_port = /dev/cu.usbserial-2130

and here is my test_main.cpp codes:

#include <Arduino.h>
#include <unity.h>
#include "events_init.h"

//conflicts with "setup" and "loop" in main, therefore I modified below 2 as 
void setup_test(); 
void loop_test(); 

void test_parlaklik_initial_value(void) {
    TEST_ASSERT_EQUAL(4, parlaklik_seviye); //testing a static variable from my project to see how testing works with my own parameters.
}

int add(int a, int b) {
    return a + b;
}

void test_add() { //testing some general stuff to learn test framework in general
    TEST_ASSERT_EQUAL(5, add(2, 3));
    TEST_ASSERT_EQUAL(0, add(-2, 2));
    TEST_ASSERT_EQUAL(-5, add(-3, -2));
}

void setup_test() {
    // Note: delay is needed to allow the serial monitor to initialize
    delay(5000);
    UNITY_BEGIN();
    RUN_TEST(test_parlaklik_initial_value);
    RUN_TEST(test_add);
    UNITY_END();
}

void loop_test() {
    // Leave empty
}

finally here is my terminal output:

➜  touchscreen git:(main) ✗ pio test -vv
Collected 1 tests (*)

Processing * in esp32-s3-devkitc-1-myboard environment
---------------------------------------------------------------------------------------------------------------------------------------------

Building & Uploading...
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32-s3-devkitc-1-myboard.html
PLATFORM: Espressif 32 (6.7.0) > Espressif ESP32-S3-DevKitC-1-N8 -ELECROW
HARDWARE: ESP32S3 240MHz, 320KB RAM, 8MB Flash
DEBUG: Current (esp-builtin) On-board (esp-builti
.
.
.
.
.
.
Writing at 0x0023a3b5... (92 %)
Writing at 0x00241e30... (94 %)
Writing at 0x0024b19a... (97 %)
Writing at 0x00252688... (100 %)
Wrote 2374640 bytes (609456 compressed) at 0x00010000 in 20.0 seconds (effective 948.3 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

[  1663][E][Wire.cpp:159] begin(): Bus already started in Master Mode.
[  1663][E][esp32-hal-gpio.c:95] __pinMode(): Invalid pin selected
[  1664][E][esp32-hal-gpio.c:95] __pinMode(): Invalid pin selected
E (1252) gpio: gpio_set_level(226): GPIO output gpio_num error
E (1257) gpio: gpio_set_level(226): GPIO output gpio_num error
E (1273) gpio: gpio_set_level(226): GPIO output gpio_num error
E (1274) gpio: gpio_set_level(226): GPIO output gpio_num error
E (1279) gpio: gpio_set_level(226): GPIO output gpio_num error
[  1747][E][esp32-hal-gpio.c:95] __pinMode(): Invalid pin selected

maxgerhardt · 2024-06-28T14:31:58Z

//conflicts with "setup" and "loop" in main, therefore I modified below 2 as 
void setup_test(); 
void loop_test();

so how are the Unit tests supposed to be executed when it doesn't reach execution of that function? Who occupies the real setup() and loop() functions? The events_init.h lirary? If yes, fix that library.

In any case, your issue is with your code, not PlatformIO itself.

kapanmc · 2024-06-28T15:20:32Z

Thanks for very fast response @maxgerhardt
main.cpp occupies the real setup() and loop() functions.

Because it's my first time trying to implement a test environment, I may get confused easily.

From examples and tutorials I read, all of them has setup() and loop() inside test_main.cpp ; however if i proceed like that, compiler complains that duplicate declaration.

maxgerhardt · 2024-06-28T15:23:06Z

Don't set test_build_src = true when the code in there defines setup() and loop(). All to-be-tested components should be in lib/<your library name> folders, so that they can be accessed independently by either the regular firmware (src) or tests.

kapanmc · 2024-06-28T15:30:49Z

[UPDATE: I've added lib_deps = Wire and the dependency issue got resolved]
The reason why i set test_build_src = true is because compiler is terminated due to a dependency(attached is the SS of terminal output)

However, if I just compile and upload the code as it is, there's no problem with this dependency because Wire.h is already under .pio/libdeps/

maxgerhardt · 2024-06-28T15:35:51Z

If a subdependency can't be found, you might need to increase the lib_ldf_mode. In any case, this issue is getting filled with lots of off-topic content. The issue about test port recognition was long solved.

kapanmc · 2024-06-28T15:36:23Z

Yes, thank you very much for your kind and very quick answers @maxgerhardt

maxgerhardt closed this as completed Oct 5, 2021

maxgerhardt changed the title ~~Unit Testing not working on ESP32~~ Unit Testing not working on ESP32 (test port not automatically recognized) Apr 30, 2022

ivankravets added bug unit testing labels May 14, 2022

ivankravets added this to the 6.0 milestone May 14, 2022

ivankravets reopened this May 14, 2022

ivankravets mentioned this issue May 14, 2022

test port not automatically recognized in My Esp32 while using GoogleTest #4263

Closed

1 task

ivankravets self-assigned this May 14, 2022

ivankravets closed this as completed in e5e2210 May 14, 2022

ivankravets mentioned this issue May 16, 2022

platformio 6.0.0 Homebrew/homebrew-core#101681

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unit Testing not working on ESP32 (test port not automatically recognized) #4076

Unit Testing not working on ESP32 (test port not automatically recognized) #4076

maxgerhardt commented Oct 4, 2021 •

edited

Loading

valeros commented Oct 5, 2021

maxgerhardt commented Oct 5, 2021

ankostis commented Apr 30, 2022

maxgerhardt commented Apr 30, 2022

ankostis commented Apr 30, 2022

maxgerhardt commented Apr 30, 2022

ankostis commented Apr 30, 2022

maxgerhardt commented Apr 30, 2022 •

edited

Loading

HareshPrajapati commented May 13, 2022 •

edited

Loading

ivankravets commented May 13, 2022

HareshPrajapati commented May 14, 2022 •

edited

Loading

ivankravets commented May 14, 2022

HareshPrajapati commented May 14, 2022 •

edited

Loading

ivankravets commented May 14, 2022

HareshPrajapati commented May 14, 2022 •

edited

Loading

ivankravets commented May 14, 2022

HareshPrajapati commented May 14, 2022

ivankravets commented May 14, 2022

HareshPrajapati commented May 16, 2022

kapanmc commented Jun 28, 2024

maxgerhardt commented Jun 28, 2024

kapanmc commented Jun 28, 2024

maxgerhardt commented Jun 28, 2024 •

edited

Loading

kapanmc commented Jun 28, 2024 •

edited

Loading

maxgerhardt commented Jun 28, 2024

kapanmc commented Jun 28, 2024

Unit Testing not working on ESP32 (test port not automatically recognized) #4076

Unit Testing not working on ESP32 (test port not automatically recognized) #4076

Comments

maxgerhardt commented Oct 4, 2021 • edited Loading

Configuration

Description of problem

Steps to Reproduce

Actual Results

Expected Results

If problems with PlatformIO Build System:

Additional info

valeros commented Oct 5, 2021

maxgerhardt commented Oct 5, 2021

ankostis commented Apr 30, 2022

maxgerhardt commented Apr 30, 2022

ankostis commented Apr 30, 2022

maxgerhardt commented Apr 30, 2022

ankostis commented Apr 30, 2022

maxgerhardt commented Apr 30, 2022 • edited Loading

HareshPrajapati commented May 13, 2022 • edited Loading

ivankravets commented May 13, 2022

HareshPrajapati commented May 14, 2022 • edited Loading

ivankravets commented May 14, 2022

HareshPrajapati commented May 14, 2022 • edited Loading

ivankravets commented May 14, 2022

HareshPrajapati commented May 14, 2022 • edited Loading

ivankravets commented May 14, 2022

HareshPrajapati commented May 14, 2022

ivankravets commented May 14, 2022

HareshPrajapati commented May 16, 2022

kapanmc commented Jun 28, 2024

maxgerhardt commented Jun 28, 2024

kapanmc commented Jun 28, 2024

maxgerhardt commented Jun 28, 2024 • edited Loading

kapanmc commented Jun 28, 2024 • edited Loading

maxgerhardt commented Jun 28, 2024

kapanmc commented Jun 28, 2024

maxgerhardt commented Oct 4, 2021 •

edited

Loading

maxgerhardt commented Apr 30, 2022 •

edited

Loading

HareshPrajapati commented May 13, 2022 •

edited

Loading

HareshPrajapati commented May 14, 2022 •

edited

Loading

HareshPrajapati commented May 14, 2022 •

edited

Loading

HareshPrajapati commented May 14, 2022 •

edited

Loading

maxgerhardt commented Jun 28, 2024 •

edited

Loading

kapanmc commented Jun 28, 2024 •

edited

Loading