Add support for GoogleTest unit testing framework

[arduhe]

I am trying to use gtest instead of unity as test framework.

Motivation - I’d like to use the same framework in all environments (embedded and native) and not two different 😉 .

In a native environment pio test works good and as expected, whereas in a embedded one (e.g. espressif8266/d1_mini) pio test neither makes highlighting of test results nor finishes - I have to abort manually each run 😮 .

I’ve dug a little bit inside pio and think I’ve located the place where this behaviour could be changed. In platformio/commands/test/embedded.py the output from the tests is scanned to make highlighting and to detect the end of the tests. Currently it supports only the strings used by unity.

Could this please be altered to support gtest in embedded environments, too?

Thx

[ivankravets]

Unity is very good testing framework. It was created to be used in embedded projects. It's light-wight. Also, it works well on native platforms as well. Could you provide a simple project to reproduce your issues?

[arduhe]

Hello Ivan,
thank you for your quick reponse. To clear up a possible misunderstanding - I don't think that unity is a bad solution, but I've a little bit more experience with gtest/gmock. That's why I prefer them ;-)

I have created a small reproducer. You find it at https://github.com/arduhe/reproducer_3572.git.

Currently when you start the tests, this happens

pio test -e esp32
Verbose mode can be enabled via `-v, --verbose` option
Collected 1 items

Processing * in esp32 environment
------------------------------------------------------------------------------------------------------------------------------------------------------------
Building...
Flash: [====      ]  39.5% (used 517445 bytes from 1310720 bytes)
Uploading...
Flash: [====      ]  39.5% (used 517445 bytes from 1310720 bytes)
Connecting.....
Writing at 0x00001000... (100 %)
Wrote 15872 bytes (10319 compressed) at 0x00001000 in 0.2 seconds (effective 535.8 kbit/s)...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (128 compressed) at 0x00008000 in 0.0 seconds (effective 4913.9 kbit/s)...
Writing at 0x0000e000... (100 %)
Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.0 seconds (effective 22091.0 kbit/s)...
Writing at 0x00010000... (6 %)
Writing at 0x00020000... (33 %)
Writing at 0x00030000... (60 %)
Writing at 0x00040000... (86 %)
Wrote 517568 bytes (244574 compressed) at 0x00010000 in 5.7 seconds (effective 722.7 kbit/s)...
Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from test_suite
[ RUN      ] test_suite.alwaysOK
[       OK ] test_suite.alwaysOK (0 ms)
[ RUN      ] test_suite.failsIfShallFailIsDefined
[       OK ] test_suite.failsIfShallFailIsDefined (0 ms)
[----------] 2 tests from test_suite (2 ms total)
[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (4 ms total)
[  PASSED  ] 2 tests.
^C
Aborted!

After the test terminates, the process still "hangs" waiting for further output. I must kill it with ctrl-c. Additionally no information is shown, whether the complete test suite is successful or not.

What I would expect is something like that

pio test -e esp32
Verbose mode can be enabled via `-v, --verbose` option
Collected 1 items

Processing * in esp32 environment
------------------------------------------------------------------------------------------------------------------------------------------------------------
Building...
Flash: [====      ]  39.5% (used 517445 bytes from 1310720 bytes)
Uploading...
Flash: [====      ]  39.5% (used 517445 bytes from 1310720 bytes)
Connecting....
Writing at 0x00001000... (100 %)
Wrote 15872 bytes (10319 compressed) at 0x00001000 in 0.2 seconds (effective 537.3 kbit/s)...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (128 compressed) at 0x00008000 in 0.0 seconds (effective 4907.1 kbit/s)...
Writing at 0x0000e000... (100 %)
Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.0 seconds (effective 23235.7 kbit/s)...
Writing at 0x00010000... (6 %)
Writing at 0x00020000... (33 %)
Writing at 0x00030000... (60 %)
Writing at 0x00040000... (86 %)
Wrote 517568 bytes (244574 compressed) at 0x00010000 in 5.7 seconds (effective 725.5 kbit/s)...
Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from test_suite
[ RUN      ] test_suite.alwaysOK
[       OK ] test_suite.alwaysOK (0 ms)
[ RUN      ] test_suite.failsIfShallFailIsDefined
[       OK ] test_suite.failsIfShallFailIsDefined (0 ms)
[----------] 2 tests from test_suite (2 ms total)
[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (4 ms total)
[  PASSED  ] 2 tests.
=============================================================== [PASSED] Took 12.64 seconds ===============================================================

Test    Environment    Status    Duration
------  -------------  --------  ------------
*       esp32          PASSED    00:00:12.638
*       d1_mini        IGNORED
*       native         IGNORED
=============================================================== 1 succeeded in 00:00:12.638 ===============================================================

I have made a small patch for embedded.py which produces the desired results - as seen above ;-). As my knowledge in python is not very high, I am not sure whether it would be sufficient for a PR, but probably it could be used as a prototype for development. The patch is attached to this comment.

Kind regards

0001-patch-for-gtest-support-1st-working-prototype.patch.txt

[arduhe]

Sorry - made accidentally the git repo with the reproducer private. I've corrected that.

Kind regards

P.S.: Why didn't you complain that? 😉

[ivankravets]

Sorry, haven’t had a time to check it :( so busy with upcoming PlatformIO Core 4.4 release. We will back to this issue soon. Thanks.

[MikeMitterer]

@ivankravets Another argument for Google-Test: IDE Integration. There is no IDE-Integration (CLion) for Unity but a very good one for Google-Test. Programming without good support from your IDE is these days simply to complex and time consuming.

[ivankravets]

See updated docs for unit testing https://docs.platformio.org/en/latest/advanced/unit-testing/index.html

Please re-test with pio upgrade --dev. Does it work for you?