How is Toit's performance for regular, real apps?

[sergei-kashlikov]

Hello,

I found your project interesting in our applications. But I didn't find any performance benchmarks or memory footprint (RAM/ROM) for regular applications, and I'm asking about real apps, not LED blinking or "hello, world" examples. It's important to understand how big an algorithm can be inside your platform, i.e. on esp32 with 500KB RAM.

Thanks

[kasperl]

Thank you for this question!

Performance-wise, it very much depends on what you're trying to achieve. On object-oriented benchmarks, we see Toit outperforming similar high-level languages on the ESP32. Recent measurements indicate more than a 20x performance improvement over Python on such devices. If you have any relevant benchmarks, I'd be happy take a look at them. I will also share our benchmarks, so you can take a look yourself.

You're right that the Toit language implementation comes with footprint overhead on two fronts:

1. Static size of the Toit virtual machine (ROM).
2. Dynamic overhead for running programs (RAM).

We use a number of supporting libraries, such as Mbed TLS, Apache Nimble, and lwIP. Isolated from those, the Toit virtual machine takes up around 165kb of ROM in the .text segment and approximately 2kb of RAM in the .data and .bss segments.

Additionally, the Toit virtual machine allocates a small heap for you application. It starts at 4kb and grows on demand. We also start a small number of FreeRTOS tasks when booting and we reserve stack space for those. All in all, it is likely to add up to ~30kb of RAM overhead.

Garbage collections

When running Toit code on an ESP32, we will sometimes log memory related information as serial output. Here is such output from running the http example, which shows that a garbage collection took less than one millisecond and the amount of allocated memory went from 16kb to 8kb in the process:

[gc @ 0x3ffcf980* | heap: 12kb -> 8kb | external: 4kb -> 0kb | free: 10kb->16kb 0.750ms]

Let me know if there are specifics you'd like to dive into and I'd be happy to provide more information.

[sergei-kashlikov]

Thanks,

Is it possible to provide some benchmark for simple algorithm, like factorial for big values (or other good small benchmarks)? It will be very helpful to understand the overall capabilities of the system. For the hobby projects may be its not necessary, but for more tech solutions its important to understand the platform. For example, I would like to see a table with the time (microseconds or instructions amount), rom and maximum heap allocated size, and comparison with raw C (this is important base value), then MicroPython and may be some other platform.

[kasperl]

Started adding benchmarks here: https://github.com/toitlang/toit/tree/master/third_party/benchmarks.