v5.3 idf.py Size Command "-DOUTPUT_FORMAT" Not Working (IDFGH-13484)

[bulutbekdemir]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

I'm using IDF v5.3.0 release on Linux <username> 6.5.0-45-generic #45~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Jul 15 16:40:02 UTC 2 x86_64 x86_64 x86_64 GNU/Linux via VS Code extension. When I tried to use idf.py size-files -DOUTPUT_FORMAT=JSON or idf.py size-files -DOUTPUT_FORMAT=csv or any other commands size-components or size with all possible combinations via same ways, I just mentioned here but getting same error, every time. I dropped a sample output here:

<Successfully builds>

CMake Warning:
  Manually-specified variables were not used by the project:

    OUTPUT_FORMAT


-- Build files have been written to: /home/<name>/esp/Workspaces/<path>/build
Running ninja in directory /home/<name>/esp/Workspaces/<path>/build
Executing "ninja all"...
[1/4] cd /home/<name>/esp/Workspaces/<path>/build/esp-idf/esptool...in /home/<name>/esp/Workspaces/<path>/build/<project_name>.bin
<project_name>.bin binary size 0xcabf0 bytes. Smallest app partition is 0x177000 bytes. 0xac410 bytes (46%) free.
[1/1] cd /home/<name>/esp/Workspaces/<path>/build/bootloader/esp-...home/<name>/esp/Workspaces/<path>/build/bootloader/bootloader.bin
Bootloader binary size 0x5180 bytes. 0x2e80 bytes (36%) free.
Running ninja in directory /home/<name>/esp/Workspaces/<path>/build
Executing "ninja size-components"...
[0/1] cd /home/<name>/esp/Workspaces/<path>/build && /usr/bin/cmake -D "IDF_SIZE_TOOL=/home/<name>/.espressif/python_env/idf5.3_py3.10_env/bin/python;-m;esp_idf_size" -D IDF_SIZE_MODE=--archives -D MAP_FILE=/home/<name>/esp/Workspaces/<path>/build/<project_name>.map -D OUTPUT_JSON= -P /home/<name>/esp/v5.3/esp-idf/tools/cmake/run_size_tool.cmake

[fhrbata]

Hello @bulutbekdemir ,

maybe $ idf.py size --help can help. The help is available also for size-components and size-files.

$ idf.py size --format csv

Where is the -DOUTPUT_FORMAT=csv or -DOUTPUT_FORMAT=JSON usage coming from? I believe this is not supported nor documented. Thank you

[bulutbekdemir]

Hello @bulutbekdemir ,

maybe $ idf.py size --help can help. The help is available also for size-components and size-files.

$ idf.py size --format csv

Where is the -DOUTPUT_FORMAT=csv or -DOUTPUT_FORMAT=JSON usage coming from? I believe this is not supported nor documented. Thank you

Hello, the command is documented here: Measuring Static Sizes and I just paste here:

Under the Note section

It is possible to add -DOUTPUT_FORMAT=csv or -DOUTPUT_FORMAT=json to get the output in CSV or JSON format.

[bulutbekdemir]

Hello @fhrbata,
The document actually mentions two possible ways. However, one of them, which is also the easiest to spot, isn't working.

[cristianfunes79]

I cannot checkout to v5.3.0 (seems it doesn't exist) so I assume you mean v5.3. Running the same command on my machine Linux ubuntu-ThinkPad-T14s-Gen-1 6.8.0-40-generic #40~22.04.3-Ubuntu SMP PREEMPT_DYNAMIC Tue Jul 30 17:30:19 UTC 2 x86_64 x86_64 x86_64 GNU/Linux works fine. Just in case this is my cmake version

cmake version 3.26.4

CMake suite maintained and supported by Kitware (kitware.com/cmake).

idf version

commit e0991facf5ecb362af6aac1fae972139eb38d2e4 (HEAD, tag: v5.3)

[bulutbekdemir]

Hello @cristianfunes79
Sorry for my error I'm using v5.3. And my Cmake version is here:

 cmake version 3.22.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).

also when I tried to update my cmake version it gives out:

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
cmake is already the newest version (3.22.1-1ubuntu1.22.04.2).
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

and my idf version is same as you, here the HEAD commit:
e0991facf5ecb362af6aac1fae972139eb38d2e4

Do you really tried
idf.py size-files -DOUTPUT_FORMAT=csv command or idf.py size --format csv command because one of them not working on my machine but both of them documented. I'm just curious about if it's working for you why not on my machine?

[cristianfunes79]

Yep, it is still working for me. May I ask you for more details about what are you building? which project? which target? Did you run all idf installation steps (install, export and so on)?

[bulutbekdemir]

I am developing a custom component for a company's project, which needs to remain closed source. The application relies on multiple libraries, and I'm able to view all details related to binary files when I run specific commands without CMake flags and with or without --format specifier. However, when I execute these same commands later, I get no output as mentioned previously.

Initially, I installed ESP-IDF version 5.2.2 and then upgraded to version 5.3 using the default installer through the 'Configure ESP-IDF Extension' settings. I also reinstalled all other components without uninstalling version 5.2.2, as I still need this version for compatibility reasons.

The project is being built for a custom ESP32-C3 device and ESP32-C3-DevKitM-1 through UART, as well as other devices like the ESP32-C6. Despite the issues with command output, the application builds successfully and runs on the target devices.

[fhrbata]

Hello @bulutbekdemir ,

Hello, the command is documented here: Measuring Static Sizes and I just paste here:

Under the Note section

It is possible to add -DOUTPUT_FORMAT=csv or -DOUTPUT_FORMAT=json to get the output in CSV or JSON format.

thank you, you are absolutely right. Information about -DOUTPUT_FORMAT was added to the size.srt and idf-py.rst documentation by 5ee663d and later removed from idf-py.rst by 09e61e8, but it remains in size.srt. I believe setting OUTPUT_FORMAT via the CMake cache has never worked and appears to be a leftover (typo in the docs) from when -DOUTPUT_JSON=1 CMake variable was used to set the JSON output format before the --format option was introduced. My assumption is that the documentation should state that the output format can be changed by setting the environment variable SIZE_OUTPUT_FORMAT=, but this will only work when the size target is called directly, such as SIZE_OUTPUT_FORMAT=json ninja size, and not via idf.py. It seems that previously it was possible to change the output by setting SIZE_OUTPUT_FORMAT even when using idf.py size commands, but this no longer appears to be possible since bfc17ce.

I believe we should simply delete the note that mentions OUTPUT_FORMAT.

Thank you once more for bringing this to our attention.

[fhrbata]

Hello @cristianfunes79 ,

idf.py size-files -DOUTPUT_FORMAT=csv

Are you actually seeing the output in CSV format or just the default table? The command works, but it does not display the output in the requested format specified by -DOUTPUT_FORMAT.

Thank you

[cristianfunes79]

I can actually see this

[0/1] cd /home/ubuntu/esp/esp-idf/examples/cxx/pthread/build && /home/linuxbrew/.linuxbrew/Cellar/cmake/3.26.4/bin/cmake -D "IDF_SIZE_TOOL=/home/ubuntu/.espressif/python_env/idf5.3_py3.11_env/bin/python;-m;esp_idf_size" -D MAP_FILE=/home/ubuntu/esp/esp-idf/examples/cxx/pthread/build/cpp_pthread.map -D OUTPUT_JSON= -P /home/ubuntu/esp/esp-idf/tools/cmake/run_size_tool.cmake
"Memory Type/Section","Used [bytes]","Used [%]","Remain [bytes]","Total [bytes]"
"Flash Code","280646","3.35","8107930","8388576"
".text","280646","3.35","",""
"DRAM","61304","19.08","259992","321296"
".text","47250","14.71","",""
".bss","8480","2.64","",""
".data","5208","1.62","",""
"Flash Data","59048","0.7","8329528","8388576"
".rodata","58792","0.7","",""
".appdesc","256","0.0","",""
"RTC SLOW","536","6.54","7656","8192"
".rtc_reserved","24","0.29","","

[fhrbata]

Hello @cristianfunes79 ,

I can actually see this

[0/1] cd /home/ubuntu/esp/esp-idf/examples/cxx/pthread/build && /home/linuxbrew/.linuxbrew/Cellar/cmake/3.26.4/bin/cmake -D "IDF_SIZE_TOOL=/home/ubuntu/.espressif/python_env/idf5.3_py3.11_env/bin/python;-m;esp_idf_size" -D MAP_FILE=/home/ubuntu/esp/esp-idf/examples/cxx/pthread/build/cpp_pthread.map -D OUTPUT_JSON= -P /home/ubuntu/esp/esp-idf/tools/cmake/run_size_tool.cmake
"Memory Type/Section","Used [bytes]","Used [%]","Remain [bytes]","Total [bytes]"
"Flash Code","280646","3.35","8107930","8388576"
".text","280646","3.35","",""
"DRAM","61304","19.08","259992","321296"
".text","47250","14.71","",""
".bss","8480","2.64","",""
".data","5208","1.62","",""
"Flash Data","59048","0.7","8329528","8388576"
".rodata","58792","0.7","",""
".appdesc","256","0.0","",""
"RTC SLOW","536","6.54","7656","8192"
".rtc_reserved","24","0.29","","

Thank you. I might be missing something, but I don't understand how using -DOUTPUT_FORMAT=csv would produce a CSV format. Could you please share the command you used to get the CSV output?

[Indastri]

Closing for inactivity, If the problem persists feel free to reopen or create a new issue.

Cheers!