Bugfix/debugging docs

[brianignacio5]

Description

Update documentation regarding debugging using other extensions and now using Eclipse CDT GDB

Fixes #918
Fixes #1164
Fixes #1138
Fixes #1168
Fixes #1177

Type of change

• This change requires a documentation update
• New feature

Steps to test this pull request

1. Create a new ESP-IDF Project
2. Build flash the project
3. Use the CDT debug configuration to start debugging.

 {
   "configurations": [
     {
       "type": "gdbtarget",
       "request": "attach",
       "name": "Eclipse CDT Remote",
     }
   ]
 }

4. Perform debug tasks. Read the updated debugging tutorial for example.

• Expected behaviour:

New debugging should work faster and better than previous debug adapter.

• Expected output:

Debugging experience should be the same as debugging tutorial.

How has this been tested?

Test running the debugging tutorial.

Test Configuration:

• ESP-IDF Version: 5.0
• OS (Windows,Linux and macOS): macOS

Dependent components impacted by this PR:

• ESP-Debug Adapter

Checklist

• PR Self Reviewed
• Applied Code formatting
• Added Documentation
• Added Unit Test
• Verified on all platforms - Windows,Linux and macOS

[github-actions]

Download the artifacts for this pull request:

• esp-idf-extension.vsix.zip

[TheHexaCube]

Hi there!
I wanted to try your debugger update on windows, but sadly I am encountering the following error message in the ESP-IDF console whilst the debugger is running, whenever it's trying to fetch the peripheral data:
(Trying to fetch data from SPI2 on an ESP32C3 in this instance)

[Debug Adapter]
2024-03-31 17:09:56,523 - Debug Adapter (ReaderThread) - DEBUG - read line: >>b'Content-Length: 107\\r\\n'<<

2024-03-31 17:09:56,523 - Debug Adapter (ReaderThread) - DEBUG - read line: >>b'\\r\\n'<<


[Debug Adapter]
2024-03-31 17:09:56,523 - Debug Adapter(Command Processor) - DEBUG - Got json: {
    "arguments": {
        "count": 244,
        "memoryReference": "0x60024000"
    },
    "command": "readMemory",
    "seq": 44,
    "type": "request"
}
2024-03-31 17:09:56,523 - Debug Adapter (WriterThread) - DEBUG - Writing: {"type": "event", "event": "output", "body": {"output": "Got json: {\n    \"arguments\": {\n        \"count\": 244,\n        \"memoryReference\": \"0x60024000\"\n    },\n    \"command\": \"readMemory\",\n    \"seq\": 44,\n    \"type\": \"request\"\n}\n", "source": {}}, "seq": 172}

2024-03-31 17:09:56,523 - Debug Adapter (main) - ERROR - %d format: a real number is required, not NoneType
Traceback (most recent call last):
  File "c:\Users\danik\.vscode\extensions\espressif.esp-idf-extension-1.7.1\esp_debug_adapter\debug_adapter\command_processor.py", line 473, in on_readMemory_request
    memory_bytes = self.da.read_memory(request.arguments.memoryReference, 4000, request.arguments.offset)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\danik\.vscode\extensions\espressif.esp-idf-extension-1.7.1\esp_debug_adapter\debug_adapter\debug_adapter.py", line 878, in read_memory
    memory_result = self._gdb.read_memory_bytes(addr, count, offset)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\danik\.espressif\python_env\idf5.2_py3.11_env\Lib\site-packages\esp_debug_backend\gdb.py", line 471, in read_memory_bytes
    res, res_body = self._mi_cmd_run('-data-read-memory-bytes -o %d %s %s' % (off, addr, count))
                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
TypeError: %d format: a real number is required, not NoneType
2024-03-31 17:09:56,523 - Debug Adapter (WriterThread) - DEBUG - Writing: {"type": "event", "event": "output", "body": {"output": "%d format: a real number is required, not NoneType\n", "source": {}}, "seq": 173}

2024-03-31 17:09:56,524 - Debug Adapter (main) - ERROR - 'TypeError' object is not iterable
Traceback (most recent call last):
  File "c:\Users\danik\.vscode\extensions\espressif.esp-idf-extension-1.7.1\esp_debug_adapter\debug_adapter\command_processor.py", line 473, in on_readMemory_request
    memory_bytes = self.da.read_memory(request.arguments.memoryReference, 4000, request.arguments.offset)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\danik\.vscode\extensions\espressif.esp-idf-extension-1.7.1\esp_debug_adapter\debug_adapter\debug_adapter.py", line 878, in read_memory
    memory_result = self._gdb.read_memory_bytes(addr, count, offset)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\danik\.espressif\python_env\idf5.2_py3.11_env\Lib\site-packages\esp_debug_backend\gdb.py", line 471, in read_memory_bytes
    res, res_body = self._mi_cmd_run('-data-read-memory-bytes -o %d %s %s' % (off, addr, count))
                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
TypeError: %d format: a real number is required, not NoneType

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\danik\.vscode\extensions\espressif.esp-idf-extension-1.7.1\esp_debug_adapter\debug_adapter\command_processor.py", line 52, in __call__
    on_request(protocol_message)
  File "c:\Users\danik\.vscode\extensions\espressif.esp-idf-extension-1.7.1\esp_debug_adapter\debug_adapter\command_processor.py", line 479, in on_readMemory_request
    kwargs = {'body': schema.ErrorResponseBody(error=dict(e))}
                                                     ^^^^^^^
TypeError: 'TypeError' object is not iterable
2024-03-31 17:09:56,524 - Debug Adapter (WriterThread) - DEBUG - Writing: {"type": "event", "event": "output", "body": {"output": "%d format: a real number is required, not NoneType\n", "source": {}}, "seq": 174}

2024-03-31 17:09:56,524 - Debug Adapter (WriterThread) - DEBUG - Writing: {"type": "event", "event": "output", "body": {"output": "'TypeError' object is not iterable\n", "source": {}}, "seq": 175}

2024-03-31 17:09:56,524 - Debug Adapter (WriterThread) - DEBUG - Writing: {"type": "event", "event": "output", "body": {"output": "'TypeError' object is not iterable\n", "source": {}}, "seq": 176} 

Any suggestion on what I might be doing wrong, or perhaps a bug that occurs?

Regards,
Daniel

[brianignacio5]

@TheHexaCube Hi Daniel thank you for your feedback. You are still using the old debug adapter. Please update your .vscode/launch.json with this new configuration and try again:

 {
   "configurations": [
     {
       "type": "gdbtarget",
       "request": "attach",
       "name": "Eclipse CDT Remote",
       "program": "${workspaceFolder}/build/${command:espIdf.getProjectName}.elf",
       "initCommands": [
         "set remote hardware-watchpoint-limit 2",
         "mon reset halt",
         "maintenance flush register-cache",
         "thb app_main",
       ],
       "gdb": "${command:espIdf.getXtensaGdb}",
       "target": {
         "port": "3333"
       },
     }
   ]
 }

[radurentea]

There are some commented lines of code which I've notice.
Other than that LGTM! Tested it on Linux (Fedora 39)

[TheHexaCube]

@TheHexaCube Hi Daniel thank you for your feedback. You are still using the old debug adapter. Please update your .vscode/launch.json with this new configuration and try again:

 {
   "configurations": [
     {
       "type": "gdbtarget",
       "request": "attach",
       "name": "Eclipse CDT Remote",
       "program": "${workspaceFolder}/build/${command:espIdf.getProjectName}.elf",
       "initCommands": [
         "set remote hardware-watchpoint-limit 2",
         "mon reset halt",
         "maintenance flush register-cache",
         "thb app_main",
       ],
       "gdb": "${command:espIdf.getXtensaGdb}",
       "target": {
         "port": "3333"
       },
     }
   ]
 }

Hi Brian,
sorry, I somehow totally missed that - I apologize!

The debugger seems to work fine now, however I noticed that the peripheral values don't update "automatically" whenever the debugger stops at a breakpoint (or is manually stopped), instead I have to "close and open" the registers for them to update.
Is that intended behaviour, or a configuration issue on my end?

Thanks for your help, I really appreciate it and the update itself!

Best regards

[brianignacio5]

@TheHexaCube Hi Daniel thank you for your feedback. You are still using the old debug adapter. Please update your .vscode/launch.json with this new configuration and try again:

 {
   "configurations": [
     {
       "type": "gdbtarget",
       "request": "attach",
       "name": "Eclipse CDT Remote",
       "program": "${workspaceFolder}/build/${command:espIdf.getProjectName}.elf",
       "initCommands": [
         "set remote hardware-watchpoint-limit 2",
         "mon reset halt",
         "maintenance flush register-cache",
         "thb app_main",
       ],
       "gdb": "${command:espIdf.getXtensaGdb}",
       "target": {
         "port": "3333"
       },
     }
   ]
 }

Hi Brian, sorry, I somehow totally missed that - I apologize!

The debugger seems to work fine now, however I noticed that the peripheral values don't update "automatically" whenever the debugger stops at a breakpoint (or is manually stopped), instead I have to "close and open" the registers for them to update. Is that intended behaviour, or a configuration issue on my end?

Thanks for your help, I really appreciate it and the update itself!

Best regards

Thanks for the feedback. The peripheral tree view values open/close trigger a read memory request to GDB, I think having too many read request on each breakpoint stop will provide a slower debugger experience and since it is just a memory address we don't know which specific peripheral value has changed for a given step. I'm not sure if it would be a good approach to read all on each stop event or allow the user to enable such as a feature. What do you think @TheHexaCube ?

[TheHexaCube]

Thanks for the feedback. The peripheral tree view values open/close trigger a read memory request to GDB, I think having too many read request on each breakpoint stop will provide a slower debugger experience and since it is just a memory address we don't know which specific peripheral value has changed for a given step. I'm not sure if it would be a good approach to read all on each stop event or allow the user to enable such as a feature. What do you think @TheHexaCube ?

Hey Brian!
I see where you're coming from, reading all values probably indeed isn't very performant.
Would it be feasible to check which registers in the peripheral view are "open" (as in, the exact fields are visible) and only update those on a breakpoint? Or a functionality akin to "add to watch" for variables, but for registers?

Alternatively I think at least allowing a user to configure a "update all on breakpoint" would be a fair decision, especially if the performance impact would be mentioned in the docs, and also would be turned off by default.
Something similar to a "you're on your own if you enable this option"

I personally feel like having to open/close the view on every breakpoint, when trying to debug a peripheral, is sort of cumbersome aswell, so at least an option would be nice in my eyes

[radurentea]

LGTM except for the console.log I've commented about

[brianignacio5]

Thanks for the feedback. The peripheral tree view values open/close trigger a read memory request to GDB, I think having too many read request on each breakpoint stop will provide a slower debugger experience and since it is just a memory address we don't know which specific peripheral value has changed for a given step. I'm not sure if it would be a good approach to read all on each stop event or allow the user to enable such as a feature. What do you think @TheHexaCube ?

Hey Brian! I see where you're coming from, reading all values probably indeed isn't very performant. Would it be feasible to check which registers in the peripheral view are "open" (as in, the exact fields are visible) and only update those on a breakpoint? Or a functionality akin to "add to watch" for variables, but for registers?

Alternatively I think at least allowing a user to configure a "update all on breakpoint" would be a fair decision, especially if the performance impact would be mentioned in the docs, and also would be turned off by default. Something similar to a "you're on your own if you enable this option"

I personally feel like having to open/close the view on every breakpoint, when trying to debug a peripheral, is sort of cumbersome aswell, so at least an option would be nice in my eyes

Actually I tested again with updating values on debug stop event (which I added to this PR) and doesn't seems to trigger many readMemory requests. If you don't mind please try the updated VSIX installer @TheHexaCube

[mhaberler]

I've tried a multi_config example with this vsix version following the steps outlined here using development, prod1 and prod2 - where the first two configs target a esp32c3, the latter an esp32s3

selecting any config, build and flash work so far as outlined in multi_config - I uploaded a snapshot .

my open questions:

• how do I go about debugging setup for these configs? it seems a single debug config is picked up from .vscode/settings.json and stuff in esp_idf_project_configuration.json is ignored for debugging?
• is there an assumption all project configs are for the same target idfTarget regarding debugging?
• how would I add profiles like @profiles/debug and @profiles/release like with a command line idf.py build as described here?
• what is the relation of these settings versus this per-config setting?

thanks in advance,

Michael

[mhaberler]

I kind of got this working with dual targets by using explicitly customized debug launch configs - not sure this is the way how it is supposed to be done

switching between targets of different archs requires vscode restart, seems some openocd config values are cached and not reread?

maybe we can mutate this example into a reference for how it should be done - learning curve and amount of custom incantations is still staggering

the new debug adapter is much better, and the Native Debug extension is a working alternative as reference

opening a terminal before debugging sends things haywire once debugging starts, but that looks more like the fickle USB hardware - the only variant working is start debugging, start terminal without touching DTR

I am still at loss regarding question 3 and 4

[brianignacio5]

what is the relation of these settings versus this per-config setting?

So the project configuration settings override some of the settings from extension settings as described in the project configuration. This allow to have multiple build flash configuration as the multi_config ESP-IDF example. It doesn't override IDF_PATH and tools settings because it is intended to generate multiple builds for same ESP-IDF version.

how would I add profiles like @profiles/debug and @profiles/release like with a command line idf.py build as described here?

The project configuration editor modifies some of the existing parameters like build arguments. If you take a look at these profiles they are basically build arguments -B build-production -DSDKCONFIG=build-production/sdkconfig -DSDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.prod" which for the extension can be achieve by modifying the build directory (build-production ), sdkconfig defaults (-DSDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.prod"`) and sdkconfig (-DSDKCONFIG=build-production/sdkconfig) settings.

About how to debug with these multiple project configuration:

I think the missing part is to update "program": "${workspaceFolder}/build/${command:espIdf.getProjectName}.elf", or just remove this line and let the extension resolve it automatically (it will locate the elf based on current profile build directory setting). I think probably the IDF_TARGET is a bit confusing. If you have a profile for esp32c3, you should have a separate build directory and sdkconfig file for each profile. When you switch between one profile and another just make sure that is using the right build elf file for the debug session, same as openOCD and GDB. Since only setting that is modified is the IDF_TARGET for each profile, the GDB should be picked correctly. I haven't tested this scenario of switching between esp32c3 and esp32s3 but openOCD args (idf.openOCDConfigFiles) should be updated and as long as previous openOCD session closed correctly it should pick new openOCD config per profile.

Let me know how it goes.

[AndriiFilippov]

@brianignacio5 hi !

Tested under:
OS - Windows 10
ESP-IDF: v5.2

Followed debug doc from https://github.com/espressif/vscode-esp-idf-extension/blob/bugfix/debugging-docs/docs/tutorial/debugging.md

Was able to run debug and repeat all steps from docs.

Only thing I think we should add to make it easier for user to navigate is just to modify illustrations a bit like this

What do you think ?

Also it would be nice if you could fix this comma typos. Caz users probably gonna copy all the setting from the our docs and they hope that this will work :)

[erhankur]

@brianignacio5 Thanks for working on the new debug adapter. Lastly, I had time to play with existing debug adapters. For me, the most crucial part is to see what is going on in the background. When users come up with an issue, we ask OpenOCD for verbose logs in a file. So IMO, it should be easy to have it, and needs to be mentioned somewhere in the debugging.md

Maybe it is related to the espidf debug adapter but I couldn't manage to get it to work when I increased the adapter log level. It gets too slow and the debug session never starts.

I wanted to share my experience so I left some comments. If I can find a time, I would also like to test and see how it works.

[mikebaylis]

Hi,

I just found this change while struggling as usual with the awful debugger, and on checking the docs noticed the new recommendation to use Eclipse CDT GDB Adapter. So I switched debug adapter from ESP-IDF Debug Adapter to Eclipse CDT GDB Adapter.

This is a HUGE improvement and can't be understated !! Thank you !!