CPU commit 7b28699 didn't fix the read/write issues with ESP32-S3 (ESPTOOL-634)

[mrdude2478]

Operating System

Windows 10 64Bit

Esptool Version

Commit: 7b28699

Python Version

3.11

Full Esptool Command Line that Was Run

esptool -b 921600 --port COM3 --chip esp32s3 read_flash 0xCA2000 0x148000 FAT9.bin

Esptool Output

No issues with the output.

What is the Expected Behaviour?

Please revert commit:
7b28699

After further investigation I was able to fix this issue, please see here at the bottom of the page:
#832

There was an issue with reading/writing esp32-s3 chip with newer builds of esptool. It was discussed in issue 832. After further investigation and installing the sdk to build the stub files, I modded the files and found what was making the read/write fails. Please see the bottom of issue 832 (which is closed), for the fix. CPU speed can be put back to 240Mhz.

Thanks

More Information

No response

Other Steps to Reproduce

No response

[radimkarnis]

Hi @mrdude2478,
I am going to answer here, not in the linked discussion.

Thanks for looking into this more. Let me clarify first - lowering the CPU frequency to 160 MHz didn't help in your case? First, you said this fix helped.

Now, let's discuss your solution:
You propose to change the SYSTEM_SYSCLK_CONF_REG from SYSTEM_BASE_REG + 0x060 to just SYSTEM_BASE_REG. This helps in your case, but you also say am unsure what adding these addresses on to those clock registers actually does.

Looking into the TRM, it says that all the registers starting with SYSTEM are relative to the base address of system registers (SYSTEM_BASE_REG). That's why the address of SYSTEM_SYSCLK_CONF_REG is defined as the base reg SYSTEM_BASE_REG plus an offset of 0x060:

This address is where the CPU frequency switch is. What happens in your case is that you change the address so you are essentially switching another random thing, not the CPU freq settings. In other words, this is like reverting the feature completely, but with the added danger of messing something else up.

Therefore, this can't be considered a real fix, it's more of a lucky coincidence.

On the other hand, this feature proves to be more and more unpredictable and unstable. Removing it completely now seems like a viable solution to me. Stability is more important than flashing speeds. We will think about this more.

[mrdude2478]

OK thanks, I do know that since changing this value, I have not experienced a failure yet when reading or writing, so that will mean yes you are probably correct, removing this value seems like a viable fix. I guess the first time when the cpu frequency was changed I might have been lucky, however I still got crashes randomly, yesterday I flashed my dongle about 50 times also dumped the entire flash maybe about 20 times to test and didn't get a failure once.

[mrdude2478]

@radimkarnis

What about doing something like this - in stubflasher.c add these two lines.

#if USE_MAX_CPU_FREQ
static bool can_use_max_cpu_freq()
{
  /* Check if any of available USB modes are being used. */
  #if ESP32S3 //added this
  return false; //added this
  #elif WITH_USB_OTG && !WITH_USB_JTAG_SERIAL
  return stub_uses_usb_otg();
  #elif !WITH_USB_OTG && WITH_USB_JTAG_SERIAL
  return stub_uses_usb_jtag_serial();
  #elif WITH_USB_OTG && WITH_USB_JTAG_SERIAL
  return stub_uses_usb_otg() || stub_uses_usb_jtag_serial();
  #else
  return false;
  #endif
}

Then if the chip was detected as esp32-s3, it would disable the max cpu frequency. Would that work?

[radimkarnis]

@mrdude2478 it would work, but if this feature is unstable on the S3, we don't want to risk these issues also on other chips. So it will be reverted for all the chips, until we figure out why it doesn't work for everyone and how to implement it in a safe way.

[mrdude2478]

OK, thanks - you could then just return false from that function then and it would do the same thing for all chips, and would save needing to re-write lots of code until it's figured out why there's an timing issue? Maybe it's down to the xtal in my board? I don't know much about electronics but surely if that deals with the timing it could be something related to that? I ordered another board to check this, I might try replacing that crystal with a different brand to see if it makes any differnce.

[JunOllyLi]

What I have observed is that the official 4.5 and the latest code (4.6-dev) in the repo behave the same, that sometimes it works most of the time, while sometimes it just does not work at all. And mrdude2478's exe works while the 4.6-dev refuses to work.

I still have not figured out exactly what is the difference between those two scenarios. I am using the same computer (Surface 8 Pro, with nothing else connected, except the charger. I have tried with both charger connected and not connected, did not see any difference), same USB cable, same boards (yes I have different boards, some powered by on-board 5V=>3.V, some powered by external buck converter, and all behave the same). All I can tell is that it normally works while I sit in front of my desk, and not work while I sit in the bed, but I don't see how that would affect the result :(

[JunOllyLi]

Reading this thread more and here are my thoughts:
So mrdude2478 was writing something intended for SYSTEM_BASE_REG + 0x060 to just SYSTEM_BASE_REG and luckily it worked. Now:

1. What is the value of that?
2. What that value mean to the register at SYSTEM_BASE_REG?
3. Would it still work if we totally just skip the write (neither writing to SYSTEM_BASE_REG + 0x060, nor SYSTEM_BASE_REG)
4. What if both registers were written with the same values?

So there are three possibilities:
a. the value written to SYSTEM_BASE_REG luckily had some bits that affects the behavior and make it more stable
b. the value originally written to SYSTEM_BASE_REG + 0x060 made it fail to work, and we should just ignore it
c. combination of both a and b.

I think it would be a good idea to understand what situation we are in first.

[JunOllyLi]

@radimkarnis if you have a quick guide on how to change that part of code and compile the exe (or simple modify python code to do that), I would be happy to help to try out.

I just don't have time to read all the code to find where this particular change is.

[mrdude2478]

@JunOllyLi , @radimkarnis allready explained that it's down to the CPU frequency using the max speed of the chip (see the code above). This is in the stub that is uploaded to the chip before dumping/flashing routines kick in. It's being looked into by people far smarter than me, well at least with far more experience in chip programming and writing software. At least those people are aware of the issues and why there's an issue so I imagine that it will be fixed in the near future as has already been stated. We just need to wait now and be patient and thank the coders for the hard work they do and for looking into this issue.

[JunOllyLi]

@JunOllyLi , @radimkarnis allready explained that it's down to the CPU frequency using the max speed of the chip (see the code above). This is in the stub that is uploaded to the chip before dumping/flashing routines kick in. It's being looked into by people far smarter than me, well at least with far more experience in chip programming and writing software. At least those people are aware of the issues and why there's an issue so I imagine that it will be fixed in the near future as has already been stated. We just meed to wait now and be patient and thank the coders for the hard work they do and for looking into this issue.

But I thought the frequency change had already been in the lates of the repo? And it did not work? At least not working after doing the "pip install"

[radimkarnis]

Thank you both for your interest and for helping with this issue.
Good news! We have possibly identified the underlying reason why this keeps happening on the S3. A fix is in the works. If that works, we don't have to revert. Will keep you guys updated.

[mrdude2478]

@radimkarnis

Thank you for the updated news, that's great that you guys managed to find the issue so quickly. Well done and thank you for looking into this, it's very much appreciated.

[radimkarnis]

The plan is to temporarily disable this feature and make a stable v4.5.1 release. Then we can properly implement a fix, re-enable it, and test it thoroughly for v4.6.

[JunOllyLi]

@radimkarnis Thanks for the hard work!

So when can we expect the change (to make it stable in v4.5.1) in the repo and try it out?

Oh, never mind, just saw the new commit. I am going to give it a try

[radimkarnis]

@JunOllyLi it is already in master, you can clone the repo and install esptool (instructions here) to try! It will be released on PyPI tomorrow.

[JunOllyLi]

it worked pretty well last night. Thanks!

[radimkarnis]

@JunOllyLi thanks for the confirmation! I am now working on the proper fix, I will ask you to kindly try it when it's ready.

[mrdude2478]

@radimkarnis

I'll try it out as well once it's done. Thanks.

[radimkarnis]

@JunOllyLi @mrdude2478 I have created a development release with a possible fix. It can be installed with pip install esptool==4.6.dev0.

Could you please try if that solves the issue for you? Thank you!

[JunOllyLi]

I tried it several times, so far it works well

[mrdude2478]

@radimkarnis

Sorry for the late replay as I just noticed you'd replied. I updated and tried but still get issues.

C:\Users\MrDude\Desktop>esptool -b 921600 --port COM1 --chip esp32s3 read_flash 0x0 0x1000000 FAT.bin
esptool.py v4.6.dev0
Serial port COM1
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: f4:12:fa:44:a5:04
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
532480 (3 %)
A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0xfb2 bytes

C:\Users\MrDude\Desktop>esptool -b 921600 --port COM1 --chip esp32s3 read_flash 0x0 0x1000000 FAT.bin
esptool.py v4.6.dev0
Serial port COM1
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: f4:12:fa:44:a5:04
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
3186688 (18 %)
A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0xf80 bytes

Out of two attempts it failed on both.

[JunOllyLi]

emm...after seeing @mrdude2478's reply, I went back to check my log. Looks like I did not really update to the test release. The pip install said everything was already satisfied.

C:\Espressif\frameworks\esp-idf-v5.0.1>pip install esptool==4.6.dev0
Requirement already satisfied: esptool==4.6.dev0 in c:\edgetx\xesp (4.6.dev0)
Requirement already satisfied: bitstring>=3.1.6 in c:\espressif\python_env\idf5.0_py3.8_env\lib\site-packages (from esptool==4.6.dev0) (4.0.1)
Requirement already satisfied: cryptography>=2.1.4 in c:\espressif\python_env\idf5.0_py3.8_env\lib\site-packages (from esptool==4.6.dev0) (36.0.2)
Requirement already satisfied: ecdsa>=0.16.0 in c:\espressif\python_env\idf5.0_py3.8_env\lib\site-packages (from esptool==4.6.dev0) (0.18.0)
Requirement already satisfied: pyserial>=3.0 in c:\espressif\python_env\idf5.0_py3.8_env\lib\site-packages (from esptool==4.6.dev0) (3.5)
Requirement already satisfied: reedsolo<=1.6.0,>=1.5.3 in c:\espressif\python_env\idf5.0_py3.8_env\lib\site-packages (from esptool==4.6.dev0) (1.5.4)
Requirement already satisfied: cffi>=1.12 in c:\espressif\python_env\idf5.0_py3.8_env\lib\site-packages (from cryptography>=2.1.4->esptool==4.6.dev0) (1.15.1)
Requirement already satisfied: pycparser in c:\espressif\python_env\idf5.0_py3.8_env\lib\site-packages (from cffi>=1.12->cryptography>=2.1.4->esptool==4.6.dev0) (2.21)
Requirement already satisfied: six>=1.9.0 in c:\espressif\python_env\idf5.0_py3.8_env\lib\site-packages (from ecdsa>=0.16.0->esptool==4.6.dev0) (1.16.0)

[radimkarnis]

@JunOllyLi @mrdude2478 thank you both for trying.
It now seems like the fix would require a large amount of code to be added to the stub. I am therefore thinking about putting this on hold. Faster flashing/reading speeds for ESP32-S3 currently do not justify the time spent debugging and implementing a fix for this and the increased stub flasher size (the stub has to be as small as possible).

The feature works for other chips and we have a stable version (v4.5.1). I will keep this open for future reference until a real need for faster transfer speeds appears.

Thanks again for the time you've spent helping us with this!

[mrdude2478]

@radimkarnis

To be honest, I didn't notice any speed difference with the increase in cpu frequency when dumping or flashing. It was pretty much the same speed with the CPU frequency disabled. I am just happy that it's now able to dump and flash properly with 4.5.1 as stability is more important than saving a few seconds. A full 16MB dump takes about 1/2 hour, flashing is much faster though and just takes a couple of minutes.

[radimkarnis]

Closing this as increasing CPU frequency won't be implemented for the ESP32-S3. If anyone feels like this should be reconsidered, feel free to reopen.