Download corrupted for large flash images

[devsaurus]

I was banging my head against this the last days while adding a new module until I found this: espressif/esptool#53

It seems that the ROM code for download contains an issue which leads to a corrupted flash contents once the image grows beyond flash address 0x70000. For details refer to https://cesanta.com/blog/esp8266-flash-erasing-issue.shtml
It happens with current themadinvetor/esptool as well as the embedded esptool here. Since dev branch exceeds this limit for the float version, we're already in trouble now (integer version is briefly below the threshold).

This issue can't be resolved without a fix or workaround in esptool and other flashers.

[MarsTechHAN]

@marcelstoer Do you have better idea how to download in Linux & MacOS, Espressif's tool only work under windows?

[marcelstoer]

@MarsTechHAN Sorry, I can't help you with this, it really isn't my area of expertise.

[MarsTechHAN]

@marcelstoer But...I do think before we fix this bug or find a better way, we should not delete the present methods...Because after it, most of the man will have no idea how to download bin in Linux or Mac OS.

[marcelstoer]

@MarsTechHAN I guess your comments refer primarily to my PR-in-the-making #595, right?

• If so, let's continue the discussion there. I have the feeling your concerns are already addressed in the comments there.
• If not, then please elaborate as I might not understand your comments. I don't want to get rid off esptool.py, not at all (I use it frequently myself), but rather make sure NodeMCU always uses the up-to-date one from the themadinventor repository.

[jiegec]

Well, but themadinventor seldom updates the esptool.

[marcoskirsch]

Well, but themadinventor seldom updates the esptool.

I think that's a different issue. If it becomes a huge problem, the community can look into branching esptool into its own repository. But it still makes sense to manage that separately from nodemcu-firmware.

[devsaurus]

Well, but themadinventor seldom updates the esptool.

If we'd follow @marcelstoer's proposal in #595 we could decide to pull in a submodule even from a different fork/branch. Eg the workaround in jkent's erase_fix branch works fine for me. ...and link back once the fix is merged to upstream.

Don't know what the git experts think about this...

[marcelstoer]

@devsaurus @marcoskirsch @jiegec Even though I believe discussing in #595 would be better I also believe a reply is better than none 😉.

Well, but themadinventor seldom updates the esptool.

Depends on how you define "seldom". The last PR was merged a month ago. The more people use esptool the more people could potentially urge the @themadinventor to merge PRs more quickly.

If it becomes a huge problem, the community can look into branching esptool into its own repository.
Don't know what the git experts think about this...

Not that I would ever claim to be a Git expert...but that's exactly the point of referenced/linked repositories. We decide which version or fork to pull in - in the worst case even maintaining our own fork.

[projectgus]

Just a heads-up, esptool.py has an additional maintainer now (me!) and we've just (finally) merged the fix for this issue to https://github.com/themadinventor/esptool

It's hard for one person to keep up with a popular project, but hopefully by sharing the work around esptool.py can continue to keep current.

[devsaurus]

The original intention of this issue is closed now - esptool.py handles the download correctly.
Nevertheless, current default settings exceed the memory for 512 kB modules as stated in #788. A decision is required whether we want to provide a fail-safe default configuration & image to the novice users.