Initial rBoot bootloader implementation #816
Comments
|
Ok, finished off the prototype. Sorry for the delay, I've been pretty busy. Spiffs is now supported, each of the two roms gets it's own spiffs after the rom (position configurable) and the correct one is mounted depending on the rom in use. I've removed the url parameter from the ota method, I realised it didn't really make sense to include that. There are pro and cons, but mostly it's not needed and doing it properly would require a better understanding of the internals of the lua interpreter than I currently have. The instructions above have been updated. Not actually had any feedback though, so I wonder if people are really that interested! |
|
@raburton I'm very interested, just not in a position to try anything at this time. |
|
I'm very tempted to try this feature out. Is it possible to include a SPIFFS filesystem with the upgrade? And also, would that filesystem be R/W? |
|
Yes, there are two ways to add the spiffs to the OTA.
Any spiffs written this way would be read/write as normal, but remember that if you are going to update the spiffs in this way each time you will loose anything that has been written to it by the user. |
|
@robertfoss I've added an rboot.otafs() method that will pull a spiffs bin file and write it to the other roms spiffs area (not the spiffs of the currently running rom), so you can do that alongside the actual rom update. Also fixed a little bug where the spiffs for the second rom was in the wrong place on a 4mb flash. |
|
@raburton Those are killer features. Thank you for taking the time to implement them. |
|
It's been a month and I've had no feedback. Doesn't look like anyone is very interested. |
|
That was the impression I had in my discussion thread too. I think it's just too early at this time. If you don't mind, keep your changes around and maybe down the track interest will grow/resume/something? |
|
Was just searching for this feature, great to see someone working on it :) unfortunately I'm not familiar with building NodeMCU so haven't had a chance to test it out. |
|
Hi @raburton . I am very interested in this. I am using my own flavor of NodeMCU + Esp-ginx + SPIFFS. I just got done implementing the file-serving methods using SPIFFS and am now working on implementing OTA updates, so I will definitely be looking into this. I was actually pretty concerned that I would have to figure out how to implement this myself until I saw this.. I should have feedback for you sometime this week. |
|
@someburner do feel free to have a play with the alternative approach too, over in #806 (branch ref in this comment). |
|
@jmattsson thanks for the heads up. Is there an example of loading either of these without LUA modules? Do I need to use rtctime_early_startup() or is there a way I can call (link?) the bootloader from user_main() ? I'm still a novice when it comes to the linker and bootloader. |
I think this is the problem, nodemcu is a distributed as a binary which users flash once and forget about, only working with their apps afterwards. Unlike, for example, sming or sdk apps, where the framework forms part of the same rom. Sure, you can get nodemcu as source, but unless you are looking to fix it or add to it why would you? And perhaps much of it's target audience are just learning a bit of programming with lua scripts so wouldn't be in a position to do these things anyway. To be useful it doesn't need to be something users play with, it needs to be built in to the official rom and the ota updates put on the projects own server so that users can just use it when a new version is released. |
Should be easy enough to add the spiffs ota as well, most if not all of the code you need to do it will already be in that branch. (Although another option to make it even easier is to append the two roms and update as a single unit.) Let me know if you need a hand. Sming uses rBoot to do ota of the app and spiffs filesystems. There is a sample sming app that demonstrates the use of this using the Sming rBootHttpUpdate class, itself a wrapper for rboot-api, if you need more examples of use. And as @jmattsson says there are other ways too, see what suits you best if you're doing something custom. |
|
Heya! I seemingly succeeded to both build and flash your fork, but after "Booting rom 0" I get nothing more on baud 74880 and nothing on 9600. This is the unit I'm trying to run it on: http://www.aliexpress.com/item/rgb-strip-WiFi-controller-1-port-control-15-rgb-lights-communicate-with-Android-phone-to-dim/32301423622.html. That said, I haven't been able to run anything but the original firmware and v0.9.2.2 AT on it so far. I would love some help on the matter :) |
That's probably not a good starting point. This devices uses a non-standard pin for the serial port, so you'll probably need to modify nodemcu to get any serial output once it's booted. Funnily enough I have one of these and had to do the same with sming, before I discovered how bad pwm is on the esp. I'm in the process of designing my own board, with hardware pwm and bigger mosfets. PCB almost done, just waiting for parts from China to test a breadboard prototype. |
|
How does that work with the bootloader? I can flash, and verify that it works. Does the bootloader push RX on all pins during download mode? Your project sounds cool, are you perhaps releasing it later on? Cheers! Edit: Could you perhaps give me a pointer as to where I can find the pin assignment for the serial output? I'm out of my depth here :) |
|
Bootloader, rom boot messages and rBoot output will go to both serial lines, but once the sdk has initialised the serial ports you need to output to them individually (as I understand it), which is what NodeMCU will be doing under the covers. Had same problem with Sming, as soon as I got past rBoot it looked like it had hung, but it was just that all output was going to uart0 while uart1 was actually wired to the header pins. rBoot runs fine on them though (no reason it shouldn't of course), had that and Sming going fine, only gave up on them when I discovered how flaky pwm is (also probably need a bit more current. Once I have the device finalised I'll publish the designs. With cheap boards from dirtypcbs and a few parts off aliexpress, probably work no more than $10 a piece (I'm not planning on selling them, making them will be up to anyone who wants one). |
|
FYI, I've updated the rBoot branch in my fork (https://github.com/raburton/nodemcu-firmware/tree/rboot) to latest master + rBoot implementation (obviously). I thought it was worth keeping it up to date, although few seem interested at present it might come in handy at some point in the future. |
|
@raburton FYI: I've been keeping a close eye on this as a potential strategy for my device. Initially, I was convinced that device updates could be managed with just Lua files, but that locks out future enhancements (or bug fixes) that may require new binaries. So, it's good to have an OTA update strategy for firmware itself when devices are deployed to the field. |
|
@raburton @jmattsson I'm also keeping an eye on this, as well as #806. At the end of my project I'll have something like 40 nodemcu devices deployed around my house, so doing serial updates would be extremely time consuming. Having OTA updates would allow flashing all devices via some automated process. |
|
I just got time to start playing with this. Here's where I'm at: I'm not using the latest NodeMCU source and mine has branched off significantly since I'm not interested in any Lua features. I updated my rboot folder (apis as well as the main rboot folder), and have everything compiling and running on a 4MB ESP12 with rBoot 1.4, split config. I re-wrote the OTA API to use my own implementation and it kind of works. But my issue is not with the OTA I don't think. I have read through your blog and readmes several times but am still having trouble wrapping my mind around the memory mapping. The actual addressing seems straightforward enough, and I've playing around quite a lot with SPIFFS in the past. But basically, whenever I try to switch ROMs, do an OTA, or anything like that, it crashes-- although I'm pretty sure there is a good reason for it: I looked through rboot-bigflash.c and I'm fairly certain that Cache_Read_Enable_New is not being linked properly because uncommenting the ets_printf doesn't give me anything. But since it's all mapped I'm not sure if there's another way I can check whether that is indeed the case. Is there any chance you could shed a little more light on the linking process for Cache_Read_Enable? From what you've written it doesn't sound like there's a definitive way to accomplish it. But perhaps you have some new info on that? The other thing I'm thinking is that I'm still a novice when it comes to makefiles and I'm not sure where to add in extra things (if it's even necessary). I noticed in the more recent NodeMCU builds there is a prototype for Cache_Read_Enable(uint32_t b0, uint32_t b1, uint32_t use_40108000) and I'm curious about that as well. Any help on any of this would be greatly appreciated, and thanks again for making the port! |
|
@raburton @marcelstoer I assume this error arises because I must add |
|
Someone else had this same issue with docker, but we never did work out why. It's not esptool2 related, although you will need that too. libmain2.a is created from libmain.a using the objcopy command (it just has one symbol weakened to allow a function to be overridden). The make file should be doing this for you, and if you look higher up the log you should see it happen, but for some reason it wasn't getting found. Have a look in the build dir and you should find the file. One easy option it so just copy it to the sdk lib folder, along side the original libmain.a, and it should find it then. |
|
@TriAnMan I think it's because you don't have libmain2 copied over, or it's not in the right directory. Here's how mine looks: The "............" is to show some separation. But these are the relevant sections that I have in my Makefile to automatically copy libmain and make a lib directory in sdk-overrides. |
|
I've just manually copied |
|
libmain2 isn't an exact copy of libmain, there wouldn't be much point in that. See the part of the Makefile that generates it, you need to use the objcopy command with the -W option, as below: |
|
@raburton my steps with docker was:
I'll try. (: |
|
I've made an objectcopy and got another error: I ran docker with |
|
My esp-rfm69 project is loosely based on NodeMCU firmware (started ~SDK 1.4.1) and I have a websocket uploader that you can drag-and-drop files to the ESP.. But it's all in C. I haven't pushed up any code for rboot OTA version of this and I cut out Lua completely, so I'd have to take some time to make a Lua version. But I think it shouldn't be too hard. Again, never used Lua before, can someone comment on what is required for loading Lua scripts? Are they just stored in SPIFFS? Are you looking more for MQTT OTA or websocket file upload? I think MQTT would probably be better for most cases, especially with public brokers being available. |
|
My task is to provide the Lua Script Language for customers, you know they could operate it like node.js, whatever they like to do, it is so easy. To achieve the goal, for my work, I have to integrate all modules into the firmware. Including the OTA not only for the firmware update but the script. I am very lucky, I found @raburton 's code for firmware update. Sure, the most important thing is let the firmware could update using OTA from cloud, It worked now. The left is how to update the script through cloud. In fact, I just want to find a shortcut, I mean it is better for me if I could find a good code from Github, I could save more time. Correct! I used the MQTT to publish a message to inform the Node who need to update now, then updating.... Maybe, like @raburton 's code, the node receive the request, then to start up a request to webserver for the correct downloading script. |
|
This OTA is soooo useful for large projects or products that target end @raburton, thanks for your great work. Anything I can do to help? Maybe I can start to add custom URL? I would like to stick with NodeMCU for my WiReboot On Fri, Jul 1, 2016 at 1:01 AM, ymxcc [email protected] wrote:
|
|
@mikewen Thanks for the offer. If you can add the custom URL option that'd be great, or if you are pretty good with the internals of nodemcu any improvements you could make to the integration (to make it more inline with the current internals, reuse stuff that's already there, etc.) would be nice. |
|
Could you give me an example for Lua Script Update? Thanks so much! |
|
@ymxcc sorry, I don't actually use lua myself so can't give you a great answer. Suggestions - have your lua scripts in a spiffs, which you can ota update, or just download them with a http request. @ALL sudden flurry of interest in this thread, but I'm leaving for a couple of weeks holiday this afternoon, so won't be able to participate till I return... |
|
@raburton Ok, thanks! Have a good trip! |
|
@ymxcc I have two scripts I used to auto update the Lua on projects of mine. One is a HTTP update script which pulls a update.json file parses the file for further instructions. Typically the instructions are just a list of files to download. I then have a compile script which runs on boot which verifies if a file needs to be compiled. If so it compiles and deletes the source version. Similar to has been suggested before I call the update script from a MQTT listener. If there is a desire I'll get a Gist posted and get a link to you some how. I hate to hijack this issue. |
|
Some comments:
|
|
@devyte sorry for slow reply, been on holiday.
|
|
@raburton , I encountered a terrible problem. Although the received correct connection with webserver, however, in the nodemcu, the update processing was unsuccessful, I got some information as follows: ets Jan 8 2013,rst cause:4, boot mode:(3,6) wdt reset rBoot v1.4.1 - [email protected] Rom 0 is bad. I checked some information published before, like some guys @pjsg, @djphoenix, @raburton said, the reason is pointed to net.cert.verify function. My question is how to "Better to embed the ca info at build time instead of trying to do it dynamically." What's meaning "ca info", and how to embed the ca info at build time instead of trying to do it dynamically? I am looking forward to receive your or other responds! Thanks so much!!!!!!! |
|
need help! thanks! load 0x40100000, len 1392, room 16 rBoot v1.4.1 - [email protected] Rom 0 is bad. ets Jan 8 2013,rst cause:4, boot mode:(3,6) wdt reset rBoot v1.4.1 - [email protected] Rom 0 is bad. |
|
Maybe related to #1390 (comment). |
|
@devsaurus it does not work! @raburton @djphoenix @pjsg |
In Makefile I see this part: $(TOP_DIR)/app/modules/server-ca.crt.h: $(TOP_DIR)/server-ca.crt
python $(TOP_DIR)/tools/make_server_cert.py $(TOP_DIR)/server-ca.crt > $(TOP_DIR)/app/modules/server-ca.crt.hSo just leave your server CA in |
|
@djphoenix @raburton I use the openSSL toolbox to create a server-ca.crt (as follows for the content in it) and put it into the rboot of the nodemcu firmware build tree, it works. MIIHyTCCBbGgAwIBAgIBATANBgkqhkiG9w0BAQUFADB9MQswCQYDVQQGEwJJTDEW When I upgrade, the problem still existed. Could you have any idea, thanks! |
|
@djphoenix @raburton Generally, if you do not set the flash size, the default size is 4Mbit with the flash command: e.g. esptool.py --baud 115200 --port /dev/tty.wchusbserial1420 write_flash 0x0000 rboot.bin 0x2000 eagle.app.v6.bin. In other word, the firmware from webserver (OTA) will be flashed from 0x4200. An example from my wrong operation as follows:
ets Jan 8 2013,rst cause:2, boot mode:(3,4) load 0x40100000, len 1392, room 16 rBoot v1.4.1 - [email protected] Rom 0 is bad. Obviously, my flash size is 32Mbit, however, the initial address is 0x4000. So, when I was updating the firmware, the part 0f rom 0 would be rewritten by the new firmware. Additionally, @raburton 's provided that "You can can then write your two roms to flash After correcting the error, the update firmware ran pretty well. Parts of correcting information I printed as follows: I hope my mistake as a suggestion or remind for somebody. By the way, about the certificate ca, I also had a try. Unfortunately, I do not think that is main problem for OTA failure. But I insist that you'd better to make your code encryption protection. |
|
Hey, what is the current status of this Issue? I'd really like to have a solution for OTA updates for nodemcu! |
|
As far as I know OTA in nodemcu never happened due to a lack of interest. |
|
Have you seen this? ==> #2060 |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
After discussion in #806 (and as previously requested in #477) I have prototyped rBoot integration into nodemcu. You can find it here: https://github.com/raburton/nodemcu-firmware/tree/rboot
This allows you to have two versions of nodemcu-firmware on the device at the same time, switch between them and perform over the-air-updates (OTA).
You can use the following commands to interact with it:
print(rboot.rom())- prints the current running rom number (0 or 1).rboot.swap()- change the active rom and reboot into itrboot.ota()- perform an OTA update of the romrboot.otafs()- perform an OTA update of the spiffs (for the other rom, not the current one!)You will need:
Instructions:
print(rboot.rom())and check it returns 0.rboot.ota()and wait a minute, a second copy of the rom will be flash and the device will reboot into this rom.print(rboot.rom())and check it now returns 1.The text was updated successfully, but these errors were encountered: