Update to sdk 2.2 #2269
Update to sdk 2.2 #2269
Conversation
|
I have tested the firmware and can confirm, that the issue mentioned in espressif/ESP8266_NONOS_SDK#46 (comment) still exists. I get a lot of connects/disconnects because of BEACON_TIMEOUT. Hardware: WeMos D1 Mini sntp.sync()
function logEvent(event)
print(event)
local f = file.open("wifievents.log", "a+")
f:writeline(rtctime.get()..event)
f:close()
end
wifi.eventmon.register(wifi.eventmon.STA_CONNECTED, function(T)
logEvent("\tSTA - CONNECTED".."\tSSID: "..T.SSID.."\tBSSID: "..T.BSSID.."\tChannel: "..T.channel)
end)
wifi.eventmon.register(wifi.eventmon.STA_DISCONNECTED, function(T)
logEvent("\tSTA - DISCONNECTED".."\tSSID: "..T.SSID.."\tBSSID: "..T.BSSID.."\treason: "..T.reason)
end)
wifi.eventmon.register(wifi.eventmon.STA_AUTHMODE_CHANGE, function(T)
logEvent("\tSTA - AUTHMODE CHANGE".."\told_auth_mode: "..T.old_auth_mode.."\tnew_auth_mode: "..T.new_auth_mode)
end)
wifi.eventmon.register(wifi.eventmon.STA_GOT_IP, function(T)
logEvent("\tSTA - GOT IP".."\tStation IP: "..T.IP.."\tSubnet mask: "..T.netmask.."\tGateway IP: "..T.gateway)
end)
wifi.eventmon.register(wifi.eventmon.STA_DHCP_TIMEOUT, function()
logEvent("\tSTA - DHCP TIMEOUT")
end)
wifi.eventmon.register(wifi.eventmon.WIFI_MODE_CHANGED, function(T)
logEvent("\tSTA - WIFI MODE CHANGED".."\told_mode: ".. T.old_mode.."\tnew_mode: "..T.new_mode)
end)I tested it on the same device with the firmware from The output with the new firmware looks like this: This means, there is a disconnect at least one per hour: When I run the same script with the SDK 2.1.0 firmware, I don't get any disconnects. |
|
There was a connectivity issue fixed post 2.2 release. I believe you have to erase the sdk settings, or erase the entire flash, as the settings bytes have changed order. |
|
Should we pull in all changes Espressif added to their 2.2.x branch post release? -> espressif/ESP8266_NONOS_SDK@master...release/v2.2.x It may be helpful for issues like the one above but traceability will definitely suffer. |
|
@devyte: thanks for the hint. I flashed the firmware again and erased the flash (I was using NodeMCU PyFlasher and set |
|
I'm not sure, having to forcibly erase the sdk settings was only one of the issues we had at our repo. |
We could pull additional commits from the SDK repo with the mechanism which was added to the Makefile to pull Espressif's former patch ZIPs. Should work well for binary blobs like libwifi.a. |
|
Here's an example of something we needed to add over at Arduino for station mode (I'm paracoding): |
|
Also: there is now a country policy api for radio compliance (start channel, number of channels). If you don't set it, it defaults to China. |
These defaults seem reasonable and I'd support to add them here as well. What happens if they are set to 0?
We'd probably need a Lua API for that at some point. Are there any drawbacks if it remains at CN default for the time being?
|
|
For reference: I've raised the question regarding init of new struct members in espressif/ESP8266_NONOS_SDK#103. |
|
@nwf there are two sets of esp_init_default_data since espressif/ESP8266_NONOS_SDK@d62e5a7. What was your reason to use v05 instead of v08? |
|
@devsaurus The commit message for espressif/ESP8266_NONOS_SDK@d62e5a7 says "Rename old bin to v05.bin" and I thought sticking with the old one was the least invasive change. |
|
That's a valid point, for sure. I was a bit confused to see that 2.2.0 ships 2 variants of the init data while being unable to find any directions in the SDK docs about differences or intended use cases. On the other hand I appreciate that Espressif established a version identifier in the 2nd byte with new v08. This enables our |
|
Stumbled upon an issue where print("Connecting to wifi...")
function connected(info)
print("connected to station " .. info.SSID)
wifi.eventmon.unregister(wifi.eventmon.STA_CONNECTED)
connected=nil
end
function got_ip(info)
print(string.format("got ip: %s, gw: %s", info.IP, info.gateway))
wifi.eventmon.unregister(wifi.eventmon.STA_GOT_IP)
got_ip=nil
end
wifi.setmode(wifi.STATION)
wifi.sta.config({ssid="SSID",
pwd="password",
connect_cb=connected,
got_ip_cb=got_ip,
auto=false, save=false})
wifi.sta.connect()Station does not connect. Added at the beginning of memset(&sta_conf.threshold, 0, sizeof(sta_conf.threshold));Station connects. Seems there's some randomness involved when threshold is not initialized at all... |
|
@devsaurus the threshold content is used to filter APs during scan prior to connect. If the values inited there aren't correct, then you get what you just encountered. |
|
Understood that @devyte. The memset wasn't meant to be a solution, rather a demonstration of the failure's sensitivity. |
|
@Code1110 I can't reproduce the periodic disconnects (after having applied the temporary kludge from #2269 (comment)). Which modules did you select for your test build from Note that you can get calendar time format with |
|
@devsaurus I can't tell you exactly atm, because I'm im holidays, but I think, I used the default set of modules and added http, sjson, sntp, rtctime, i2c, si7021. Please note, that I haven't applied any patches/workarounds and just cloned the repo from nwf:for-upstream-sdk22 and built it using the docker image from marcelstoer. |
|
@devsaurus Are node major and minor synchronized to the SDK? That is, should it now be Separately, if |
Yes, though I can't provide a reference atm. Stringification is a good idea 👍 |
Initial commit for nodemcu#2225 . Replay patches from Espressif's repository at https://github.com/espressif/ESP8266_NONOS_SDK between tags v2.1.0 and v2.2.0: 0001-sync-from-ccca00f2.patch Superseded by existing changes, but lines reordered in app/driver/key.c to minimize divergences. 0002-sync-from-3f38ad5a.patch Upstream files only 0003-Update-links.patch Not meaningful to NodeMCU 0004-sync-from-01990ad0.patch 0005-sync-from-cdf6877d.patch Upstream files only 0006-sync-from-f29e744c.patch Upstream files only, user_interface.h override non-conflicting 0009-feat-lwip-Move-lwip-source-code-to-third_party-folde.patch Merged change to lwip/app/espconn_udp.c; rest is just moves or appears to not apply. 0010-feat-mbedtls-Add-mbedtls-source-code-in-third_party-.patch Does not apply; we use our own mbedtls 0011-added-C-support.patch Merged to Makefile 0012-feat-mbedtls-Rebuild-libmbedtls.patch Already applied 0013-fix-at-Fix-some-bugs-of-AT.patch Upstream files only 0014-feat-err_t-Redefine-err_t-to-s32_t.patch Merged to app/include/arch/cc.h and ./app/include/lwip/app/espconn.h; the rest is upstream files. 0015-fix-wpa-Fix-wpa-wpa2-ptk-gtk-reinstallation-vulnerab.patch 0016-fix-wifi-Remove-group-key-entry-before-connecting-to.patch 0017-feat-lib-Remove-time-function-in-libmain.patch Upstream files only 0018-feat-espconn-Modification-for-espconn.patch Merged to app/include/lwip/app/espconn.h, app/include/lwip/app/espconn_tcp.h, app/lwip/app/espconn.c, app/lwip/app/espconn_tcp.c 0019-feat-at-Use-new-espconn_recv-to-fix-tcp-server-issue.patch 0020-feat-examples-Update-mqtt-demo-and-auto-bin-generate.patch Upstream files only 0021-wifi-Add-scan-threshold-and-dwell-time.patch 0022-feat-wifi-Add-country-code-API.patch 0023-feat-wifi-Record-more-information-of-scanned-ap.patch Upstream files only, user_interface.h override non-conflicting 0024-fix-example-Fix-IoT_Demo-user-sector-error.patch Upstream files only 0025-fix-lwip-Fix-sequence-number-error-of-RST-ACK.patch Merged app/lwip/core/tcp_in.c 0026-fix-mbedtls-Fix-memory-leak.patch Merged app/mbedtls/app/lwIPSocket.c 0027-fix-mbedtls-Fix-call-send-callback-function-failed.patch Merged app/mbedtls/app/espconn_mbedtls.c 0028-feat-Add-USE_OPTIMIZE_PRINTF-in-third_party-Makefile.patch Merged app/Makefile 0029-fix-api-Fix-ets_delay_us-declaration.patch Upstream files only, osapi.h override non-conflicting 0030-fix-wifi-Remove-max_tx_power-in-wifi_country_t-in-li.patch 0031-fix-wifi-Fix-softAP-wrong-behavior-after-call-system.patch 0032-fix-wifi-bugfix-of-scan-fail-after-connected-if-max-.patch 0033-feat-at-Enable-scan-time-scan-type-and-add-scan-resu.patch 0034-feat-at-Add-command-AT-CWCOUNTRY.patch 0035-fix-at-Fix-that-AT-CIPSTART-causes-busy-if-the-serve.patch Upstream files only 0036-feat-mbedtls-Speed-up-mbedtls-handshake-process.patch Merged app/mbedtls/app/espconn_mbedtls.c 0037-fix-api-Fix-os_calloc-declaration.patch Merged app/include/lwip/mem.h; sdk-overrides/include/mem.h non-conflicting. 0038-fix-mbedtls-Fix-disconnect-callback-function-never-b.patch Merged app/mbedtls/app/espconn_mbedtls.c; minor revision to logic in 6576af9. Whitespace fixes. 0039-feat-at-Add-country-code-start-channel-in-AT-CWCOUNT.patch 0040-fix-net80211-Fix-Null-pointer-in-ieee80211_rfid_locp.patch Upstream files only 0041-feat-wifi-Add-new-esp_init_data_default-v08-bin.patch Upstream files only, but impacts Makefile 0042-fix-mbedtls-Fix-load-cert-fail-when-the-private-key-.patch Merged app/mbedtls/app/espconn_mbedtls.c 0043-fix-wifi-The-start-channel-can-be-any-valid-channel.patch 0044-fix-wifi-Fix-scan-do-not-start-after-connect.patch 0045-feat-wifi-Add-keep-connection-for-station-to-keep-co.patch 0046-feat-at-Update-AT-version-to-1.6.0.0.patch 0047-fix-at-Fix-GSLP-too-long-time.patch 0048-fix-at-Fix-the-message-is-incorrect-when-creating-UD.patch 0049-feat-at-Add-AT-CIPSERVERMAXCONN.patch Upstream files only 0050-feat-system-Add-softap-distributes-station-ip-event.patch Upstream files only, user_interface.h override non-conflicting 0051-feat-example-Use-libmbedtls.a-instead-of-libssl.a-in.patch Upstream files only 0052-feat-mesh-Remove-mesh-support.patch Upstream files only, but go ahead and remove comment from ld/nodemcu.ld. 0053-fix-example-Fix-forget-to-add-integer-parameter-when.patch Upstream files only 0054-fix-mbedtls-Fix-reconnect_callback-is-not-triggered-.patch Merged app/mbedtls/app/espconn_mbedtls.c 0055-feat-at-Add-AT-SYSMSG-to-enable-some-report-informat.patch 0056-fix-at-Fix-the-incorrect-link-id-when-client-connect.patch 0057-fix-at-Fix-the-bug-that-it-should-be-error-when-the-.patch 0058-fix-smartconfig-Fix-the-smartconfig-scan-time-issue.patch 0059-fix-lwip-Fix-the-bug-of-lwip-output.patch Upstream files only 0060-fix-lwip-Fix-the-length-of-TCP-data-in-one-packet-is.patch 0061-fix-lwip-Fix-send-TCP-data-with-two-or-more-pbuf.patch Merged app/lwip/core/tcp_out.c 0062-fix-wifi-Fix-assert-happen-when-smartconfig-start-th.patch Upstream files only 0063-fix-mbedtls-Fix-memory-leak-when-ESP8266-as-SSL-TLS-.patch Merged app/mbedtls/app/espconn_mbedtls.c 0064-fix-mbedtls-Fix-already-freed-and-exception-bug-when.patch Merged app/mbedtls/app/lwIPSocket.c 0065-fix-at-Fix-bug-that-there-is-no-result-when-sending-.patch 0066-feat-example-Add-AT-bin-version.patch 0067-feat-version-Update-version-to-2.2.0-and-add-version.patch 0068-feat-bin-Update-AT-bin-for-SDK-2.2.0.patch Upstream files only Apply local changes to build: app/include/lwip/app/espconn.h pulls changes (and license decl) from upstream SDK. Makefile is altered to use this file ahead of the SDK's. Remove lwip's sntp support, since it was never really wired in anyway. See nodemcu#2042 for more information. Patch Makefile to strip time.o, the consumer of lwip's sntp functionality, from libmain.a, resulting in much easier-to-understand error messages. This has consequences for mbedtls. The simplest thing to do, which is, impressively, not a change in behavior, is to completely disable TLS certificate time validation; a later patch can optionally couple this to RTCTIME support. Similarly, it happens that the sqlite3 import was calling time(), but this was not going to work out well for it. Just stub it out to always return unix timestamp 0, as would have happened anyway. Changes unprocessed: 0007-sync-from-080c37e1.patch 0008-feat-lib-Compile-some-libraries-with-ffunction-secti.patch These two make changes to the linker script; perhaps they are worth porting over, but I have not done so here. This is build-tested (ADC, BIT, COLOR_UTILS, CRON, CRYPTO, DHT, ENCODER, FILE, GPIO, HTTP, I2C, MQTT, NET, NODE, OW, PCM, PERF, PWM, RTCFIFO, RTCMEM, RTCTIME, SNTP, SPI, SQLITE3, STRUCT, TLS, TMR, UART, WIFI, WS2812, WS2812_EFFECTS) and boots, but only limited run-time testing has been performed. Testing done does, however, include having made a few TLS connections through the HTTP module, so things are not hopelessly broken, at the very least.
Derive NODE_VERSION by concatenating string constants.
|
New version just pushed, which kicks user_version.h over to being a wrapper around ESP's version.h. The branch has been rebased for ease of pulling, too. :) |
in light of changes made as part of nodemcu/nodemcu-firmware#2269
|
Can I assume that the ESP8266_NONOS_SDK fix Here is a statement from Espressif Systems: Information for VU#228519: https://www.kb.cert.org/vuls/id/CHEU-ARFFHX
|
|
@MarkMartinec I have no visibility into the upstream .a files (other than a disassembler, I suppose), but assuming that the commit does what it says on the tin, that would seem to be their fix for KRACK, yes. With luck, this PR will get committed to |
|
I still observe a regression with #2269 (comment) |
|
@devsaurus it failed to connected to the persisted AP after the update. I then ran |
|
RE #2269 (comment) "there are two sets of esp_init_default_data" What are the differences between the two defaults, and when should either be used? |
|
I applied the fix mentioned in #2269 (comment), but still get disconnects. Actually, I'm testing with the new patch (#2309). To confirm, that the issue really exists, I yesterday flashed two factory new nodes. One with the |
|
I did a lot of testing regarding the disconnect issue and it's hard to spot. It seems, it has something to do with the selected modules. Got disconnects with these builds: No disconnects with these builds: |
|
@Code1110 thanks for the continued testing! |
in light of changes made as part of nodemcu/nodemcu-firmware#2269
…atch in nodemcu#2269 espconn_tcp_reconnect() was removed instead of espconn_list_delete()
* fix application of patch 0018-feat-espconn-Modification-for-espconn.patch in #2269 espconn_tcp_reconnect() was removed instead of espconn_list_delete()
Initial commit for
#2225 .
Replay patches from Espressif's repository at
https://github.com/espressif/ESP8266_NONOS_SDK between tags v2.1.0 and
v2.2.0:
Apply local changes to build:
Changes unprocessed:
This is build-tested (ADC, BIT, COLOR_UTILS, CRON, CRYPTO, DHT, ENCODER,
FILE, GPIO, HTTP, I2C, MQTT, NET, NODE, OW, PCM, PERF, PWM, RTCFIFO,
RTCMEM, RTCTIME, SNTP, SPI, SQLITE3, STRUCT, TLS, TMR, UART, WIFI,
WS2812, WS2812_EFFECTS) and boots, but only limited run-time testing has
been performed. Testing done does, however, include having made a few
TLS connections through the HTTP module, so things are not hopelessly
broken, at the very least.
devbranch rather than formaster.docs/en/*.