ESP32: https request no longer works #1613

wilberforce · 2019-02-04T03:30:30Z

See:

http://forum.espruino.com/conversations/330049/#comment14604279

var http = require("http");
http.get("https://www.google.com", function(res) {
  res.on('data', function(data) {
    console.log(data);
  });
});

Uncaught InternalError: Failed! mbedtls_ssl_setup: Not enough memory

The text was updated successfully, but these errors were encountered:

GabCores · 2019-02-09T22:55:22Z

yes , I have verified same problem on version 2.01 for ESP32 but ESP8266 still working fine for https, on same version.

wilberforce · 2019-02-12T06:15:18Z

Working:

>process.env
={
  "VERSION": "1v97",
  "GIT_COMMIT": "74b2511",
  "BOARD": "ESP32",
  "FLASH": 0, "RAM": 524288,
  "SERIAL": "240ac400-2f90",
  "CONSOLE": "Serial1",
  "MODULES": "Flash,Storage,fs," ... "r,crypto,neopixel",
  "EXPTR": 1073417244 }
> 

{ "free": 2601, "usage": 2399, "total": 5000, "history": 53,
  "gc": 0, "gctime": 2.061 }
E (64222) esp_ota_ops: ota data invalid, no current app. Assuming factory
{
  "sdkVersion": "v2.1",
  "freeHeap": 129480, "addr": 65536,
  "partitionBoot": "factory"
 }


Fail:

process.env
={
"VERSION": "1v98",
"GIT_COMMIT": "1e06400",
"BOARD": "ESP32",
"FLASH": 0, "RAM": 524288,
"SERIAL": "240ac400-2f90",
"CONSOLE": "Serial1",
"MODULES": "Flash,Storage,fs," ... "r,crypto,neopixel",
"EXPTR": 1073485304 }
ESP32.getState()
={
"sdkVersion": "v3.0",
"freeHeap": 37668 }


Current:

>   console.log(process.env);
{
  "VERSION": "2v01.33",
  "GIT_COMMIT": "adb3b7d",
  "BOARD": "ESP32",
  "FLASH": 0, "RAM": 524288,
  "SERIAL": "240ac400-2f90",
  "CONSOLE": "Serial1",
  "MODULES": "Flash,Storage,heatshrink,fs,net,dgram,tls,http,NetworkJS,Wifi,TelnetServer,crypto,neopixel",
  "EXPTR": 1073484860 }
=undefined
{ "free": 3965, "usage": 135, "total": 4100, "history": 17,
  "gc": 0, "gctime": 3.262 }
=undefined
>  console.log( ESP32.getState());
{
  "sdkVersion": "v3.1.2-dirty",
  "freeHeap": 20412, "BLE": false, "Wifi": true, "minHeap": 10984 }
=undefined

wilberforce · 2019-02-12T10:12:00Z

Got this working by reducing jsvars. The current esp-idf build is using more heap. Looking at reducing the dynamic buffers - also set to 16 top buffers when 10 in the past.

gfwilliams · 2019-02-12T10:13:22Z

Normal Espruino builds actually allocate space for HTTPS out of JsVars and that was disabled for ESP32 - potentially that could be re-added I guess?

jumjum123 · 2019-02-12T10:24:14Z

@wilberforce, did you check minHeap in ESP32.getState ?
In theory, this should show minimum size of free heap, happened sometime during execution
Idea was to use this to adjust number of jsvars
freeHeap is 37 kb, which is a lot.

wilberforce · 2019-02-12T10:26:32Z

Ah - thanks - that explains it.
I think that when @jumjum123 changed to use the esp-idf version on mbedTLS that was lost. This was due to the esp-odd version being several versions ahead of the version use in espruino and there where missing functions and it would not compile.

wilberforce · 2019-02-12T10:29:49Z

@jumjum123 - I don't think free heap is that large on the current travis build - perhaps 32k.

I also recall there was a https buffer size setting in espruino - as the default is something like 15k.

A bit more reseach!

wilberforce · 2019-02-13T09:28:57Z

Looking at the available heap in the versions above v2 has much more heap than v3!

It looks like the callow used can be changed here:

https://github.com/espressif/esp-idf/blob/master/components/mbedtls/Kconfig

And the buffer size from 16k to 512 and also turn off suites not used to match the espruino/mdebtls/config.h

MaBecker · 2019-03-07T21:21:28Z

@Gabrielgab

but ESP8266 still working fine for https, on same version.

Nice, do you like to share some details using https with ESP8266?

thedamian · 2019-08-23T14:19:28Z

@MaBecker

    var http = require("http");
    http.get("https://www.google.com", function(res) {
      res.on('data', function(data) {
        console.log(data);
      });
    });

But let's get back to how we'll fix this.

thedamian · 2019-08-25T21:52:10Z

I can confirm that this is a FIXED issue on v2.04 for BOTH ESP32 and ESP8266

I suggest this issue is marked "SOLVED"

This was the code I used in BOTH devices successfully.
Devices it was successfully tested on:
ESP8266: D1 Mini & NodeMcu Lua ESP8266 ESP-12F
ESP32: ESP32-CAM , the LILYGO® TTGO & official expressif ESP32-WROOM-32D

Write up with screenshots and details is here: http://forum.espruino.com/conversations/330049/#comment14873565

var ssid = '***';
var password = '****';
var port = 80;
var wifi = require('Wifi');
console.log("started");
wifi.connect(ssid, {password: password}, function() {
    console.log('Connected to Wifi.  IP address is:', wifi.getIP().ip);
    var http = require('http');
    http.get("https://www.google.com", function(res) {
      res.on('data', function(data) {
        console.log(data);
      });
    });
    // wifi.save(); // Next reboot will auto-connect
});

olliephillips · 2019-10-12T11:34:02Z

@wilberforce I see this in v2.04 on ESP32. I'm struggling to understand what the solution is from the issue thread, although it's closed. Are there any pointers you can provide?

wilberforce · 2019-10-13T10:15:12Z

@olliephillips

With some sdk updates, related to ble, the mbedtls was switched from the esprunio version, for the esp-idf version. This was due to functions that the ble code used that was not in the espruino version( it is behind the esp-idf version). The espruino code was modified to use jsvars as memory buffers, so the current implementation is trying to allocated from the heap and there is not enough free memory for this.

olliephillips · 2019-10-13T10:51:37Z

Appreciated. Thanks. So the standard builds not very useful for http. You do a custom build to get past this? Any tips? Sorry, I’m really late to the ESP32 - my first one landed this weekend. All flashed and connected, just not very useful as is :)

…

Sent from my iPhone On 13 Oct 2019, at 11:15, wilberforce <[email protected]<mailto:[email protected]>> wrote: @olliephillips<https://github.com/olliephillips> With some sdk updates, related to ble, the mbedtls was switched from the esprunio version, for the esp-idf version. This was due to functions that the ble code used that was not in the espruino version( it is behind the esp-idf version). The espruino code was modified to use jsvars as memory buffers, so the current implementation is trying to allocated from the heap and there is not enough free memory for this. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#1613?email_source=notifications&email_token=ABWIVA4KA6BBBLKCO6G76ODQOLYLTA5CNFSM4GUB7TCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBCTBIY#issuecomment-541405347>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABWIVA5KAIP6AVDFI3CV2RLQOLYLTANCNFSM4GUB7TCA>.

wilberforce · 2019-10-13T22:30:06Z

@olliephillips
You mean https? - it should be fine for http.

A work around for a custom build would be to reduce the number of jsvars, so that these is more heap space left.

wilberforce · 2019-10-13T22:30:07Z

@olliephillips
You mean https? - it should be fine for http.

A work around for a custom build would be to reduce the number of jsvars, so that these is more heap space left.

wilberforce · 2019-10-13T22:30:13Z

@olliephillips
You mean https? - it should be fine for http.

A work around for a custom build would be to reduce the number of jsvars, so that these is more heap space left.

olliephillips · 2019-10-14T09:28:13Z

Sorry yes https. I’ll look at building with fewer jsvars. Thanks for your help @wilberforce

…

Sent from my iPhone On 13 Oct 2019, at 23:30, wilberforce <[email protected]<mailto:[email protected]>> wrote: @olliephillips<https://github.com/olliephillips> You mean https? - it should be fine for http. A work around for a custom build would be to reduce the number of jsvars, so that these is more heap space left. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#1613?email_source=notifications&email_token=ABWIVAYOIDOUSN6RGNQOQSTQOOOPRA5CNFSM4GUB7TCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBDB2YQ#issuecomment-541465954>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABWIVA3BFVCTGDPIALMEF3TQOOOPRANCNFSM4GUB7TCA>.

MaBecker · 2020-03-13T17:15:51Z

@olliephillips did you get https working? If yes please share your changes - thanks.

olliephillips · 2020-03-13T17:34:50Z

Hey Mark, I didn't, no. I had a go at it, but quickly parked it I recollect, and I've not had much time to spend with Espruino recently.

23RoMax · 2020-05-17T09:40:48Z

@MaBecker I've tried it with three different heapsizes/heapvar configurations.

As you have recommended here:

heapVars = (esp_get_free_heap_size() - 60000) / 16;
--> HTTPS Calls are possible
--> New issue I encountered: mbedtls_ssl_handshake returned -0x4290

heapVars = (esp_get_free_heap_size() - 65000) / 16;
--> HTTPS Calls are possible
--> Same issue
--> Additionally: Memory warnings

heapVars = (esp_get_free_heap_size() - 70000) / 16;
--> Memory warnings, no connection to wifi has been established by my ESP32 (coincidence? - not sure if it is related)

Now that I got HTTPS working, I still can't make use of it due to the handshake error above.
Some research suggested a memory leak.
EDIT: Second resource suggesting a memory issue here.

Did anyone encounter this issue and solve it?

23RoMax · 2020-05-24T16:40:59Z

Hi @MaBecker ,

any chance you would find a minute to have a look at my previous comment :) ?
Highly appreciated to get a hint :)

MaBecker · 2020-05-24T18:43:42Z

Hi 23RoMax,

sure, please share the url that is causing the mbedtls_ssl_handshake returned -0x4290.

Your build is using this: heapVars = (esp_get_free_heap_size() - 70000) / 16;

Have you removed some modules ?

If you like we can chat over gitter.im.

23RoMax · 2020-05-24T22:01:27Z

Thank you!
I tried several RESTful APIs as I'm trying to extend my coin tracker.

The first endpoint I tried was https://api.coinbase.com/v2/prices/BTC-EUR/buy

Another one I am successfully using with http requests is: https://api.coindesk.com/v1/bpi/currentprice/EUR.json

Also https://google.com yields the same result.

My latest build that seemed stable was with
heapVars = (esp_get_free_heap_size() - 60000) / 16;

I noticed that anything higher caused issues and crashes.
Didn't remove any other modules.

I don't have a gitter.im account yet, will set one up.

Thank you for your help!

MaBecker · 2020-05-30T05:05:02Z

Ok, I also disabled the dynamic var allocation by commenting

Espruino/boards/ESP32.py

Line 40 in 808f7af

'DEFINES+=-DJSVAR_MALLOC', # Allocate space for variables at jsvInit time

and set

Espruino/boards/ESP32.py

Line 22 in 808f7af

    
           'variables'                : 2500, # JSVAR_MALLOC is defined below - so this can vary depending on what is initialised

to 1800.

this will give you a very large heap space

MaBecker · 2020-08-30T11:44:12Z

It is all about free heap space - best choice is to buy a ESP32 Wrover board because it has PSRAM.

wilberforce added the ESP32 This is only a problem on ESP32-based devices label Feb 4, 2019

wilberforce mentioned this issue Mar 7, 2019

freeHeap and vars #1625

Closed

gfwilliams closed this as completed Aug 27, 2019

wilberforce reopened this Oct 13, 2019

MaBecker mentioned this issue Mar 21, 2020

Improve ESP32 implementation #1777

Open

33 tasks

MaBecker closed this as completed Aug 30, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ESP32: https request no longer works #1613

ESP32: https request no longer works #1613

wilberforce commented Feb 4, 2019 •

edited

Loading

GabCores commented Feb 9, 2019 •

edited

Loading

wilberforce commented Feb 12, 2019 •

edited

Loading

wilberforce commented Feb 12, 2019

gfwilliams commented Feb 12, 2019

jumjum123 commented Feb 12, 2019

wilberforce commented Feb 12, 2019 •

edited

Loading

wilberforce commented Feb 12, 2019 •

edited

Loading

wilberforce commented Feb 13, 2019

MaBecker commented Mar 7, 2019

thedamian commented Aug 23, 2019

thedamian commented Aug 25, 2019

olliephillips commented Oct 12, 2019

wilberforce commented Oct 13, 2019

olliephillips commented Oct 13, 2019 via email

wilberforce commented Oct 13, 2019

wilberforce commented Oct 13, 2019

wilberforce commented Oct 13, 2019

olliephillips commented Oct 14, 2019 via email

MaBecker commented Mar 13, 2020

olliephillips commented Mar 13, 2020

23RoMax commented May 17, 2020 •

edited

Loading

23RoMax commented May 24, 2020

MaBecker commented May 24, 2020

23RoMax commented May 24, 2020

MaBecker commented May 30, 2020

MaBecker commented Aug 30, 2020

ESP32: https request no longer works #1613

ESP32: https request no longer works #1613

Comments

wilberforce commented Feb 4, 2019 • edited Loading

GabCores commented Feb 9, 2019 • edited Loading

wilberforce commented Feb 12, 2019 • edited Loading

wilberforce commented Feb 12, 2019

gfwilliams commented Feb 12, 2019

jumjum123 commented Feb 12, 2019

wilberforce commented Feb 12, 2019 • edited Loading

wilberforce commented Feb 12, 2019 • edited Loading

wilberforce commented Feb 13, 2019

MaBecker commented Mar 7, 2019

thedamian commented Aug 23, 2019

thedamian commented Aug 25, 2019

olliephillips commented Oct 12, 2019

wilberforce commented Oct 13, 2019

olliephillips commented Oct 13, 2019 via email

wilberforce commented Oct 13, 2019

wilberforce commented Oct 13, 2019

wilberforce commented Oct 13, 2019

olliephillips commented Oct 14, 2019 via email

MaBecker commented Mar 13, 2020

olliephillips commented Mar 13, 2020

23RoMax commented May 17, 2020 • edited Loading

23RoMax commented May 24, 2020

MaBecker commented May 24, 2020

23RoMax commented May 24, 2020

MaBecker commented May 30, 2020

MaBecker commented Aug 30, 2020

wilberforce commented Feb 4, 2019 •

edited

Loading

GabCores commented Feb 9, 2019 •

edited

Loading

wilberforce commented Feb 12, 2019 •

edited

Loading

wilberforce commented Feb 12, 2019 •

edited

Loading

wilberforce commented Feb 12, 2019 •

edited

Loading

23RoMax commented May 17, 2020 •

edited

Loading