-
Notifications
You must be signed in to change notification settings - Fork 13.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
memory leak 184 bytes at a time #1923
Comments
This memory should be reclaimed after a couple of minutes, once TCP connection exists TIME_WAIT state. Are you sure that free heap doesn't return to the original value after a few minutes? |
not leek, but 20k eating this code: ` WiFiServer server(80); const char* ssid = "???"; static const char eatHeap[] = size_t orig; void setup() void loop() ` |
and f5 key down esp die :)) |
edit tcp.c and recompile lwip to stop heap eating: /* Check if this PCB has stayed long enough in TIME-WAIT */ F5 never die :)) |
ESP8266WebServer library handles this stuff for you, without breaking TCP protocol spec. |
so this isn't a memory leak issue it is just performing according the tcp spec? In my case I use ajax to do some interaction between user and the server every second. |
This branch adds a close_abort() method to the WiFiClient class. How it works, what it does: Calling `close_abort()` will close and abort the client connection it is invoked on and, as a result, free its resources (i.e. memory). **WARNING:** aborting connections without a good reason violates the TCP protocol, because a closed connection would normally need to spend some time in `TIME_WAIT` state before its resources are freed. Usage example: WiFiClient client; // set up a client { /* do things with your client */ } client.stop_abort() // when you're done,abort the // connection if you must Why it's useful: 1. Give programmers a way to shut down connections immediately if need be. The underlying `tcp.c` file has an abort function, but this has not been directly accessible via the `WiFiClient` class until now. 2. There are a number of reported issues for the repository addressing the heap corruption that can result from trying to retain too many connections in `TIME_WAIT` state (most notably: esp8266#230, esp8266#1070, esp8266#1923). Although the warning above holds, there may be circumstances where this isn't very important. For example an ESP8266 running in AP mode hosting a page, which requests a new connection every second via an AJAX script to monitor a sensor/button/etc. continusously. Currently existing alternative approach: When building a project, defining the `-D MEMP_NUM_TCP_PCB_TIME_WAIT=5`compiler directive will limit the maximum number of clients allowed to stay in TIME_WAIT state. `5` is the default, lower it as necessary. See reference [here](https://github.com/esp8266/Arduino/blob/master/tools/sdk/lwip/include/lwipopts.h#L263) Thanks: Thank you to @me-no-dev, @everslick and @Palatis for bringing the ` MEMP_NUM_TCP_PCB_TIME_WAIT` option to my attention.
I am a little bit confused about the wifi client heap issue. I checked: 184 heap bytes are really lost each time we receive a request (this formerly produced a hangup in my application), BUT leaks are sometimes retreived after a while (a while = several minutes). If the number of client requests is somewhat high, any app will quickly run in troubles. Even when the client is destroyed in the app (local client variable after the function has returned) or if the client is stopped (client.stop()) the TCP connection should be closed. Does anyone has an idea how to proceed to leave a clean situation without waiting for minutes ? (my configuration: Arduino IDE 1.8.1 / esp8266 SDK 2.3.0 as mentionned in the arduino menu) Connecting to xxx...... connected |
since that pull request isnt merged yet and it will take some time to reach upstream, here's a workaround that actually cleanups the TIME_WAITS and frees memory:
update: This goes in your main file. Call tcpCleanup regulary, for example from your loop() function. |
psy0rz: Thank you very much! I have been struggling with this problem for way too long. This did the job!!! However, it did take me a little while to figure out the details and just for others who are as dumb as me, here is what I didn't get at first. The code you posted goes into the user's app. At first I thought it went into tcp.c like one of the previous suggestions. Also, what was pretty obvious but maybe should be said is that you need a call to tcpCleanup somewhere in the loop( ). I put it at the bottom. Again, thanks for solving this problem for me! You're a peach! |
@igrr there are quite a few threads referring to memory leaks and I'm pretty sure many of Espressif's clients will be suffering with it in some way or other. My first question relates to the free Heap change from 2.3.0 to the latest master branch. One of our sketches has 35kb free with 2.3.0 and only 30kb free with the master branch. Why is this and is there any easy way of recovering the 5kb? Below is a sketch that tries to implement the memory leak fix indicated in this thread. It seems to do the job but there are still some unknown occasions when the leak appears to be permanent. Do you have any idea how ESP users can ensure the free Heap always remains unchanged?
|
The RAM usage difference between 2.3.0 and master is tracked in this ticket: #3227. Regarding permanent heap loss, AFAIK there was a |
@psy0rz Thanx man ! that did the trick !!!!!!!!! Telegram is a piece of memory hog there, and my esp would reboot every time heap memory goes under 6500~6000(¿bits/bytes?). With that piece of code thing gets stable: |
Thanks!!! I was actually actually hitting an exception: Exception 29: StoreProhibited: A store referenced a page mapped with an attribute that does not permit stores I was using ajax and was consuming 34KB of memory in about 1 minute. Looked at the stack and saw it was probably a failed malloc. Saw this was a 184 byte issue and google lead me to this thread. Now it's worked around in 5 minutes and I was prepared for a long debug session. I'm generally a low level programmer, but the pieces are so easy to put together with this esp8266 that I'm making huge progress. I really appreciate this community! Hope to see this mainlined so others don't hit it. |
Does it look like #2925 ? |
d-a-v, it looks similar to #2925 and I think it is likely related. However, there are some differences in the original bug description there and what I'm seeing. The original bug there says it doesn't happen for them with version 2.3.0, but I am seeing the problem on that version. Also they say the problem only happens with messages over 2048 bytes, and my messages are well under that size. I am fine sharing my code by giving access to a private git repo (my code will eventually be open sourced, but I'm keeping it private until the kickstarter project launches). However, it does require some special hardware to run. Perhaps a better solution is I'm happy to test a solution when one becomes available. |
So you can give a try to #3362 (you'll use lwip-v2 instead of v1.4). |
I tried master and that actually solved my issue. It consumed 4 KB more memory than the hack above, but it didn't crash like 2.3.0. Then I tried #3362 and it seemed to cause my esp8266 to reboot without any kind of exception posted to serial (normally I see the exception and can decode the dump file). Not sure what was happening. I think any esp8266 should reproduce this issue. I posted my code and some libraries you'll need to reproduce the issue at: http://glacialwanderer.com/files/2017/Esp8266.zip |
Glad to know it works now. |
@mribble, thanx for sharing your experience. It made me try the master branch which seems to be more compact or compiler friendly or whatever... Now I have enough free heap for safe execution of my code. |
It looks like the FAQ helped get you from not compiling to the code compiling. That is a step in the right direction. Now the problem is you still have a memory leak. Where are you calling tcpCleanup()? It should be after every client.stop(). You also asked how you reuse a connection, which is the better way to fix this. This is handled by the place making these httprequests (perhaps an ajax webpage?). There if you just keep reusing the httprequest then new versions won't be created by this esp8266 code and things will run much faster than it would if you just work around this memory 'leak'. |
thx @mribble for your answer, so the 1- first i connect to the ESP the ESP receive the data and transmit to the serial, close the connection and then start the cleaning but when i try with my android app that i created with xamarin the esp receive the data and transmit it to the serial and close connection but never clean the TCP. the more i send data the more i got memory leak, here's what i got in serial : this is so weird why the ESP accept connection from both HERCULES and Android App extract the data and send it back to the serial and then disconnect. but if the connection was made through HERCULES app the ESP start cleaning but not with my android app ? is there any debug option that i can look for to see what happen exactly ? i only have this one defined in my code |
I'm not sure how your android or pc apps are implemented or if they have the correct options to avoid a new connection for each data transfer. If you were willing to write your own webpage with javascript (this wouldn't be very hard as I learned how to do it about 6 months ago in a few hours) then you would control this by calling:
only once. If you called it once for each transfer then you'll have many connections and have this memory problem where esp8266 needs to track all the connections because the tcp spec says it should. As for tools that let you see the raw tcp packet headers, I haven't ever had to do that so I'm not sure what exists. There are probably software tools that let you see the headers for tcp packets. Certainly hardware like this would let you see all this data. |
I am not sure if this problem has been answered in this thread but here is
what I did that fixed the problem 100%:
Add the following line at the top of the file
#include <https://www.facebook.com/hashtag/include?source=feed_text>
"lwip/tcp_impl.h"
And add the following method somewhere:
void tcpCleanup()
{
while(tcp_tw_pcbs!=NULL)
{
tcp_abort(tcp_tw_pcbs);
}
}
And finally add a call to tcpCleanup( ) at the bottom of the loop method.
This is the solution for apps written in C, not sure about Lua or any other
language.
…-- Don
On Sat, Mar 24, 2018 at 2:36 PM, Maurice Ribble ***@***.***> wrote:
I'm not sure how your android or pc apps are implemented or if they have
the correct options to avoid a new connection for each data transfer.
If you were willing to write your own webpage with javascript (this
wouldn't be very hard as I learned how to do it about 6 months ago in a few
hours) then you would control this by calling:
new XMLHttpRequest();
only once. If you called it once for each transfer then you'll have many
connections and have this memory problem where esp8266 needs to track all
the connections because the tcp spec says it should.
As for tools that let you see the raw tcp packet headers, I haven't ever
had to do that so I'm not sure what exists. There are probably software
tools that let you see the headers for tcp packets. Certainly hardware like
this
<https://www.kickstarter.com/projects/mossmann/hackrf-an-open-source-sdr-platform>
would let you see all this data.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1923 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AEVNXJVo0-7omogAfrmUEPOlV2bH9X5jks5thrxigaJpZM4IJDi->
.
|
i maybe know why for example in my android app i have a button send which is responsible to connect to the ESP and send a data. for information i add the
i don't have a knowledge in the TCP Protocol so i can't explain why this happen and how the TCP deal with these situation. i succeeded to get around with this problem in my Xamarin app but i'm not convinced with my solution. |
In my javascript I created a ringbuffer of XMLHttpRequest objects so that I could reuse these. My understanding is this should reuse the connection and prevent the 184 byte memory growth. This is important to me because I'm using a dynamic ajax that causes 4 transfers per second and I'll quickly exhaust memory. This worked fine for lwIP v1.4 Higher Bandwidth. It also works for lwIP v2.0 Higher Bandwidth. However the default is now lwIP v2.0 Lower Memory, and it causes the memory to grow until I run out of memory. I find this odd that you are growing the table of connections even when I'm reusing connections. It seems like there might be a bug here, but I haven't looked at this code. It feels like perhaps you are only reusing the latest connection (since I have a small ring buffer that doesn't help me), but still the code is wasting memory to track more than just one connection. Could someone explain if this makes sense? Calling tcpCleanup() does work around the issue. |
without hacking sources, with tcpdump you can check if the same connection is indeed reused just by looking at the TCP port number. |
regarding this code below when i include "lwip/tcp_impl.h" i got an error , no such file or directory, so include it the whole directory "C:/ ........lwip/tcp_impl.h" I have been searching a lot, still unable to figure out how to get this library? Where to place this library? Will appreciate any support, thanks |
Check the snippet in the documentation's FAQ.
Change is needed because of lwip v2.
|
thanks for the prompt response @d-a-v , can you please help me with the link to FAQ/documentation, apologies for the inconvinience |
It's right on your screen: |
Thanks for your prompt response, it helped big time |
Seems like I am facing this issue in universal-telegram-bot where I have to call bot.Update method every second resulting in Exception (29) when heap falls to about 6k. Secure client is used for establishing connection to telegram bot api. Any ideas on how to overcome this? |
@MrSuhov if you are using 2.4.1 core, there is a known leak which is fixed in the current git version. |
@d-a-v Thanks a lot! A fresh git clone https://github.com/esp8266/Arduino.git esp8266 was really helpful - heap keeps on a constant level! |
@d-a-v many thanks :) |
FYI, tcpCleanup is not anymore needed in latest git. |
Is it possible to get the spot fix? |
@ivelichkevich personally i've moved on from this usecase towards using raspberry pi zero/orange pi zero as a server which does all the logic, including telegram commands/messages/etc. |
What does it mean ?
That's a choice. There's less fun :) This issue is not around anymore since core-2.4.2. |
commits hashes for this fix or pease of code to fix leak in 2.4.1 Sorry, Where can I find FAQ? |
The FAQ is in the documentation. Documentation link has been recently made more visible in the main page (Quick Links) in https://github.com/esp8266/Arduino/. How it has been fixed is an idea of @me-no-dev: Links are there: This patch will be proposed to lwIP developpers at some point. |
Thanks! |
Hi, i also have this "memory leak" issue on an ESP which is quite busy (constantly updating LED's). the pattern is that sometimes, but not always the heap is decreased by 184 bytes after the call to ESP8266 Webserver handleclient. And some random time (sometimes after a few minutes, sometimes after several hours) a lot of calls to handleclient cause this 184 byte heap decrease, resulting in no more heap memory and a restart of the ESP (due to watchdog or exception) I applied the "solution in the FaQ" with the tcpcleanup() function. Result is that no more memory is leaked at first, but then again after some random time (again somewhere between a few minutes or a few hours) the handleclient starts "leaking" a lot of memory again... restarting the ESP. The application is a wordclock, so the restart is very annoying... I know this is an old topic, but hopefully anyone can give me a hint how to fix... my code can be found here |
Hi akamming and fellows, I tried tcpcleanup() and it doesn't work because Why ? The answer is because the instruction "extern struct tcp_pcb* tcp_tw_pcbs;" does NOT initialise tcp_tw_pcbs to the one used by the Client. It is only a declaration and because it is only a declaration, tcp_tw_pcbs is set to NULL. Consecuently tcp_abort(tcp_tw_pcbs) is never called. On the following lines of my message I indicate an efficient way to sove the problem. For this I added an API to WiFiClient class. Here is the code: In WiFiClient.h add the code and comments: // Api for saving precious heap. In WiFiClient.cpp add the code and comments: // Api for heap saving. Must be call just before WiFiClient::stop(). With this code you will not be annoyed by sockets in TIME_WAIT state after stopping clients. I would be gratefull if someone could make a pull request in order to put this API in the GIT repository for use by everybody because I am not familiar with pull requests. Robert |
I Fixed my problem by using another led library. The one i used had nointerrupts() builtin, which apparently caused my issue |
Basic Infos
looks like 184 bytes get lost every time you post to the page ie :'http://192.168.0.6/gpio/1'
Hardware
Hardware: ESP-12, wemos mini, esp-7
Core Version: 2.2.0-rc1
memory leak?
using the example sketch WifiWebServer added one line of code at the end of sketch
Serial.println( ESP.getFreeHeap(),DEC);
Settings in IDE
Module: wemos mini
Flash Size: ?4MB/1MB?
CPU Frequency: ?80Mhz?
Flash Mode: ?qio?
Flash Frequency: ?40Mhz?
Upload Using: ?OTA / SERIAL?
Reset Method: ?ck / nodemcu?
Sketch
/*
*/
include <ESP8266WiFi.h>
const char* ssid = "your-ssid";
const char* password = "your-password";
// Create an instance of the server
// specify the port to listen on as an argument
WiFiServer server(80);
void setup() {
Serial.begin(115200);
delay(10);
// prepare GPIO2
pinMode(2, OUTPUT);
digitalWrite(2, 0);
// Connect to WiFi network
Serial.println();
Serial.println();
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected");
// Start the server
server.begin();
Serial.println("Server started");
// Print the IP address
Serial.println(WiFi.localIP());
}
void loop() {
// Check if a client has connected
WiFiClient client = server.available();
if (!client) {
return;
}
// Wait until the client sends some data
Serial.println("new client");
while(!client.available()){
delay(1);
}
// Read the first line of the request
String req = client.readStringUntil('\r');
Serial.println(req);
client.flush();
// Match the request
int val;
if (req.indexOf("/gpio/0") != -1)
val = 0;
else if (req.indexOf("/gpio/1") != -1)
val = 1;
else {
Serial.println("invalid request");
client.stop();
return;
}
// Set GPIO2 according to the request
digitalWrite(2, val);
client.flush();
// Prepare the response
String s = "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\n\r\n\r\n\r\nGPIO is now ";
s += (val)?"high":"low";
s += "\n";
// Send the response to the client
client.print(s);
delay(1);
Serial.println("Client disonnected");
// The client will actually be disconnected
// when the function returns and 'client' object is detroyed
Serial.println( ESP.getFreeHeap(),DEC);
}
The text was updated successfully, but these errors were encountered: