calloc fails even with very large memory block available (IDFGH-10391)

[HamzaHajeir]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue (Unhelpful).

IDF version.

v4.4

Operating System used.

Windows

How did you build your project?

VS Code IDE

If you are using Windows, please specify command line type.

None

Development Kit.

ESP32DevKit1

Power Supply used.

USB

What is the expected behavior?

Hi there,
I'm maintaining an Arduino-based H4AsyncTCP library, developing TLS over LwIP TCP by utilizing ALTCP, I've setup everything (Certificates, altcp_tls_config(s)), but the altcp_tls_new() call ended up failing to return a pcb.

After further debugging, I've found that a calloc has failed to allocate memory at:
https://github.com/espressif/mbedtls/blob/15b55d406db3918bac88aaf5ef2c6e036d1e0f0e/library/ssl_tls.c#L3857

Wherein I've diagnosed heap_caps_get_largest_free_block(MALLOC_CAP_DEFAULT) and heap_caps_get_largest_free_block(MALLOC_CAP_8BIT) (which gave me the same value).

Debug Output:
For a MQTT client _connect():

16:31:04.240 > altcp_tls: TCP_WND is smaller than the RX decrypion buffer, connection RX might stall!
16:31:04.245 > H4AT:2: H=146908 M=110580 B=110580 S=4684 ONE WAY TLS conf=0x3ffda348 PCB=0x0
16:31:04.247 > IDF/components/mbedtls/mbedtls/library/ssl_tls.c:3857: alloc(16717 bytes) failed
16:31:04.248 > mbedtls_ssl_setup failed
16:31:04.248 > tcp_close: closing in State: CLOSED

Note that H: Free Heap, M: Largest block (default), B: Largest block (8bit), S: StackHighWaterMark.

A similar output resulted by a server initiating.

I'm using a customized build of Arduino libraries, activating LWIP_ALTCP, LWIP_ALTCP_TLS, and LWIP_ALTCP_TLS_MBEDTLS macros, Plaase refer to this Feature request for more information.

What is the actual behavior?

Instead of allocating memory (even if it's available in large volume), it fails.

Steps to reproduce.

I can setup a whole test environment of the project run if really needed, which simplifies the test and installs everything on the fly with switchable TCP/TLS by a macro set. It runs very solid in TCP mode, over PlatformIO.

[HamzaHajeir]

It (might) be a misconfiguration (I'm not sure), but here's the issue:
espressif/esp32-arduino-lib-builder#125

[mahavirj]

@HamzaHajeir

Please check the memory allocation configuration for mbedTLS stack as per below reference:

esp-idf/components/mbedtls/port/include/mbedtls/esp_config.h

Lines 100 to 128 in 2135042

	* \def MBEDTLS_PLATFORM_MEMORY
	*
	* Enable the memory allocation layer.
	*
	* By default mbed TLS uses the system-provided calloc() and free().
	* This allows different allocators (self-implemented or provided) to be
	* provided to the platform abstraction layer.
	*
	* Enabling MBEDTLS_PLATFORM_MEMORY without the
	* MBEDTLS_PLATFORM_{FREE,CALLOC}_MACROs will provide
	* "mbedtls_platform_set_calloc_free()" allowing you to set an alternative calloc() and
	* free() function pointer at runtime.
	*
	* Enabling MBEDTLS_PLATFORM_MEMORY and specifying
	* MBEDTLS_PLATFORM_{CALLOC,FREE}_MACROs will allow you to specify the
	* alternate function at compile time.
	*
	* Requires: MBEDTLS_PLATFORM_C
	*
	* Enable this layer to allow use of alternative memory allocators.
	*/
	#define MBEDTLS_PLATFORM_MEMORY
	/** Override calloc(), free() except for case where memory allocation scheme is not set to custom */
	#ifndef CONFIG_MBEDTLS_CUSTOM_MEM_ALLOC
	#include "esp_mem.h"
	#define MBEDTLS_PLATFORM_STD_CALLOC esp_mbedtls_mem_calloc
	#define MBEDTLS_PLATFORM_STD_FREE esp_mbedtls_mem_free
	#endif

Additionally, putting some debug prints in components/mbedtls/port/esp_mem.c might also help if you are indeed using the ESP port allocation layer.

[HamzaHajeir]

I've found the root cause, which is caused by LwIP overriding mbedtls calloc/free functions here:

User calls:

https://github.com/lwip-tcpip/lwip/blob/e29870c15e8bf28eac9c811dd236c474f3f2008f/src/apps/altcp_tls/altcp_tls_mbedtls.c#LL863C33-L863C33

to:
https://github.com/lwip-tcpip/lwip/blob/e29870c15e8bf28eac9c811dd236c474f3f2008f/src/apps/altcp_tls/altcp_tls_mbedtls.c#LL784C19-L784C19
to:
https://github.com/lwip-tcpip/lwip/blob/e29870c15e8bf28eac9c811dd236c474f3f2008f/src/apps/altcp_tls/altcp_tls_mbedtls_mem.c#LL167C20-L167C20

And wherein LwIP wrongly identify the MBEDTLS config with this preprocessor:
https://github.com/lwip-tcpip/lwip/blob/e29870c15e8bf28eac9c811dd236c474f3f2008f/src/apps/altcp_tls/altcp_tls_mbedtls_mem.c#L71-L77

In which its calloc function compares with a fixed preprocessor value of 1600 if not overriden:
https://github.com/lwip-tcpip/lwip/blob/e29870c15e8bf28eac9c811dd236c474f3f2008f/src/apps/altcp_tls/altcp_tls_mbedtls_mem.c#L116

MEM_SIZE:
https://github.com/lwip-tcpip/lwip/blob/e29870c15e8bf28eac9c811dd236c474f3f2008f/src/include/lwip/opt.h#L290

[bmkuter]

How were you able to resolve this issue? I'm having issues myself where the dynamic buffer for mbedTLS isn't freeing itself after reading packets greater than 2900 bytes.

[HamzaHajeir]

How were you able to resolve this issue? I'm having issues myself where the dynamic buffer for mbedTLS isn't freeing itself after reading packets greater than 2900 bytes.

You can increase MEM_SIZE of LwIP configuration to a good number (I've did try for 95000) if you don't want to take a greater risk by using my own patch of esp-lwip.

However I've made a good progress and have good tests of it. Except this issue.

I'm happy to guide you further if you wanted.

[bmkuter]

Thanks! Where had you found MEM_SIZE?

[HamzaHajeir]

You can find it in lwip/opt.h

…

On Sat, Jul 8, 2023, 12:16 bmkuter ***@***.***> wrote: Thanks! Where had you found MEM_SIZE? — Reply to this email directly, view it on GitHub <#11647 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH3O7J3ZTTTHP25GQYYPRM3XPEQOXANCNFSM6AAAAAAZCLXCAQ> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>