heap_caps.c:heap_caps_get_largest_free_block() returns incorrect value (IDFGH-6129)

[glenn20]

Environment

• Module or chip used: ESP32-WROOM-32D
• IDF version(s): v4.2.1, v4.2.2, release/v4.4, v5.0-dev-79-gbcbef9a8db
• Build System: idf.py
• Compiler version: 8.4.0
• Operating System: Linux
• Power Supply: USB

Problem Description

heap_caps_get_largest_free_block(MALLOC_CAP_8BIT) always returns a value which is the largest available allocatable block rounded down to the nearest power of 2. This is in contradiction with the documentation and the semantics prior to implementation of the TLSF allocator.

The error arises from this line in the multi_heap_get_info_impl() function:

    info->largest_free_block = info->largest_free_block ? 1 << (31 - __builtin_clz(info->largest_free_block)) : 0;

The semantics of this function were changed in commit bd9b921713d80 in October 2020 as part of the implementation of the TLSF allocator. (See multi_heap.c:L374)

This bug has a significant impact on the Micropython port to all ESP32 platforms by significantly reducing the amount of RAM available to the Micropython heap when built against any IDF version since the TLSF allocator was implemented.

Expected Behavior

heap_caps_get_largest_free_block(MALLOC_CAP_8BIT) should return the size of the largest allocatable block - in agreement with the documentation and the semantics prior to implementation of the TLSF allocator.

Actual Behavior

heap_caps_get_largest_free_block(MALLOC_CAP_8BIT) returns the size of the largest allocatable block rounded down to the nearest power of 2.

[igrr]

@glenn20 Thanks for reporting the issue. We did run into the same issue recently, and there is a fix submitted for this internally. Please try changing that line as follows:

    if (info->largest_free_block) {
        uint32_t sl_interval = (1 << (31 - __builtin_clz(info->largest_free_block))) / SL_INDEX_COUNT;
        info->largest_free_block = info->largest_free_block & ~(sl_interval - 1);
    }

[glenn20]

@igrr I can confirm that fix works for me and resolves the issue for the Micropython ESP32 port. Will this fix be applied to updated v4.XX release tags, or only to future v5.X releases?

[igrr]

Should be fixed on master branch in 0028e2c, we will backport to release branches as well.

[philippe44]

Thanks for fixing that - Took me a while to figure out because I was trying to reserve 65535 bytes for DMA. Because of this issue, it split the block in two, but ended creating a pool of 65524 and ... trying to create another of 11 which failed obviously!