M5 crashes receiving Sigma3 msg - Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled.

[kean-apple]

Problem

Tried to pair M5 board with iOS chiptool app. M5 crashes during commissioning process after receiving Sigma 3 message..

SHA: 2e10854

1. Flash M5
   2.Try to pair M5 board using iOS chiptool, M5 crashes during commissioning

note: M5 continues to crash loop after commissioning fails

M5:
I (36261) chip[SC]: Received Sigma3 msg
E (36281) chip[SC]: The device does not support GetClock_RealTimeMS() API: 6c. Falling back to Last Known Good UTC Time
D (36761) esp_netif_lwip: esp_netif_nd6_cb lwip-netif:0x3ffde2a8
D (36761) event: running post IP_EVENT:3 with handler 0x4014132c and context 0x3ffde440 on loop 0x3ffdcf50
0x4014132c: chip::DeviceLayer::PlatformManagerImpl::HandleESPSystemEvent(void*, char const*, int, void*) at /Users/keanlim/connectedhomeip/examples/all-clusters-app/esp32/build/esp-idf/chip/../../../../../../config/esp32/third_party/connectedhomeip/src/platform/ESP32/PlatformManagerImpl.cpp:155

E (37671) chip[CSL]: PacketBuffer: pool EMPTY.
E (37671) chip[IN]: Cannot copy received pbuf of size 191
Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled.

Core 0 register dump:
PC : 0x401d313e PS : 0x00060d30 A0 : 0x8013ad78 A1 : 0x3ffdcb50
0x401d313e: chip::System::PacketBuffer::MaxDataLength() const at /Users/keanlim/connectedhomeip/examples/all-clusters-app/esp32/build/esp-idf/chip/../../../../../../config/esp32/third_party/connectedhomeip/src/system/SystemPacketBuffer.cpp:239

A2 : 0x00000000 A3 : 0x00000001 A4 : 0x3f42423c A5 : 0x3ffdcb70
A6 : 0x3ffdcb50 A7 : 0x0000000c A8 : 0x00000018 A9 : 0x3ffdc9c0
A10 : 0x3ffdcb00 A11 : 0x00000001 A12 : 0x3f42423c A13 : 0x3ffdcb70
A14 : 0x3ffdcb50 A15 : 0x3ffdcac0 SAR : 0x00000004 EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000004 LBEG : 0x400014fd LEND : 0x4000150d LCOUNT : 0xfffffff6

Backtrace:0x401d313b:0x3ffdcb500x4013ad75:0x3ffdcb70 0x40157ccf:0x3ffdcb90 0x40181071:0x3ffdcbd0 0x40184eca:0x3ffdcc20 0x40187905:0x3ffdcc60 0x4017ae7d:0x3ffdcc80 0x40095655:0x3ffdccb0
0x401d313b: chip::System::PacketBuffer::MaxDataLength() const at /Users/keanlim/connectedhomeip/examples/all-clusters-app/esp32/build/esp-idf/chip/../../../../../../config/esp32/third_party/connectedhomeip/src/system/SystemPacketBuffer.cpp:238

0x4013ad75: chip::System::PacketBuffer::SetDataLength(unsigned short, chip::System::PacketBuffer*) at /Users/keanlim/connectedhomeip/examples/all-clusters-app/esp32/build/esp-idf/chip/../../../../../../config/esp32/third_party/connectedhomeip/src/system/SystemPacketBuffer.cpp:214

0x40157ccf: chip::System::PacketBuffer::SetDataLength(unsigned short) at /Users/keanlim/connectedhomeip/examples/all-clusters-app/esp32/build/esp-idf/chip/../../../../../../config/esp32/third_party/connectedhomeip/src/system/SystemPacketBuffer.h:210
(inlined by) chip::Inet::UDPEndPointImplLwIP::LwIPReceiveUDPMessage(void*, udp_pcb*, pbuf*, ip_addr const*, unsigned short) at /Users/keanlim/connectedhomeip/examples/all-clusters-app/esp32/build/esp-idf/chip/../../../../../../config/esp32/third_party/connectedhomeip/src/inet/UDPEndPointImplLwIP.cpp:378

0x40181071: udp_input at /Users/keanlim/tools/esp-idf/components/lwip/lwip/src/core/udp.c:404

0x40184eca: ip6_input at /Users/keanlim/tools/esp-idf/components/lwip/lwip/src/core/ipv6/ip6.c:1080

0x40187905: ethernet_input at /Users/keanlim/tools/esp-idf/components/lwip/lwip/src/netif/ethernet.c:229

0x4017ae7d: tcpip_thread_handle_msg at /Users/keanlim/tools/esp-idf/components/lwip/lwip/src/api/tcpip.c:180
(inlined by) tcpip_thread at /Users/keanlim/tools/esp-idf/components/lwip/lwip/src/api/tcpip.c:154

0x40095655: vPortTaskWrapper at /Users/keanlim/tools/esp-idf/components/freertos/port/xtensa/port.c:131
M5-crashing.txt
Chiptool-log-M5-crashing.txt

[bzbarsky-apple]

From the log above:

E (37671) chip[CSL]: PacketBuffer: pool EMPTY.
E (37671) chip[IN]: Cannot copy received pbuf of size 191

Looks like fallout from #20923

[gjc13]

We cannot reproduce the issue on the latest master. I tried the commit 2e10854, the line Cannot copy received pbuf of size 191 should not show up in this version. Can we double-check the all-cluster-app and chip-tool version used in the test?

[innovation-matters-iot]

Same behaviour for me on ESP32C3:

E (972098) chip[CSL]: PacketBuffer: allocation too large.
E (972098) chip[IN]: Cannot copy received pbuf of size 1452
Guru Meditation Error: Core  0 panic'ed (Load access fault). Exception was unhandled.

Stack dump detected
Core  0 register dump:
MEPC    : 0x42056754  RA      : 0x42056772  SP      : 0x3fcadaf0  GP      : 0x3fc8e000  
0x42056754: chip::System::PacketBuffer::MaxDataLength() const at config/esp32/third_party/connectedhomeip/src/system/SystemPacketBuffer.cpp:239

[kean-apple]

I'm still seeing the crash with M5 (all cluster app) on SHA: b94e1a1. I've tried using ESP-IDF 4.4 and 4.4.2.

iOS CHIPtool was built with 2e10854.

E (5698) chip[CSL]: PacketBuffer: allocation too large.
E (5698) chip[IN]: Cannot copy received pbuf of size 1451
Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.

I was able to pair the M5 board using iOS chiptool but the M5 board crashes frequently afterwards. Seen this on multiple M5 boards.
M5-crashing-log-SHA-b94e1.txt

[kean-apple]

this was also reported earlier - #21683

[gjc13]

@kean-apple Would you try this patch and check if it fixes the C3 and M5 crash issue?

gjc13@838fe7f

[kean-apple]

Looks like the patch is helping, I have not seen this crash yet

[mrjerryjohns]

I can repro this pretty reliably on my M5. You just have to keep a subscription with a full wildcard running, and soon after, it will crash.

[dhrishi]

@mrjerryjohns Is this without or with the patch here ?

[mrjerryjohns]

Haven't had a chance to try out the patch yet...