ipv4 packet not forwarded between SLIP and WIFI STA interfaces. (IDFGH-5517)

[clementfumey-inventhys]

Environment

• Module or chip used: ESP32-WROOM-32
• IDF version : v4.3-194-g5b5e46971
• Build System : CMake or idf.py
• Compiler version : xtensa-esp32-elf-gcc (crosstool-NG esp-2021r1) 8.4.0
• Operating System: Linux
• Using an IDE?: Yes (Visual Studio with esp-idf extension) but same problem without and with idf.py
• Power Supply: USB

Problem Description

Goal: I am trying to give access to internet to a node (computer or esp32 for example) through an ESP32 that act as router through a SLIP connection and MESH STA.

My current work is based on esp-idf branch v4.3

The idea is to an ESP32 that is connected using UART through a SLIP interface with static IP addresses (slip-modem example in esp-idf) to an computer (or an another ESP32). The ESP32 is also connected to internet throught its Wifi Sta.

On the node, IP_FORWARDING and NAPT is enable.
On the node, the incoming packet on the SLIP interface is not forwarded to the wifi interface. It seems lwip do not know what is the default gw for ip addresses he doesn't know.

I have tested a similar configuration with ESP32 acting Wifi AP and forwarding Wifi traffic to a SLIP interface and it works.

Expected Behavior

lwip should forward ip packets from slip interface to esp-mesh wifi interface

Actual Behavior

I (57890) esp-slip_modem: 61 36 04 74 0d 9f f3 e0 8f f6 9b 0a 2c 4b c6 1c 
I (57900) esp-slip_modem: 01 02 0b 06 7a 54 cf 7e ae a4 2b 11 5b e8 78 8c 
I (57910) esp-slip_modem: 63 0a 04 99 5b 2a 94 b7 52 48 b5 21 85 68 fb 75 
I (57920) esp-slip_modem: c0 
ip_input: iphdr->dest 0xffb37f03 netif->ip_addr 0x10a010a (0x0, 0x0, 0xffb37f03)
ip_input: iphdr->dest 0xffb37f03 netif->ip_addr 0xe42ba8c0 (0xb37f03, 0x2ba8c0, 0xff000000)
ip_input: iphdr->dest 0xffb37f03 netif->ip_addr 0x100007f (0x3, 0x7f, 0xffb37f00)
ip4_input: packet not for us.
ip4_forward: not bouncing packets back on incoming interface.
IP packet dropped due to bad version number 11
IP header:
+-------------------------------+
|11 | 3 |  0xff |     43996     | (v, hl, tos, len)
+-------------------------------+
|      443      |010|    7762   | (id, flags, offset)
+-------------------------------+
|  200  |  147  |    0xb9da     | (ttl, proto, chksum)
+-------------------------------+
|   42  |  130  |  128  |   24  | (src)
+-------------------------------+
|   14  |   77  |  197  |  254  | (dest)
+-------------------------------+
ip_input: iphdr->dest 0x3415f09d netif->ip_addr 0x10a010a (0x0, 0x0, 0x3415f09d)
ip_input: iphdr->dest 0x3415f09d netif->ip_addr 0xe42ba8c0 (0x15f09d, 0x2ba8c0, 0x34000000)
ip_input: iphdr->dest 0x3415f09d netif->ip_addr 0x100007f (0x9d, 0x7f, 0x3415f000)
ip4_input: packet not for us.
ip4_forward: not bouncing packets back on incoming interface.
I (58030) esp-slip_modem: rx 1024 bytes
ip_input: iphdr->dest 0xffb37f03 netif->ip_addr 0x10a010a (0x0, 0x0, 0xffb37f03)
ip_input: iphdr->dest 0xffb37f03 netif->ip_addr 0xe42ba8c0 (0xb37f03, 0x2ba8c0, 0xff000000)
ip_input: iphdr->dest 0xffb37f03 netif->ip_addr 0x100007f (0x3, 0x7f, 0xffb37f00)
I (58040) esp-slip_modem: 74 3a 20 64 65 74 65 63 74 70 6f 72 74 61 6c 2e 
ip4_input: packet not for us.
ip4_forward: not bouncing packets back on incoming interface.
I (58070) esp-slip_modem: 66 69 72 65 66 6f 78 2e 63 6f 6d 0d 0a 55 73 65 
I (58080) esp-slip_modem: 72 2d 41 67 65 6e 74 3a 20 4d 6f 7a 69 6c 6c 61 

Code to reproduce this issue

slip-modem example with this main : https://gist.github.com/clementfumey-inventhys/d19b30f109f3bcaf6e93b22e86e96624
CONFIG_LWIP_L2_TO_L3_COPY, CONFIG_LWIP_IP_FORWARD and CONFIG_LWIP_IPV4_NAPT should be enable in sdkconfig

[david-cermak]

@clementfumey-inventhys Do you really enable the napt for that interface? Didn't see a call to napt_enable() in the referenced gist. (i'd expect something similar to https://github.com/espressif/esp-idf/blob/master/examples/mesh/ip_internal_network/main/mesh_netif.c#L390 in the ip mesh example)

[clementfumey-inventhys]

You are right, I forgot it on both STA interface and SLIP. But I still have the same problem, when for example pinging from my computer (through the SLIP interface) to the wifi router. on wich the ESP32 STA is connected.
The "final goal" is to give access to internet to a Wifi node (smartphone for example) through an ESP-MESH network. With two "twins" esp32 node connected with SLIP, one on the esp-mesh and one as Wifi AP for the smartphone. So I'm taking it step by step to make sure every part works. If you have any advice of how-to do it I will greatly appreciate. I have difficulties to understand how the routing table is build and used.

Anyway this seems not related to any bug or issue in esp-idf, but more of my lack of knowledge in lwip routing. I will thus close the issue.

Sorry for the inconvenience.

[david-cermak]

TBH, the SLIP interface is a little different from standard netifs of lwip (it is very simple and uses predefined addresses) so I'm not sure if the address translation would work here (I've never tried). Alternatively you can use PPP netif, as the setup pppos-client + Wifi-AP works correctly. The only trouble might be the server part, as it's not supported in IDF Kconfig and needs to be enabled directly in lwip.

[clementfumey-inventhys]

Here is an update of the case, if anyone is interested in SLIP/STA forwarding :

To make IP Forwarding worked with SLIP, I added a netmask and a gateway to the SLIP interface.

static const esp_netif_ip_info_t s_slip_ip4 = {
        .ip = { .addr = ESP_IP4TOADDR( 10, 1, 10, 1) },
        .gw = { .addr = ESP_IP4TOADDR( 10, 1, 10, 1) },
        .netmask = { .addr = ESP_IP4TOADDR( 255, 255, 255, 0) },
};

It get rid of the previous log message ip4_forward: not bouncing packets back on incoming interface. because SLIP interface with netmask 0.0.0.0 always trigger the route search on SLIP.

Next problem is that it correctly forward my ICMP ping request from the SLIP to Wifi but on the ICMP ping reply from the Wifi router, it never enter the NAPT receive function and never forward back to the SLIP. Here is the lines of code I do not understands in ip4.c.

#if ESP_LWIP
#if IP_NAPT
  /* for unicast packet, check NAPT table and modify dest if needed */
  if (!inp->napt && ip4_addr_cmp(&iphdr->dest, netif_ip4_addr(inp)))
    ip_napt_recv(p, iphdr);
#endif
#endif /* ESP_LWIP */

Why !inp->napt ??? If I replace it with inp->napt wich make sense to my eyes, it works and the ICMP reply is correctly forwarded back. But It so obvious that I might be missing something !?

[martin-ger]

The interface with inp->napt set is the interface that is "hidden" behind the NAPT. Outgoing packets here are mapped in ip_forward(). This pice of code handles packet incoming packets on other interfaces (no inp->napt) , that are checked here, whether they have to be rewritten to be forwarded back into "hidden" network.

[clementfumey-inventhys]

Thank you @martin-ger , I definitely misunderstood this napt part. My mistake was to enable NAPT on the wrong interfaces.

@david-cermak I found a small bug in esp-netif that prevent the SLIP interface to be set as default. On the esp32 with Wifi AP and SLIP interface, the default interface should be SLIP (prio 16 > prio 10). Here is the snippet I use to fix it in esp-idf/components/esp_netif/lwip/esp_netif_lwip.c :

static void esp_netif_set_default_netif(esp_netif_t *esp_netif)
{
    if (_IS_NETIF_POINT2POINT_TYPE(esp_netif, PPP_LWIP_NETIF)) {
#if CONFIG_PPP_SUPPORT
        esp_netif_ppp_set_default_netif(esp_netif->netif_handle);
#endif
    }else if (_IS_NETIF_POINT2POINT_TYPE(esp_netif, SLIP_LWIP_NETIF)) {
        netif_set_default(esp_netif->lwip_netif);
    }else {
        netif_set_default(esp_netif->netif_handle);
    }
}

I just added the SLIP_LWIP_NETIF condition where the lwip netif is not store in netif_handle like the default case for wifi but in lwip_netif.

Now I am able to have an ESP-MESH with internal ip adressing, connected to an internet router, where some nodes are connected to an other esp32 via SLIP. The other node provide an AP to a smartphone or computer to access internet through the Mesh. It is very slow for now but there is place for improvements.

[david-cermak]

@clementfumey-inventhys Thanks for pointing out this bug and suggesting the fix!

I would prefer using the lwip_netif for setting the default interface for both the vanilla lwip and the slip interface, since it's a typed value and always holds a struct netif* pointer:

--- a/components/esp_netif/lwip/esp_netif_lwip.c
+++ b/components/esp_netif/lwip/esp_netif_lwip.c
@@ -178,7 +178,7 @@ static void esp_netif_set_default_netif(esp_netif_t *esp_netif)
         esp_netif_ppp_set_default_netif(esp_netif->netif_handle);
 #endif
     } else {
-        netif_set_default(esp_netif->netif_handle);
+        netif_set_default(esp_netif->lwip_netif);
     }
 }

Could we close this issue with this esp-netif bugfix, or is there anything else that doesn't work as expected?