[Help]: Hostapd silently loses hairpin mode after some time

[fhteagle]

Checklist

• I acknowledge that support is provided on a best-effort basis.
• I acknowledge that the authors and contributors to this repository cannot be held responsible for the results of my use of any information contained in or linked from this repository.

uname

Linux 1756r 6.10.7-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 29 Aug 2024 16:48:57 +0000 x86_64 GNU/Linux

lsusb

...
Bus 003 Device 005: ID 148f:5572 Ralink Technology, Corp. RT5572 Wireless Adapter
Bus 003 Device 006: ID 0e8d:0608 MediaTek Inc. Wireless_Device 
Bus 003 Device 008: ID 0e8d:7961 MediaTek Inc. Wireless_Device
...

rfkill

0: phy0: Wireless LAN 	Soft blocked: no 	Hard blocked: no 
1: phy2: Wireless LAN 	Soft blocked: no 	Hard blocked: no 
2: phy1: Wireless LAN 	Soft blocked: no 	Hard blocked: no

dkms

Error! Could not locate dkms.conf file.
File: /var/lib/dkms/r8152/2.17.1.20230903/source/dkms.conf does not exist.

Probably due to not rebooting after my last kernel upgrade, and pacman-keep-modules hook wasn't firing apparently. Oops.

iw

text
phy#1
	Interface cfWLAN
		ifindex 8
		wdev 0x100000001
		addr e0:e1:a9:36:54:XX
		ssid censored-ssid-1
		type AP
		channel 1 (2412 MHz), width: 40 MHz, center1: 2422 MHz
		txpower 3.00 dBm
		multicast TXQ:
			qsz-byt	qsz-pkt	flows	drops	marks	overlmt	hashcol	tx-bytes	tx-packets
			0	0	934687	12	0	0	1289	94948478		946378
phy#2
	Interface pandaWLAN
		ifindex 6
		wdev 0x200000001
		addr 7c:dd:90:86:9d:XX
		ssid censored-ssid-2
		type AP
		channel 11 (2462 MHz), width: 20 MHz, center1: 2462 MHz
		txpower 30.00 dBm
		multicast TXQ:
			qsz-byt	qsz-pkt	flows	drops	marks	overlmt	hashcol	tx-bytes	tx-packets
			0	0	81168	0	0	0	10	5244210		81187
phy#0
	Interface mt7921WLAN
		ifindex 4
		wdev 0x1
		addr a8:93:4a:e1:e9:XX
		ssid censored-ssid-1
		type AP
		channel 149 (5745 MHz), width: 80 MHz, center1: 5775 MHz
		txpower 3.00 dBm
		multicast TXQ:
			qsz-byt	qsz-pkt	flows	drops	marks	overlmt	hashcol	tx-bytes	tx-packets
			0	0	559098	0	0	0	159	60588570		569181

What happened?

Using bleeding edge hostapd compiled from w1.fi git, mediatek firmware modules from arch package linux-firmware, currently on version 20240809 :

Without hairpin mode on, visibility between STAs is compromised, from mdns to most especially KDEConnect that extensively utilizes UDP multicast to do its thing. Therefore, I have set hairpin mode to on for each of my 3 APs attached to my main router, all bridged together as br0, in order to pass packets between STAs connected to these three bridged APs. I have of course bridge=br0 set, and also used the ap_isolate=1 and bridge_hairpin=1 hostapd configuration lines to accomplish this, based on Steps 1 and 2 of StackExchange Q/A in each of my per-interface hostapd.conf files. Step 3 from that Q/A appears to not actually be needed due to the way I have firewalld configured.

After I restart each of my custom systemd services that launches / monitors one AP hostapd instance per service, hairpin functionality seems to be restored immediately. However, after some amount of uptime (12+ hours maybe), hairpin mode silently fails. Nothing in dmesg or journalctl that corresponds to the loss of hairpin functionality that I can see. Though I do have these blocks repeated intermittently / randomly, usually in groups of 2-5 repeats of the block:

[377168.719115] mt7921u 3-6.4:1.3 cfWLAN: left promiscuous mode
[377168.974753] mt7921u 3-6.4:1.3 cfWLAN: entered allmulticast mode
[377168.974863] mt7921u 3-6.4:1.3 cfWLAN: entered promiscuous mode

As far as I can tell, there is NOT a way for me to catch loss of hairpin programatically on the router itself and restart the hostapd systemd services automatically. I.e., pings from the router to STAs proceed fine with hairpin mode off. Any ideas for this would be welcome.

I do not see somewhere to report bugs against hostapd itself, either, otherwise I would go straight to the source on that. But this "forum" has some very knowledgeable people that have helped with other issues, so I am asking here first. Is anyone else seeing this issue? Any mitigation? Plan for a real fix?

Thanks

[fhteagle]

After doing a ton of troubleshooting and watching tshark outputs until my eyes bleed, I think I have come up with a root cause problem for this. Docker is the Veruca Salt (spoiled brat) of the networking world, and was adding iptables rules for bridge interfaces it did not create / own / control. Forcing "iptables": false, in /etc/docker/daemon.json has improved the function of kdeconnect and mdns significantly. This has broken a few other things about my docker setup, but that's not a usb-wifi problem anymore.

Of course I will keep monitoring for new information, but I am comfortable closing this for the moment.

[morrownr]

@fhteagle

Glad you seem to be headed in the right direction. I could not think of anything and sometimes it is best to leave issues unanswered as it can cause others to skip taking a look. This shit can be complicated.