Tailscale fails to start on HA Yellow after updating to 0.19.0

[acohn]

Problem/Motivation

A clean-installed (no restored backup, no config, not even yet linked to a tailnet) copy of this addon, version 0.19.0, fails to start on a HA Yellow.

Expected behavior

Tailscale starts normally

Actual behavior

The addon immediately stops after starting. Error messages in the addon log might point to something in the post-tailscale init script - note the netlink error: no device matches name (offset 24), netlink error: No such device, and s6-rc: warning: unable to start service post-tailscaled: command exited 92 errors.

Here's the full log, with public IP addresses replaced with RFC3849/5737:

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service base-addon-banner: starting

-----------------------------------------------------------
 Add-on: Tailscale
 Zero config VPN for building secure networks
-----------------------------------------------------------
 Add-on version: 0.19.0
 You are running the latest version of this add-on.
 System: Home Assistant OS 12.3  (aarch64 / yellow)
 Home Assistant Core: 2024.5.4
 Home Assistant Supervisor: 2024.05.1
-----------------------------------------------------------
 Please, share the above information when looking for help
 or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
s6-rc: info: service base-addon-banner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service base-addon-log-level: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service base-addon-log-level successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service init-nginx: starting
s6-rc: info: service web: starting
s6-rc: info: service tailscaled: starting
s6-rc: info: service web successfully started
s6-rc: info: service tailscaled successfully started
s6-rc: info: service post-tailscaled: starting
[02:33:02] INFO: Starting Tailscale web...
[02:33:02] INFO: Starting Tailscale...
[02:33:02] INFO: Allow UDP GRO for forwarding on 65535
netlink error: no device matches name (offset 24)
netlink error: No such device
s6-rc: warning: unable to start service post-tailscaled: command exited 92
[02:33:02] NOTICE: Tailscale logs will be suppressed after 200 lines, set add-on's configuration option 'log_level' to 'debug' to see further logs
s6-rc: info: service init-nginx successfully started
s6-rc: info: service nginx: starting
s6-rc: info: service nginx successfully started
/run/s6/basedir/scripts/rc.init: warning: s6-rc failed to properly bring all the services up! Check your logs (in /run/uncaught-logs/current if you have in-container logging) for more information.
2024/05/21 02:33:02 You have disabled logging. Tailscale will not be able to provide support.
2024/05/21 02:33:02 logtail started
2024/05/21 02:33:02 Program starting: v1.66.3-tc231c3da2-g7febb3078, Go 1.22.3: []string{"/opt/tailscaled", "--state=/data/tailscaled.state", "--statedir=/data/state", "--no-logs-no-support", "--tun=userspace-networking"}
2024/05/21 02:33:02 LogID: e30b4dddecfeb4f22d78af903341b8c33e36f05b0c01e863b04fa83e7d768ef7
2024/05/21 02:33:02 logpolicy: using system state directory "/var/lib/tailscale"
/run/s6/basedir/scripts/rc.init: fatal: stopping the container.
logpolicy.ConfigFromFile /var/lib/tailscale/tailscaled.log.conf: open /var/lib/tailscale/tailscaled.log.conf: no such file or directory
logpolicy.Config.Validate for /var/lib/tailscale/tailscaled.log.conf: config is nil
2024/05/21 02:33:02 dns: [resolved-ping=yes rc=unknown ret=direct]
2024/05/21 02:33:02 dns: using "direct" mode
2024/05/21 02:33:02 dns: using *dns.directManager
2024/05/21 02:33:02 wgengine.NewUserspaceEngine(tun "userspace-networking") ...
s6-rc: info: service nginx: stopping
s6-rc: info: service tailscaled: stopping
2024/05/21 02:33:02 dns: using dns.noopManager
2024/05/21 02:33:02 link state: interfaces.State{defaultRoute=end0 ifs={docker0:[172.30.232.1/23 llu6] end0:[192.0.2.0/24 2001:db8::1/64 2001:db8::2/64 2001:db8::3/64 llu6] hassio:[172.30.32.1/23 llu6] wpan0:[2001:db8::4/64 2001:db8::5/64 2001:db8::6/64 llu6]} v4=true v6=true}
2024/05/21 02:33:02 onPortUpdate(port=53818, network=udp6)
2024/05/21 02:33:02 onPortUpdate(port=45215, network=udp4)
2024/05/21 02:33:02 magicsock: disco key = d:66acfd54622cf879
2024/05/21 02:33:02 Creating WireGuard device...
2024/05/21 02:33:02 Bringing WireGuard device up...
2024/05/21 02:33:02 Bringing router up...
2024/05/21 02:33:02 Clearing router settings...
2024/05/21 02:33:02 Starting network monitor...
2024/05/21 02:33:02 Engine created.
2024/05/21 02:33:02 pm: migrating "_daemon" profile to new format
2024/05/21 02:33:02 envknob: TS_NO_LOGS_NO_SUPPORT="true"
2024/05/21 02:33:02 logpolicy: using system state directory "/var/lib/tailscale"
2024/05/21 02:33:02 got LocalBackend in 38ms
[02:33:02] INFO: Service NGINX exited with code 256 (by signal 15)
[02:33:02] INFO: Service tailscaled exited with code 256 (by signal 15)
s6-rc: info: service nginx successfully stopped
s6-rc: info: service init-nginx: stopping
s6-rc: info: service web: stopping
s6-rc: info: service tailscaled successfully stopped
s6-rc: info: service init-nginx successfully stopped
[02:33:02] INFO: Service Tailscale web exited with code 256 (by signal 15)
s6-rc: info: service web successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service base-addon-log-level: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service base-addon-log-level successfully stopped
s6-rc: info: service base-addon-banner: stopping
s6-rc: info: service base-addon-banner successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped

Steps to reproduce

Install this addon on a HA Yellow, and try to start it.

[lmagyar]

Gosh, what are the outputs of the below 2 commands?

ip -4 route show 0/0
ip -6 route show ::/0

[acohn]

$ ip -4 route show 0/0
default via 10.0.2.1 dev end0  src 10.0.2.27  metric 100 
$ ip -6 route show ::/0
default via fe80::1 dev end0  metric 100 
default dev wpan0  metric 65535 

This is from within the Advanced SSH & Web Terminal addon, but the output from a ssh -p 22222 [email protected] login shell is the same. Only thing that sticks out to me is the "end0" device name and the wpan0 device, but I don't know enough about the default networking config on HA OS to know if that's weird.

[lmagyar]

The "problem" is the last line, without "via". This is a default network route without a gateway, strange. Is it an OpenThread Border Router?

Hmmm, this is for enabling rx-udp-gro-forwarding on "the" default interface. This is needed for Tailscale, if I understand correctly for their UPD packages, so this setting will not make any sense on a network without a router, I will skip these.

[acohn]

Yes, there's an OpenThread Border Router running on this machine.

Thanks for looking into this so quickly. Should I disable OTBR for now?

[lmagyar]

I have zero experience with OT, so any temporary config change that removes the default dev wpan0 metric 65535 line or at least makes it non-default.

[acohn]

Just confirming that temporarily stopping the OTBR addon allows v0.19.0 of the Tailscale addon to start, which IMO is a reasonable workaround until a new version is released.

Thanks for the quick fix!

[lmagyar]

After the TS add-on started, you can start OTBR manually.

[dreem2001]

Colleagues, i face same issue on 0.19.1.