Add DNS server to host-agent to use native host resolver

[jandubois]

This is is required to correctly resolve hostnames while using conditional forwarding (split-DNS) when connected to a VPN.

The hostagent must be compiled with CGO_ENABLED=1 to use the native resolver.

The DNS server is listening for UDP connections on the same port number used for SSH:

jan@lima-default:~$ dig @192.168.5.2 -p 60022 google.com
; <<>> DiG 9.16.8-Ubuntu <<>> @192.168.5.2 -p 60022 google.com
[...]
;; ANSWER SECTION:
google.com.		0	IN	A	172.217.14.206

It is then forwarded via iptables to 192.168.5.3:53, replacing the DNS supplied by QEMU.

Fixes #270
See also rancher-sandbox/rancher-desktop#702

[jandubois]

I have not been able to redirect this to port 53 on a loopback IP, to put it into /etc/resolve.conf. My iptables-fu is weak; I tried this (but it didn't work):

root@lima-default:~# iptables --flush
root@lima-default:~# iptables -A FORWARD -d 192.168.5.2 -i eth0 -p udp -m udp --dport 8053 -j ACCEPT
root@lima-default:~# iptables -t nat -A PREROUTING -d 192.168.5.2 -p udp -m udp --dport 8053 -j DNAT --to-destination 127.0.53.53:53
root@lima-default:~# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
root@lima-default:~# iptables -vnxL
# Warning: iptables-legacy tables present, use iptables-legacy to see them
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination
       0        0 ACCEPT     udp  --  eth0   *       0.0.0.0/0            192.168.5.2          udp dpt:8053

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination
root@lima-default:~# dig @127.0.53.53 google.com

; <<>> DiG 9.16.8-Ubuntu <<>> @127.0.53.53 google.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

Help wanted!

Also cross-compiling for Linux with CGO_ENABLED=1 throws this error:

GOOS=linux GOARCH=amd64 make clean binaries
rm -rf _output
mkdir -p _output/bin
cp -a ./cmd/lima _output/bin/lima
CGO_ENABLED=1 go build -ldflags="-s -w -X github.com/lima-vm/lima/pkg/version.Version=v0.6.4-61-gf56f620.m" -o _output/bin/limactl ./cmd/limactl
# runtime/cgo
linux_syscall.c:67:13: error: implicit declaration of function 'setresgid' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
linux_syscall.c:67:13: note: did you mean 'setregid'?
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/unistd.h:593:6: note: 'setregid' declared here
linux_syscall.c:73:13: error: implicit declaration of function 'setresuid' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
linux_syscall.c:73:13: note: did you mean 'setreuid'?
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/unistd.h:595:6: note: 'setreuid' declared here

[jandubois]

I forgot to mention that Ubuntu also needs this:

sysctl -w net.ipv4.ip_forward=1

Still didn't work though...

[jandubois]

I believe the only outstanding issue with this PR is the cross-compilation problem with CGO_ENABLED=1.

It looks like cross-compilation for arm64 on amd64 works as long as GOOS for host and target are the same, so I'm planning to look into using separate builders for the Linux and macOS artifacts on Github actions.

But otherwise I'm pretty pleased with how this turned out. When useHostResolver is true, the QEMU DNS is transparently replaced with the hostagent DNS as soon as iptables is installed, and all names are resolved on the host from then on.

[AkihiroSuda]

It looks like cross-compilation for arm64 on amd64 works as long as GOOS for host and target are the same, so I'm planning to look into using separate builders for the Linux and macOS artifacts on Github actions.

SGTM.

actions/upload-artifact@v2 and actions/download-artifact@v2 can be used for copying files from the macOS job to the Linux job.

https://github.com/rootless-containers/usernetes/blob/f1845d2048de2a83aada679d1d294bf22ceb747d/.github/workflows/main.yaml#L37-L41

[AkihiroSuda]

BTW if we can just fix libslirp to use the sane resolver, that will be helpful for non-Lima QEMU users and we can avoid CGO.

https://gitlab.freedesktop.org/slirp/libslirp/-/blob/v4.6.1/src/slirp.c#L244

[jandubois]

BTW if we can just fix libslirp to use the sane resolver, that will be helpful for non-Lima QEMU users and we can avoid CGO.
https://gitlab.freedesktop.org/slirp/libslirp/-/blob/v4.6.1/src/slirp.c#L244

I'm afraid that is not as easy as it sounds. Afaict libslirp doesn't implement a DNS server; it simply looks up nameserver addresses, picks one (at random?), and then forwards the traffic to that server. And since it is just forwarding packets, it can't even fail over to another server if the chosen one fails.

The line you linked to gets the nameserver address from /etc/resolv.conf and is only used on Linux. The macOS/iOS code is using libresolv, but the end effect is the same: it picks an address and forwards all DNS traffic to that address:

https://gitlab.freedesktop.org/slirp/libslirp/-/blob/v4.6.1/src/slirp.c#L138

Afaik the local resolver on macOS is not exposed via DNS, so you would have to include a DNS server & client implementation inside libslirp, and we obviously couldn't use the Go library for that.

So while this would be great to have, I'm not volunteering to work on it, given that it seems to be a rather huge effort.

With systemd-resolved you get this out of the box: you configure the resolver, and it then exposes the resolver via DNS again at 127.0.0.53:53. But we don't have this for macOS.

For Lima I think this PR provides all the functionality we need (famous last words alert).

You can even add names to /etc/hosts on the host, and have them immediately resolve inside the guest, or inside containers without having to restart anything.

[jandubois]

@AkihiroSuda I think the PR is now ready for final review. I will test the darwin-aarch64 binaries tomorrow; but don't have real hardware to test the linux-arm64 binaries right now.

If you think everything looks fine, maybe you could create a test-action-release-* tag, so we can do validation with the CI builds, to make sure the cross-compiled bits actually work?

[jandubois]

I've tested the CGO_ENABLED=1 build cross-compiled for M1 and it works fine (entry has A record, but no AAAA or MX record):

jan@m1 ~ % echo "1.2.3.4 foobar.example.com" | sudo tee -a /etc/hosts
Password:
1.2.3.4 foobar.example.com
jan@m1 ~ % lima host foobar.example.com
foobar.example.com has address 1.2.3.4
Host foobar.example.com not found: 3(NXDOMAIN)
Host foobar.example.com not found: 3(NXDOMAIN)

So I think this is ready for merging 😄

[AkihiroSuda]

Thanks