DHCP not working on Fedora rawhide

[AlanGriffiths]

Describe the bug
Trying to package a snap using snapcraft on Fedora rawhide. Initially this failed with a permission error on /var/snap/multipass/common/multipass_socket

$ ls -lh /var/snap/multipass/common/multipass_socket
srw-rw----. 1 root adm 0 Mar 26 12:17 /var/snap/multipass/common/multipass_socket

After adding myself to adm I then get:

Projects/egmde-snap/
[alan@localhost egmde-snap]$ snapcraft
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
Launching a VM.
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
start failed: The following errors occurred:                                    
snapcraft-egmde: timed out waiting for response

Logs

Mar 26 12:17:27 localhost.localdomain dnsmasq[4414]: using local addresses only for domain multipass
Mar 26 12:17:27 localhost.localdomain dnsmasq[4414]: using nameserver 192.168.1.1#53
Mar 26 12:17:27 localhost.localdomain dnsmasq[4414]: read /etc/hosts - 2 addresses
Mar 26 12:17:27 localhost.localdomain dnsmasq[4414]: cannot read /var/snap/multipass/common/data/multipassd/network/dnsmasq.hosts: No such file or directory
Mar 26 12:17:28 localhost.localdomain multipassd[4303]: gRPC listening on unix:/var/snap/multipass/common/multipass_socket, SSL:on
Mar 26 12:19:44 localhost.localdomain multipassd[4303]: process working dir '/var/lib/snapd/snap/multipass/1784/qemu'
Mar 26 12:19:44 localhost.localdomain multipassd[4303]: process program 'qemu-system-x86_64'
Mar 26 12:19:44 localhost.localdomain multipassd[4303]: process arguments '--enable-kvm, -device, virtio-scsi-pci,id=scsi0, -drive, file=/var/snap/multipass/common/data/multipassd/vault/instances/s>
Mar 26 12:19:44 localhost.localdomain multipassd[4303]: Resuming from a suspended state
Mar 26 12:19:44 localhost.localdomain multipassd[4303]: process started
Mar 26 12:19:45 localhost.localdomain multipassd[4303]: QMP: {"QMP": {"version": {"qemu": {"micro": 1, "minor": 11, "major": 2}, "package": "(Debian 1:2.11+dfsg-1ubuntu7.23)"}, "capabilities": []}}
Mar 26 12:19:45 localhost.localdomain multipassd[4303]: QMP: {"return": {}}
Mar 26 12:20:18 localhost.localdomain multipassd[4303]: No mounts to stop for instance "snapcraft-egmde"
Mar 26 12:20:18 localhost.localdomain multipassd[4303]: error: program: qemu-system-x86_64; error: Process crashed
Mar 26 12:20:18 localhost.localdomain multipassd[4303]: attempting to release non-existant addr: 52:54:00:73:7d:aa
Mar 26 12:20:34 localhost.localdomain multipassd[4303]: process working dir '/var/lib/snapd/snap/multipass/1784/qemu'
Mar 26 12:20:34 localhost.localdomain multipassd[4303]: process program 'qemu-system-x86_64'
Mar 26 12:20:34 localhost.localdomain multipassd[4303]: process arguments '--enable-kvm, -device, virtio-scsi-pci,id=scsi0, -drive, file=/var/snap/multipass/common/data/multipassd/vault/instances/s>
Mar 26 12:20:34 localhost.localdomain multipassd[4303]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
Mar 26 12:20:34 localhost.localdomain multipassd[4303]: process started
Mar 26 12:20:34 localhost.localdomain multipassd[4303]: QMP: {"QMP": {"version": {"qemu": {"micro": 1, "minor": 11, "major": 2}, "package": "(Debian 1:2.11+dfsg-1ubuntu7.23)"}, "capabilities": []}}
Mar 26 12:20:34 localhost.localdomain multipassd[4303]: QMP: {"return": {}}
Mar 26 12:20:38 localhost.localdomain multipassd[4303]: QMP: {"timestamp": {"seconds": 1585225238, "microseconds": 599403}, "event": "NIC_RX_FILTER_CHANGED", "data": {"name": "net0", "path": "/mach>

[Saviq]

Hmm I was unable to reproduce…

1. I took a Fedora 31 cloud image
2. launched it in Multipass - I was already in the adm group
3. upgraded to Rawhide
4. enabled snapd (including classic)
5. installed Multipass inside
6. and multipass shell worked just fine…

Maybe it was a temporary problem? Can you reproduce still?

[AlanGriffiths]

Same system (installed from the rawhide daily a couple of weeks ago, and updated since):

[alan@localhost egmde-snap]$ snapcraft
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
Launching a VM.
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
start failed: The following errors occurred:                                    
snapcraft-egmde: timed out waiting for response
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
An error occurred with the instance when trying to start with 'multipass': returned exit code 2.
Ensure that 'multipass' is setup correctly and try again.
[alan@localhost egmde-snap]$ multipass list
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
Name                    State             IPv4             Image
snapcraft-egmde         Delayed Shutdown  UNKNOWN          Ubuntu Snapcraft builder for Core 18

[Saviq]

OK reproduced. The instance isn't getting an IP for some reason.

[Saviq]

LXD has the same problem: lxc/lxd#7150, it's firewalld blocking the DHCP requests.

Disabling firewalld is a workaround: sudo systemctl stop firewalld.

[Saviq]

The workaround:

firewall-cmd --zone=trusted --change-interface=mpqemubr0 --permanent

[AlanGriffiths]

firewall-cmd --zone=trusted --change-interface=mpqemubr0 --permanent

Until multipass gets confined, that could be added to the install hook:

if [ "$(sed -Ene 's/^ID=(.*)/\1/p' /etc/os-release)" == "fedora" ]
then
  firewall-cmd --zone=trusted --change-interface=mpqemubr0 --permanent
fi

😉

[Saviq]

I think we'll have to document that instead.

[AlanGriffiths]

Doesn't work for me

[abitrolly]

I think we'll have to document that instead.

LXD is also broken on Fedora 32.

✗ lxc launch ubuntu:18.04 xxx      
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
Creating xxx
Starting xxx
✗ lxc exec xxx -- bash       
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
root@xxx:~# ping google.com
ping: google.com: Temporary failure in name resolution
root@xxx:~# 

Why podman and docker containers don't have such problems on Fedora?

[danielbisar]

I had the same issue trying out snapcraft on Fedora 33 for the first time. I did two things, not sure if the first is necessary:

1. snap connect multipass:libvirt
2. firewall-cmd --zone=trusted --change-interface=mpqemubr0 --permanent
3. sudo systemctl restart firewalld

Now

> multipass list WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement Name State IPv4 Image snapcraft-my-first-snap Delayed Shutdown 10.137.247.131 Ubuntu Snapcraft builder for Core 18

So at least I have an IP, but still the connection inside the VM does not work. - If I find out more, will let you know.

[JmaJeremy]

I'm having this issue on Debian using UFW. Any suggestion what the equivalent rule would be for UFW to work without having to disable it?

[JmaJeremy]

OK I answered my own question, in case any other UFW users are having this issue, the following rule worked for me:
sudo ufw allow in on mpqemubr0 to any

[yan-foto]

@JmaJeremy I also came across the same problem as you and ended with having my snaps built remotely (snapcraft remote-build). It's not a solution, not even a work around and to make it worse, it's even super slow but it works!

[proninyaroslav]

So at least I have an IP, but still the connection inside the VM does not work. - If I find out more, will let you know.

Yes. Even disabling firewalld didn't help.

[abitrolly]

I've run into this again. What is needed to make multipass work correctly on Fedora out of the box?

[townsend2010]

Hey @abitrolly,

I commented at #2564 (comment) about the challenges of automatically making Multipass work with firewalld.

Thanks!