Is there a way to run lima machines on an isolated network

[jaepetto]

We are currently testing DHCP and PXE servers for our infrastructure. For the time being I am using qemu and vde_switch to create the infrastructure.

Typically, I do have a pfsense vm with two links. The WAN link is connected to a user network and the LAN link is connected to the vde switch. Pfsense is nice to use because it has a DHCP server than can be configured to send a next server in the DHCP offer and it includes tcpdump directly.

Then I have my TFTP server running on a VM connected only to the vde switch. The last machine is a simple VM trying to boot up.

This setup works pretty well aside from some slowness related to the use of x86_64 architecture on a M1 mac.

I would like to move this infrastructure to lima. Unfortunately, I could not find any reference on how to create an isolated network (a custom switch).

Does anyone know if it's feasible?

[jandubois]

Please check https://github.com/lima-vm/lima/blob/master/docs/network.md#managed-vmnet-networks-192168105024 for more information on how to create virtual networks between Lima VMs.

Look on slide 22 from our talk for a visualization of how networking is configured: https://static.sched.com/hosted_files/kccnceu2022/5f/lima.pdf.

[jaepetto]

@jandubois Thank you very much for the resources. The only point I'm not really clear about is the way to declare a new network in the $LIMA_HOME/_config/networks.yaml.

Would you happen to have an example?

[jandubois]

Just copy any of the 3 pre-defined networks (host, bridged, shared) and give them a different name (and subnet, for non-bridged networks).

These 3 networks are just predefined, but not special in any other way. You can define more, but there is a limit (from Apple's VMNET framework) on how many you can actually instantiate at the same time at runtime.

[jaepetto]

I just tried to create an isolated network (without any configuration - which is my intent):

paths:
  vdeSwitch: /opt/homebrew/Cellar/vde/2.3.3/bin/vde_switch
  vdeVMNet: /opt/vde/bin/vde_vmnet
  varRun: /private/var/run/lima
  sudoers: /private/etc/sudoers.d/lima

group: everyone

networks:
  shared:
    mode: shared
    gateway: 192.168.105.1
    dhcpEnd: 192.168.105.254
    netmask: 255.255.255.0
  bridged:
    mode: bridged
    interface: en0
    # bridged mode doesn't have a gateway; dhcp is managed by outside network
  host:
    mode: host
    gateway: 192.168.106.1
    dhcpEnd: 192.168.106.254
    netmask: 255.255.255.0
  isolated:

After failing because of sudoers rights (which was fixed as per documentation with limactl sudoers | sudo tee /etc/sudoers.d/lima), it is now failing with the below error message:
FATA[0000] networks.yaml field paths.vdeSwitch error: file "/opt/homebrew/Cellar/vde/2.3.3/bin/vde_switch" is not owned by "root" (uid: 0), but by uid 501

This is too in the documentation. Unfortunately, it is kind of hard to change the rights because vde_switch was installed by brew and I don't want to mess the rights in the folders belonging to it.