Inherit or specify DNS configuration

[WarriorXK]

Hi there,

Since we have a lot of internal services (Private gitlab, internal packages service, etc) that we need access to from our CI jobs we need the tart VM to be configured to use a specific DNS server, currently we have our own custom implementation to start tart VMs for gitlab CI but we'd like to switch to this package instead. In order to do that we need to configure the DNS server before cloning.

To do that we currently run networksetup -setdnsservers Ethernet $HOST_DNS in the prepare stage inside the tart VM where $HOST_DNS is the DNS server configured on the physical Mac read using HOST_DNS="$(scutil --dns | grep nameserver | head -n 1 | awk '{print $3}')".

I would suggest a --dns-server option to gitlab-tart-executor prepare allowing the value inherit to inherit the config from the host, or an IP address to allow for a custom value.

[fkorotkov]

Hey @WarriorXK, do you happen to use a Tart VM that has DNS configured? For example, our images were setting up DNS which override the host settings.

In case networksetup -getdnsservers Ethernet returns something for your image, you can run sudo networksetup -setdnsservers Ethernet "Empty" to reset it in your pre_get_sources_script. This way guest will be using DNS settings of the host.

[PaulWoitaschek]

@fkorotkov

Is there some special thing we need to do in order to let the VM's use the host network?

We have a hosted mac which is connected to a VPN.
In the regular shell runner, there is no issue with connecting to the vpn but with the tart runner this doesn't work

Running with gitlab-runner 15.10.1 (dcfb4b66)
  on m2-tart2 h-KdyTkV, system ID: s_1543d887871b
  feature flags: FF_USE_FASTZIP:true, FF_RESOLVE_FULL_TLS_CHAIN:false
Resolving secrets 00:00
Preparing the "custom" executor 06:41
Using Custom executor...
2024/01/17 08:51:11 Pulling the latest version of ghcr.io/cirruslabs/macos-ventura-base:latest...
2024/01/17 08:57:39 Cloning and configuring a new VM...
2024/01/17 08:57:39 Waiting for the VM to boot and be SSH-able...
2024/01/17 08:57:51 Was able to SSH!
2024/01/17 08:57:51 VM is ready.
Preparing environment 00:01
Running on admins-Virtual-Machine.local...
Getting source from Git repository 01:15
Fetching changes with git depth set to 20...
Initialized empty Git repository in /private/tmp/builds/apps/.git/
Created fresh repository.
fatal: unable to access 'https:/redacted.com/apps.git/': Failed to connect to redacted.com port 443 after 75029 ms: Couldn't connect to server
2024/01/17 08:59:07 Process exited with status 128
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit status 1

[edigaryev]

Hello @PaulWoitaschek,

Consider using a VPN solution with Network Extension support, for example, WireGuard. If you're using OpenVPN, macOS networking machinery won't be able to tell that it is a VPN and configure the proper packet filter rules.

See cirruslabs/tart#278 (comment) for more details.

[fkorotkov]

I'm also not sure if Ventura images have this change cirruslabs/macos-image-templates#87 which removes DNS override within the VM. You can also try the latest Sonoma images even if the host is still Ventura.

EDIT: seems redacted.com resolves so you don't have this issue.

[PaulWoitaschek]

Thank you! Switching our companies VPN to WireGuard is unfortunatelly not in my power.

I don't understand the comments regarding orchard. Does this mean that running tart in an OpenVPN enviornment is not possible?

[edigaryev]

I don't understand the comments regarding orchard. Does this mean that running tart in an OpenVPN enviornment is not possible?

It just notes that with Orchard, it's easier to connect to the VMs behind the NAT, be it SSH or any other port on the VM and this VPN might not be needed at all.

As for the OpenVPN, Tart VMs (and any other Virtualization.Framework-based solutions) won't get a proper routing to the VPN's subnet if such VPN isn't built on top of the Network Extension.

[PaulWoitaschek]

So the only option I have is to let each job install openvpn, and pass the credentials as env variables?

[edigaryev]

So the only option I have is to let each job install openvpn, and pass the credentials as env variables?

You might also have a luck toying with pfctl, see pfctl -s nat -a "com.apple.internet-sharing/shared_v4" when the VM is started. It adds NAT rules for the normal network interfaces and Network Extension-based VPNs, but not for utun*-based VPNs like OpenVPN.

We've considered doing this automatically in Tart, but running pfctl requires root and it is pretty hacky because it might break at any moment.

[PaulWoitaschek]

So I'd run pfctl -s nat -a "com.apple.internet-sharing/shared_v4" in a pre_get_sources_script?

[fkorotkov]

@PaulWoitaschek unfortunately, this is a question beyond general support and the answer will be most likely specific to your use case. We'll need to specifically work with you, try to reproduce your use case and find a solution. Due to resource limitation on our end we only do so for folks on Priority Support.

As @edigaryev mentioned, please take a look at pfctl -s nat -a "com.apple.internet-sharing/shared_v4 output before and after you run a Tart VM. Then you'll need to figure out your OpenVPN networking situation on the host and manually add rules for OpenVPN for proper routing.