podman DNS does not use macOS DNS

[zeha]

/kind bug

(could also be a feature I guess.)

Description

Steps to reproduce the issue:

1. Be on macOS 12.0.1, have a VPN solution (here: Cisco AnyConnect), and a Container Registry (here: GitLab) on an internal network. The registry name is not resolvable without the VPN.

2. Have podman machine setup normally, i.e.

podman machine init --cpus 2
podman machine start

3. Try logging into the registry:

podman login gitlab-registry.localdomain
Username: chofstaedtler
Password:
Error: authenticating creds for "gitlab-registry.localdomain": pinging container registry gitlab-registry.localdomain: Get "https://gitlab-registry.localdomain/v2/": dial tcp: lookup gitlab-registry.localdomain on [2a02:1748:fad4:5021::1]:53: no such host

Describe the results you received:

Describe the results you expected:

Additional information you deem important (e.g. issue happens only occasionally):

Note that 2a02:1748:fad4:5021::1 is one of the IP addresses of my LAN router. Popular VPN clients do not actually rewrite /etc/resolv.conf, especially if the VPN does not capture all the traffic.

Output of podman version:

Client:
Version:      3.4.1
API Version:  3.4.1
Go Version:   go1.17.2
Built:        Tue Oct 19 23:14:42 2021
OS/Arch:      darwin/arm64

Server:
Version:      3.4.0
API Version:  3.4.0
Go Version:   go1.16.8
Built:        Thu Sep 30 21:40:10 2021
OS/Arch:      linux/arm64

(No idea why the server is still 3.4.0, I just re-created the podman machine.)

Output of podman info --debug:

host:
  arch: arm64
  buildahVersion: 1.23.1
  cgroupControllers: []
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc34.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 2
  distribution:
    distribution: fedora
    variant: coreos
    version: "34"
  eventLogger: journald
  hostname: localhost
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.14.11-200.fc34.aarch64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 1307897856
  memTotal: 2050248704
  ociRuntime:
    name: crun
    package: crun-1.2-1.fc34.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.2
      commit: 4f6c8e0583c679bfee6a899c05ac6b916022561b
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc34.aarch64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 0
  swapTotal: 0
  uptime: 1m 34.76s
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 0
  runRoot: /run/user/1000/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 3.4.0
  Built: 1633030810
  BuiltTime: Thu Sep 30 19:40:10 2021
  GitCommit: ""
  GoVersion: go1.16.8
  OsArch: linux/arm64
  Version: 3.4.0

Package info (e.g. output of rpm -q podman or apt list podman):

$ brew info podman
podman: stable 3.4.1 (bottled), HEAD
Tool for managing OCI containers and pods
https://podman.io/
/opt/homebrew/Cellar/podman/3.4.1 (170 files, 40.3MB) *
  Poured from bottle on 2021-10-25 at 17:02:57
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/podman.rb
License: Apache-2.0
==> Dependencies
Build: go ✘, go-md2man ✘
Required: qemu ✔
==> Options
--HEAD
	Install HEAD version
==> Caveats
zsh completions have been installed to:
  /opt/homebrew/share/zsh/site-functions

$ brew info qemu
qemu: stable 6.1.0 (bottled), HEAD
Emulator for x86 and PowerPC
https://www.qemu.org/
/opt/homebrew/Cellar/qemu/6.1.0_1 (161 files, 552.2MB) *
  Poured from bottle on 2021-10-04 at 00:04:03
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/qemu.rb
License: GPL-2.0-only
==> Dependencies
Build: libtool ✔, meson ✔, ninja ✔, pkg-config ✔
Required: glib ✔, gnutls ✔, jpeg ✔, libpng ✔, libslirp ✔, libssh ✔, libusb ✔, lzo ✔, ncurses ✔, nettle ✔, pixman ✔, snappy ✔, vde ✔
==> Options
--HEAD
	Install HEAD version

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

• macOS 12.0.1, aarch64
• Cisco AnyConnect 4.10.03104

[Luap99]

@jwhonce Do you think this is fixed with #11976?

[jwhonce]

@zeha #11976 allows podman to access the mDNS resolution of the mac when running on the host. That is, podman-remote honors mDNS.

/cc @baude @ashley-cui Is there a reason to not make this change to the homebrew formula? No code change is required in podman

[ashley-cui]

/cc @baude @ashley-cui Is there a reason to not make this change to the homebrew formula? No code change is required in podman

Yep, that's pretty simple to do, I can do it and test it :)

[zeha]

I've tried to change my local homebrew formula like this, but it does not help. I guess the VM process actually makes the DNS request?

diff --git a/Formula/podman.rb b/Formula/podman.rb
index ae1358889cc..606da5759d2 100644
--- a/Formula/podman.rb
+++ b/Formula/podman.rb
@@ -36,6 +36,7 @@ class Podman < Formula

   def install
     os = OS.kernel_name.downcase
+    ENV["CGO_ENABLED"] = "1"

     system "make", "podman-remote-#{os}"
     if OS.mac?

(Or maybe this homebrew formula change did not work...)

[ashley-cui]

@zeha when you made the change to your local homebrew, did you use brew install --build-from-source?

[zeha]

@zeha when you made the change to your local homebrew, did you use brew install --build-from-source?

Yes!

/opt/homebrew/Cellar/podman/3.4.1 (170 files, 40.3MB) *
  Built from source on 2021-11-02 at 18:49:13

[Luap99]

I think you have to use the podman main branch and not 3.4.1 to make sure #11976 is used.

[zeha]

@Luap99 argh, thanks. I didn't think of passing --HEAD to brew install. So #11976 really seems to solve this for me. Now on to SSL certificate trouble, but thats another problem.

Thanks everyone!