Support '--add-host=host.docker.internal:host-gateway'

[djmattyg007]

/kind feature

Description

The next version of docker will support the following CLI argument to docker run:

--add-host=host.docker.internal:host-gateway

The magic string host-gateway will automatically map to the gateway inside the container (ie. the host machine), allowing you to provide a well-known hostname to use to access the host from inside a container.

This feature was discussed and implemented on the following pull request:

moby/moby#40007

As far as I can tell, the latest version of podman does not support this yet.

Output of podman version:

Version:      2.1.1
API Version:  2.0.0
Go Version:   go1.15.2
Git Commit:   9f6d6ba0b314d86521b66183c9ce48eaa2da1de2
Built:        Sun Sep 27 01:50:37 2020
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.16.1
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: Unknown
    path: /usr/bin/conmon
    version: 'conmon version 2.0.21, commit: 35a2fa83022e56e18af7e6a865ba5d7165fa2a4a'
  cpus: 8
  distribution:
    distribution: arch
    version: unknown
  eventLogger: journald
  hostname: shacklebolt
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.9.4-arch1-1
  linkmode: dynamic
  memFree: 416030720
  memTotal: 8226013184
  ociRuntime:
    name: runc
    package: Unknown
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc92
      commit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
      spec: 1.0.2-dev
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  rootless: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 8585076736
  swapTotal: 8589930496
  uptime: 418h 1m 22.36s (Approximately 17.42 days)
registries:
  search:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 2
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 5
  runRoot: /var/run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 2.0.0
  Built: 1601135437
  BuiltTime: Sun Sep 27 01:50:37 2020
  GitCommit: 9f6d6ba0b314d86521b66183c9ce48eaa2da1de2
  GoVersion: go1.15.2
  OsArch: linux/amd64
  Version: 2.1.1

Package info (e.g. output of rpm -q podman or apt list podman):

Name            : podman
Version         : 2.1.1-1
Description     : Tool and library for running OCI-based containers in pods
Architecture    : x86_64
URL             : https://github.com/containers/libpod
Licenses        : Apache
Groups          : None
Provides        : None
Depends On      : cni-plugins  conmon  device-mapper  iptables  libseccomp  runc  slirp4netns  libsystemd  fuse-overlayfs  skopeo
Optional Deps   : podman-docker: for Docker-compatible CLI
                  btrfs-progs: support btrfs backend devices [installed]
                  catatonit: --init flag support [installed]
                  crun: support for unified cgroupsv2 [installed]
Required By     : None
Optional For    : None
Conflicts With  : None
Replaces        : None
Installed Size  : 77.95 MiB
Packager        : Morten Linderud <[email protected]>
Build Date      : Sun 27 Sep 2020 01:50:37
Install Date    : Sun 27 Sep 2020 13:21:27
Install Reason  : Explicitly installed
Install Script  : No
Validated By    : Signature

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

Personal physical hardware.

[rhatdan]

@djmattyg007 Interested in opening a PR to enable this?

[rhatdan]

Is the basic idea of this that, if I create a container with

--add-host=host.podman.internal:host-gateway
And my host IP is 1.2.3.4

Then I will end up with an entry in /etc/hosts that looks like
1.2.3.4 host.podman.internal
?

[djmattyg007]

@djmattyg007 Interested in opening a PR to enable this?

I don't have the know-how to make this happen sorry.

Is the basic idea of this that, if I create a container with

--add-host=host.podman.internal:host-gateway
And my host IP is 1.2.3.4

Then I will end up with an entry in /etc/hosts that looks like
1.2.3.4 host.podman.internal
?

I'm not sure if it's a requirement to insert it into /etc/hosts (I'm not familiar enough with how all the networking works). I just know it's supposed to point to the container's gateway IP address (which is supposed to be the host). For the default bridge network created by podman in my installation, this would be 10.88.0.1.

[rhatdan]

Could someone verify what Docker does in this situation?

[mheon]

I don't think it really matters how Docker implemented this - they may have chosen to use DNS instead of /etc/hosts but they also have a more flexible DNS solution than we do. If we want to support this, our only option is to add an entry to /etc/hosts for the container in question. There are still some implementation details that would have to be settled, though - how does Docker handle a container connected to multiple networks? Do they add details for all of them?

[djmattyg007]

According to the PR I linked:

This PR allows containers to connect to Linux hosts
by appending a special string "host-gateway" to --add-host
e.g. "--add-host=host.docker.internal:host-gateway" which adds
host.docker.internal DNS entry in /etc/hosts and maps it to host-gateway-ip
This PR also add a daemon flag call host-gateway-ip which defaults to
the default bridge IP

I suspect it adds just one record for the default bridge network gateway for that container. Presumably if you created another bridge network and made that the default for your container, it would work that out and use that. It's difficult for me to verify exactly how it behaves until there's been a full docker release with this feature included.

[mheon]

Based on their description, I imagine it's just grabbing the default gateway and using that IP.

[djmattyg007]

Even that as a starting point would be great, as I suspect one of the most common use cases for this (myself included) is to use it with the globally default network with one-off containers.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[djmattyg007]

Not stale. Waiting on maintainers (who have likely been enjoying their Christmas break <3) to respond.

[rhatdan]

@mheon what is the latest on this Issue?

[mheon]

No activity that I’m aware of. May need to be implemented in the dnsname plugin?

…

On Wed, Jan 6, 2021 at 06:26 Daniel J Walsh ***@***.***> wrote: @mheon <https://github.com/mheon> what is the latest on this Issue? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8466 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB3AOCBA4QU3YYZQNFCECRDSYRCHNANCNFSM4UA3DMNA> .

[rhatdan]

Lets do that.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[djmattyg007]

Not stale. I've been checking regularly for updates only to find none.

[rhatdan]

Correct, but best way to get something done is open a PR. :^)

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[djmattyg007]

Not stale.

[mheon]

#9972 will fix

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[mheon]

I'm going to close. We don't support the flag, but we are adding the entry to /etc/hosts by default now.

[asbachb]

@mheon any details how to use this? I saw there's a "route" in /etc/hosts:

root@d9e153cabee0:/# cat /etc/hosts 
127.0.0.1	localhost
127.0.1.1	vm-virtualbox

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
# used by slirp4netns
10.0.2.100	d9e153cabee0 jolly_borg
10.0.2.2 host.containers.internal

But the container does not seem to be able to route to the host:

vm@vm-virtualbox:~$ python3 -m http.server 8000
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...

root@d9e153cabee0:/# curl http://127.0.1.1:8000
curl: (7) Failed to connect to 127.0.1.1 port 8000: Connection refused
root@d9e153cabee0:/# curl http://10.0.2.2:8000
curl: (7) Failed to connect to 10.0.2.2 port 8000: Network is unreachable

[mheon]

Please open a bug about this - we should probably change over to the host's public IP for rootless containers.

[flowl]

Hello,
is this feature available to Docker?
Because there is nothing about host-gateway in their docs.
Does it work with Docker version 20.10.17, build 100c701 ?

[devurandom]

Hello, is this feature available to Docker? Because there is nothing about host-gateway in their docs.

It is in the code: e.g. https://github.com/docker/cli/blob/a32cd16160f1b41c1c4ae7bee4dac929d1484e59/opts/hosts.go#L32