Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker compatibility: Container Inspect returns {} for NetworkSettings.Networks #10261

Closed
candrews opened this issue May 7, 2021 · 10 comments
Closed
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@candrews
Copy link

candrews commented May 7, 2021

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Steps to reproduce the issue:
Java 11 (or later) is a prerequisite for running this test.

  1. Clone https://github.com/candrews/podman-testcontainers-issue

  2. Run ./test-podman.sh

Describe the results you received:
The test fails. Looking at the stdout of the failing test:

[main] ERROR 🐳 [curlimages/curl:7.76.0] - Log output from the failed container:
curl: (6) Could not resolve host: host.testcontainers.internal

Describe the results you expected:
The test should pass; run ./test-docker.sh to run against docker and observe the test passing.

Additional information you deem important (e.g. issue happens only occasionally):

Here's a log of the socket when run against docker (which works):
docker.log

And here's a log of the same test run against podman (which fails):
podman.log

Output of podman version:

podman version 3.2.0-rc1

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.20.2-dev
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.0.27-2.fc34.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.27, commit: '
  cpus: 8
  distribution:
    distribution: fedora
    version: "34"
  eventLogger: journald
  hostname: craigatwork
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.11.18-300.fc34.x86_64
  linkmode: dynamic
  memFree: 584716288
  memTotal: 16647507968
  ociRuntime:
    name: crun
    package: crun-0.19.1-2.fc34.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.19.1
      commit: 1535fedf0b83fb898d449f9680000f729ba719f5
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.9-1.fc34.x86_64
    version: |-
      slirp4netns version 1.1.8+dev
      commit: 6dc0186e020232ae1a6fcc1f7afbc3ea02fd3876
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 13761241088
  swapTotal: 17024671744
  uptime: 21h 14m 26.11s (Approximately 0.88 days)
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/candrews/.config/containers/storage.conf
  containerStore:
    number: 14
    paused: 0
    running: 14
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.5.0-1.fc34.x86_64
      Version: |-
        fusermount3 version: 3.10.3
        fuse-overlayfs: version 1.5
        FUSE library version 3.10.3
        using FUSE kernel interface version 7.31
  graphRoot: /home/candrews/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 6
  runRoot: /run/user/1000
  volumePath: /home/candrews/.local/share/containers/storage/volumes
version:
  APIVersion: 3.2.0-rc1
  Built: 1620248626
  BuiltTime: Wed May  5 17:03:46 2021
  GitCommit: ""
  GoVersion: go1.16.3
  OsArch: linux/amd64
  Version: 3.2.0-rc1

Package info (e.g. output of rpm -q podman or apt list podman):

podman-3.2.0-0.1.rc1.fc34.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

Fedora 34

@openshift-ci-robot openshift-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label May 7, 2021
@candrews
Copy link
Author

candrews commented May 7, 2021

Doing some more debugging, I noticed that inside the curl container in this example, when docker is used, /etc/hosts looks like:

127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
172.17.0.19	host.testcontainers.internal
172.17.0.20	1c3a400a510b

But, when run with podman, /etc/hosts looks like:

127.0.0.1 view-localhost
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 craigatwork
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6 craigatwork
# used by slirp4netns
10.0.2.100	d3c2d1fd3eab mystifying_torvalds

The problem is that the entry for host.testcontainers.internal is missing when run with podman.

@candrews candrews changed the title docker compatibility: Testcontainers.exposeHostPorts doesn't work, host.testcontainers.internal not resolvable in container docker compatibility: Container Inspect returns {} for NetworkSettings.Networks May 7, 2021
@candrews
Copy link
Author

candrews commented May 7, 2021

The cause is that container Inspect returns empty {} for NetworkSettings.Networks.

Here's an exchange with podman:

GET /v1.30/containers/f407f6c5d34fb31c379b1e48b3085e7b03406c6201e479d5441207e9d1ec3a51/json HTTP/1.1\r
accept: application/json\r
Host: docker.socket\r
Connection: Keep-Alive\r
Accept-Encoding: gzip\r
User-Agent: okhttp/3.14.4\r
\r
< 2021/05/07 15:44:17.109943  length=4914 from=0 to=4913
HTTP/1.1 200 OK\r
Api-Version: 1.40\r
Content-Type: application/json\r
Libpod-Api-Version: 3.2.0-rc1\r
Server: Libpod/3.2.0-rc1 (linux)\r
Date: Fri, 07 May 2021 19:44:17 GMT\r
Transfer-Encoding: chunked\r
\r
800\r
{"Id":"f407f6c5d34fb31c379b1e48b3085e7b03406c6201e479d5441207e9d1ec3a51","Created":"2021-05-07T15:44:14.728015833-04:00","Path":"sh","Args":["-c","echo \\"root:$PASSWORD\\" | chpasswd \\u0026\\u0026 /usr/sbin/sshd -D -o PermitRootLogin=yes -o AddressFamily=inet -o GatewayPorts=yes"],"State":{"Status":"running","Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"Dead":false,"Pid":256790,"ExitCode":0,"Error":"","StartedAt":"2021-05-07T15:44:17.095985179-04:00","FinishedAt":"0001-01-01T00:00:00Z"},"Image":"docker.io/testcontainers/sshd:1.0.0","ResolvConfPath":"/run/user/1000/overlay-containers/f407f6c5d34fb31c379b1e48b3085e7b03406c6201e479d5441207e9d1ec3a51/userdata/resolv.conf","HostnamePath":"/run/user/1000/overlay-containers/f407f6c5d34fb31c379b1e48b3085e7b03406c6201e479d5441207e9d1ec3a51/userdata/hostname","HostsPath":"/run/user/1000/overlay-containers/f407f6c5d34fb31c379b1e48b3085e7b03406c6201e479d5441207e9d1ec3a51/userdata/hosts","LogPath":"/home/candrews/.local/share/containers/storage/overlay-containers/f407f6c5d34fb31c379b1e48b3085e7b03406c6201e479d5441207e9d1ec3a51/userdata/ctr.log","Name":"/zen_shamir","RestartCount":0,"Driver":"overlay","Platform":"linux","MountLabel":"system_u:object_r:container_file_t:s0:c924,c960","ProcessLabel":"system_u:system_r:container_t:s0:c924,c960","AppArmorProfile":"","ExecIDs":[],"HostConfig":{"Binds":[],"ContainerIDFile":"","LogConfig":{"Type":"json-file","Config":null},"NetworkMode":"slirp4netns","PortBindings":{"22/tcp":[{"HostIp":"","HostPort":"46701"}]},"RestartPolicy":{"Name":"","MaximumRetryCount":0},"AutoRemove":false,"VolumeDriver":"","VolumesFrom":null,"CapAdd":[],"CapDrop":["AUDIT_WRITE","MKNOD","NET_RAW"],"CgroupnsMode":"","Dns":[],"DnsOptions":[],"DnsSearch":[],"ExtraHosts":[],"GroupAdd":[],"IpcMode":"private","Cgroup":"","Links":null,"OomScoreAdj":0,"PidMode":"private","Privileged":false,"PublishAllPorts":false,"ReadonlyRootfs":false,"SecurityOpt":[],"UTSMode":"private","UsernsMode":"","ShmSize":65536000,"Runtime":"oci","ConsoleSize":[0,0],"Isolatio\r
800\r
n":"","CpuShares":0,"Memory":0,"NanoCpus":0,"CgroupParent":"","BlkioWeight":0,"BlkioWeightDevice":null,"BlkioDeviceReadBps":null,"BlkioDeviceWriteBps":null,"BlkioDeviceReadIOps":null,"BlkioDeviceWriteIOps":null,"CpuPeriod":0,"CpuQuota":0,"CpuRealtimePeriod":0,"CpuRealtimeRuntime":0,"CpusetCpus":"","CpusetMems":"","Devices":[],"DeviceCgroupRules":null,"DeviceRequests":null,"KernelMemory":0,"KernelMemoryTCP":0,"MemoryReservation":0,"MemorySwap":0,"MemorySwappiness":0,"OomKillDisable":false,"PidsLimit":0,"Ulimits":[],"CpuCount":0,"CpuPercent":0,"IOMaximumIOps":0,"IOMaximumBandwidth":0,"MaskedPaths":null,"ReadonlyPaths":null},"GraphDriver":{"Data":{"LowerDir":"/home/candrews/.local/share/containers/storage/overlay/802b211bfc74359439a1f8383eb0fe7a14184a54bad35386832f753be750fdcb/diff:/home/candrews/.local/share/containers/storage/overlay/f566c57e6f2da2364c14195c832b922fd8f4813fd139b8fe45e3454c16e33975/diff","MergedDir":"/home/candrews/.local/share/containers/storage/overlay/b8f8aaa4d8597ddab1d041bd1ec3befd38474f89f27e0e6e1678e77f517898df/merged","UpperDir":"/home/candrews/.local/share/containers/storage/overlay/b8f8aaa4d8597ddab1d041bd1ec3befd38474f89f27e0e6e1678e77f517898df/diff","WorkDir":"/home/candrews/.local/share/containers/storage/overlay/b8f8aaa4d8597ddab1d041bd1ec3befd38474f89f27e0e6e1678e77f517898df/work"},"Name":"overlay"},"SizeRootFs":0,"Mounts":[],"Config":{"Hostname":"f407f6c5d34f","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"ExposedPorts":{"22/tcp":{}},"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","TERM=xterm","container=podman","PASSWORD=7fc9cc6b-e3a5-4390-8130-bca83e0a0aba","HOME=/root","HOSTNAME=f407f6c5d34f"],"Cmd":["sh","-c","echo \\"root:$PASSWORD\\" | chpasswd \\u0026\\u0026 /usr/sbin/sshd -D -o PermitRootLogin=yes -o AddressFamily=inet -o GatewayPorts=yes"],"Image":"docker.io/testcontainers/sshd:1.0.0","Volumes":null,"WorkingDir":"/","Entrypoint":[],"OnBuild":null,"Labels":{"org.t\r
250\r
estcontainers":"true","org.testcontainers.sessionId":"0862da37-c836-4faa-a86c-8936b847c2f2"},"StopSignal":"15","StopTimeout":0},"NetworkSettings":{"Bridge":"","SandboxID":"","HairpinMode":false,"LinkLocalIPv6Address":"","LinkLocalIPv6PrefixLen":0,"Ports":{"22/tcp":[{"HostIp":"","HostPort":"46701"}]},"SandboxKey":"/run/user/1000/netns/cni-e9c3eb7d-0852-393c-3988-ca4628b6891d","SecondaryIPAddresses":null,"SecondaryIPv6Addresses":null,"EndpointID":"","Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"IPAddress":"","IPPrefixLen":0,"IPv6Gateway":"","MacAddress":"","Networks":{}}}

NetworkSettings.Networks is {}. It was set to that in #10206

However, that's different than what docker does. Here's the same exchange with docker:

GET /v1.30/containers/32e10322299e47cd7c0c3fae44fdc6f80221aace1ee0b37461e516655c681f03/json HTTP/1.1\r
accept: application/json\r
Host: docker.socket\r
Connection: Keep-Alive\r
Accept-Encoding: gzip\r
User-Agent: okhttp/3.14.4\r
\r
< 2021/05/07 13:27:51.829470  length=5330 from=0 to=5329
HTTP/1.1 200 OK\r
Api-Version: 1.41\r
Content-Type: application/json\r
Docker-Experimental: false\r
Ostype: linux\r
Server: Docker/20.10.6 (linux)\r
Date: Fri, 07 May 2021 17:27:51 GMT\r
Transfer-Encoding: chunked\r
\r
13f3\r
{"Id":"32e10322299e47cd7c0c3fae44fdc6f80221aace1ee0b37461e516655c681f03","Created":"2021-05-07T17:27:51.447140305Z","Path":"sh","Args":["-c","echo \\"root:$PASSWORD\\" | chpasswd && /usr/sbin/sshd -D -o PermitRootLogin=yes -o AddressFamily=inet -o GatewayPorts=yes"],"State":{"Status":"running","Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"Dead":false,"Pid":183709,"ExitCode":0,"Error":"","StartedAt":"2021-05-07T17:27:51.809185887Z","FinishedAt":"0001-01-01T00:00:00Z"},"Image":"sha256:b665446a05c2fe2d551eccd8d29f6ee3d42c00f15597fd0f571900721380fd5d","ResolvConfPath":"/var/lib/docker/containers/32e10322299e47cd7c0c3fae44fdc6f80221aace1ee0b37461e516655c681f03/resolv.conf","HostnamePath":"/var/lib/docker/containers/32e10322299e47cd7c0c3fae44fdc6f80221aace1ee0b37461e516655c681f03/hostname","HostsPath":"/var/lib/docker/containers/32e10322299e47cd7c0c3fae44fdc6f80221aace1ee0b37461e516655c681f03/hosts","LogPath":"","Name":"/strange_tharp","RestartCount":0,"Driver":"overlay2","Platform":"linux","MountLabel":"system_u:object_r:container_file_t:s0:c531,c981","ProcessLabel":"system_u:system_r:container_t:s0:c531,c981","AppArmorProfile":"","ExecIDs":null,"HostConfig":{"Binds":[],"ContainerIDFile":"","LogConfig":{"Type":"journald","Config":{}},"NetworkMode":"default","PortBindings":{},"RestartPolicy":{"Name":"","MaximumRetryCount":0},"AutoRemove":false,"VolumeDriver":"","VolumesFrom":[],"CapAdd":null,"CapDrop":null,"CgroupnsMode":"host","Dns":null,"DnsOptions":null,"DnsSearch":null,"ExtraHosts":[],"GroupAdd":null,"IpcMode":"shareable","Cgroup":"","Links":null,"OomScoreAdj":0,"PidMode":"","Privileged":false,"PublishAllPorts":true,"ReadonlyRootfs":false,"SecurityOpt":null,"UTSMode":"","UsernsMode":"","ShmSize":67108864,"Runtime":"runc","ConsoleSize":[0,0],"Isolation":"","CpuShares":0,"Memory":0,"NanoCpus":0,"CgroupParent":"","BlkioWeight":0,"BlkioWeightDevice":null,"BlkioDeviceReadBps":null,"BlkioDeviceWriteBps":null,"BlkioDeviceReadIOps":null,"BlkioDeviceWriteIOps":null,"CpuPeriod":0,"CpuQuota":0,"CpuRealtimePeriod":0,"CpuRealtimeRuntime":0,"CpusetCpus":"","CpusetMems":"","Devices":null,"DeviceCgroupRules":null,"DeviceRequests":null,"KernelMemory":0,"KernelMemoryTCP":0,"MemoryReservation":0,"MemorySwap":0,"MemorySwappiness":null,"OomKillDisable":false,"PidsLimit":null,"Ulimits":[{"Name":"nofile","Hard":1024,"Soft":1024}],"CpuCount":0,"CpuPercent":0,"IOMaximumIOps":0,"IOMaximumBandwidth":0,"MaskedPaths":["/proc/asound","/proc/acpi","/proc/kcore","/proc/keys","/proc/latency_stats","/proc/timer_list","/proc/timer_stats","/proc/sched_debug","/proc/scsi","/sys/firmware"],"ReadonlyPaths":["/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]},"GraphDriver":{"Data":{"LowerDir":"/var/lib/docker/overlay2/a16f25cbe3e7be425d9035a4310cd15c38dec682104d654c73d865eeb23b1271-init/diff:/var/lib/docker/overlay2/c15567da41b8ad450ccc121c02d175904df8eacbfbf1592c185bd2a1adb3e3c3/diff:/var/lib/docker/overlay2/569b37dff748becb02407bc50dd54f17265f067b44cbabfe795d93b2ab164a7b/diff","MergedDir":"/var/lib/docker/overlay2/a16f25cbe3e7be425d9035a4310cd15c38dec682104d654c73d865eeb23b1271/merged","UpperDir":"/var/lib/docker/overlay2/a16f25cbe3e7be425d9035a4310cd15c38dec682104d654c73d865eeb23b1271/diff","WorkDir":"/var/lib/docker/overlay2/a16f25cbe3e7be425d9035a4310cd15c38dec682104d654c73d865eeb23b1271/work"},"Name":"overlay2"},"Mounts":[],"Config":{"Hostname":"32e10322299e","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"ExposedPorts":{"22/tcp":{}},"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PASSWORD=34e69642-0084-43cb-be30-364e374800d5","PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":["sh","-c","echo \\"root:$PASSWORD\\" | chpasswd && /usr/sbin/sshd -D -o PermitRootLogin=yes -o AddressFamily=inet -o GatewayPorts=yes"],"Image":"testcontainers/sshd:1.0.0","Volumes":null,"WorkingDir":"","Entrypoint":null,"OnBuild":null,"Labels":{"org.testcontainers":"true","org.testcontainers.sessionId":"85434cfd-ab8d-4bb0-809b-c8c89bc8503a"}},"NetworkSettings":{"Bridge":"","SandboxID":"3e98458e78edd1eb9269cf161cd84bd0c86ec5dabdd84d8477acbeda31b59bd7","HairpinMode":false,"LinkLocalIPv6Address":"","LinkLocalIPv6PrefixLen":0,"Ports":{"22/tcp":[{"HostIp":"0.0.0.0","HostPort":"49165"},{"HostIp":"::","HostPort":"49165"}]},"SandboxKey":"/var/run/docker/netns/3e98458e78ed","SecondaryIPAddresses":null,"SecondaryIPv6Addresses":null,"EndpointID":"d091082b2bf6f1feeb7f2319015a6d44560345f1e0c0b1c4c4b44330e2abf555","Gateway":"172.17.0.1","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"IPAddress":"172.17.0.14","IPPrefixLen":16,"IPv6Gateway":"","MacAddress":"02:42:ac:11:00:0e","Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"50f6517cf3aab7ca4345d22e48cfeb2245bb1ffeadbf9a3f10126de50847af2f","EndpointID":"d091082b2bf6f1feeb7f2319015a6d44560345f1e0c0b1c4c4b44330e2abf555","Gateway":"172.17.0.1","IPAddress":"172.17.0.14","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:11:00:0e","DriverOpts":null}}}}

For easier comparison, here is docker's NetworkSettings.Networks:

  "NetworkSettings": {
    "Bridge": "",
    "SandboxID": "353c0a8c511c4cae906231bafcad6d5a2ecc1ec072e7facda979ffa9018202bc",
    "HairpinMode": false,
    "LinkLocalIPv6Address": "",
    "LinkLocalIPv6PrefixLen": 0,
    "Ports": {},
    "SandboxKey": "/var/run/docker/netns/353c0a8c511c",
    "SecondaryIPAddresses": null,
    "SecondaryIPv6Addresses": null,
    "EndpointID": "f710e9bd49b67ac67da05bd835ef88533d0f039baf60131edaae68a5b808e6cb",
    "Gateway": "172.17.0.1",
    "GlobalIPv6Address": "",
    "GlobalIPv6PrefixLen": 0,
    "IPAddress": "172.17.0.15",
    "IPPrefixLen": 16,
    "IPv6Gateway": "",
    "MacAddress": "02:42:ac:11:00:0f",
    "Networks": {
      "bridge": {
        "IPAMConfig": null,
        "Links": null,
        "Aliases": null,
        "NetworkID": "50f6517cf3aab7ca4345d22e48cfeb2245bb1ffeadbf9a3f10126de50847af2f",
        "EndpointID": "f710e9bd49b67ac67da05bd835ef88533d0f039baf60131edaae68a5b808e6cb",
        "Gateway": "172.17.0.1",
        "IPAddress": "172.17.0.15",
        "IPPrefixLen": 16,
        "IPv6Gateway": "",
        "GlobalIPv6Address": "",
        "GlobalIPv6PrefixLen": 0,
        "MacAddress": "02:42:ac:11:00:0f",
        "DriverOpts": null
      }
    }
  }

and here is podman's:

  "NetworkSettings": {
    "Bridge": "",
    "SandboxID": "",
    "HairpinMode": false,
    "LinkLocalIPv6Address": "",
    "LinkLocalIPv6PrefixLen": 0,
    "Ports": {},
    "SandboxKey": "",
    "SecondaryIPAddresses": null,
    "SecondaryIPv6Addresses": null,
    "EndpointID": "",
    "Gateway": "",
    "GlobalIPv6Address": "",
    "GlobalIPv6PrefixLen": 0,
    "IPAddress": "",
    "IPPrefixLen": 0,
    "IPv6Gateway": "",
    "MacAddress": "",
    "Networks": {}
  }

testcontainers checks for for NetworkSettings.Networks to be set to something before it adds extra hosts which explains why host.testcontainers.internal isn't getting added to /etc/hosts resulting in this issue. Furthermore, it gets the IP address to use for the host.testcontainers.internal host entry from NetworkSettings.Networks[0].IPAddress See:

@Luap99
Copy link
Member

Luap99 commented May 7, 2021

Do you run podman as rootless? If so you do not get any networks by default.

@candrews
Copy link
Author

candrews commented May 7, 2021

Do you run podman as rootless? If so you do not get any networks by default.

I do run it rootless.

Shouldn't there be a network by default?

I haven't tried rootless docker, but it's documentation seems to indicate that container inspect returns NetworkSettings:

IPAddress shown in docker inspect and is namespaced inside RootlessKit’s network namespace. This means the IP address is not reachable from the host without nsenter-ing into the network namespace.

@Luap99
Copy link
Member

Luap99 commented May 7, 2021

The default network mode as rootless is slirp4netns. If you need to join a network you have to create and specify them.

@candrews
Copy link
Author

candrews commented May 7, 2021

The default network mode as rootless is slirp4netns. If you need to join a network you have to create and specify them.

I installed docker-rootless to see how it behaved. Here's the transcript of the same container inspect exchange:

GET /v1.30/containers/08f712e1368183c9a0d3f2fe020bd502aa88c8eb97870f551b4c88f293e813f5/json HTTP/1.1\r
accept: application/json\r
Host: docker.socket\r
Connection: Keep-Alive\r
Accept-Encoding: gzip\r
User-Agent: okhttp/3.14.4\r
\r
< 2021/05/07 16:39:07.794822  length=5610 from=0 to=5609
HTTP/1.1 200 OK\r
Api-Version: 1.41\r
Content-Type: application/json\r
Docker-Experimental: false\r
Ostype: linux\r
Server: Docker/20.10.6 (linux)\r
Date: Fri, 07 May 2021 20:39:07 GMT\r
Transfer-Encoding: chunked\r
\r
150b\r
{"Id":"08f712e1368183c9a0d3f2fe020bd502aa88c8eb97870f551b4c88f293e813f5","Created":"2021-05-07T20:39:07.331227686Z","Path":"sh","Args":["-c","echo \\"root:$PASSWORD\\" | chpasswd && /usr/sbin/sshd -D -o PermitRootLogin=yes -o AddressFamily=inet -o GatewayPorts=yes"],"State":{"Status":"running","Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"Dead":false,"Pid":284655,"ExitCode":0,"Error":"","StartedAt":"2021-05-07T20:39:07.723202231Z","FinishedAt":"0001-01-01T00:00:00Z"},"Image":"sha256:b665446a05c2fe2d551eccd8d29f6ee3d42c00f15597fd0f571900721380fd5d","ResolvConfPath":"/home/candrews/.local/share/docker/containers/08f712e1368183c9a0d3f2fe020bd502aa88c8eb97870f551b4c88f293e813f5/resolv.conf","HostnamePath":"/home/candrews/.local/share/docker/containers/08f712e1368183c9a0d3f2fe020bd502aa88c8eb97870f551b4c88f293e813f5/hostname","HostsPath":"/home/candrews/.local/share/docker/containers/08f712e1368183c9a0d3f2fe020bd502aa88c8eb97870f551b4c88f293e813f5/hosts","LogPath":"/home/candrews/.local/share/docker/containers/08f712e1368183c9a0d3f2fe020bd502aa88c8eb97870f551b4c88f293e813f5/08f712e1368183c9a0d3f2fe020bd502aa88c8eb97870f551b4c88f293e813f5-json.log","Name":"/festive_hugle","RestartCount":0,"Driver":"fuse-overlayfs","Platform":"linux","MountLabel":"","ProcessLabel":"","AppArmorProfile":"","ExecIDs":null,"HostConfig":{"Binds":[],"ContainerIDFile":"","LogConfig":{"Type":"json-file","Config":{}},"NetworkMode":"default","PortBindings":{},"RestartPolicy":{"Name":"","MaximumRetryCount":0},"AutoRemove":false,"VolumeDriver":"","VolumesFrom":[],"CapAdd":null,"CapDrop":null,"CgroupnsMode":"host","Dns":null,"DnsOptions":null,"DnsSearch":null,"ExtraHosts":[],"GroupAdd":null,"IpcMode":"shareable","Cgroup":"","Links":null,"OomScoreAdj":0,"PidMode":"","Privileged":false,"PublishAllPorts":true,"ReadonlyRootfs":false,"SecurityOpt":null,"UTSMode":"","UsernsMode":"","ShmSize":67108864,"Runtime":"runc","ConsoleSize":[0,0],"Isolation":"","CpuShares":0,"Memory":0,"NanoCpus":0,"CgroupParent":"","BlkioWeight":0,"BlkioWeightDevice":null,"BlkioDeviceReadBps":null,"BlkioDeviceWriteBps":null,"BlkioDeviceReadIOps":null,"BlkioDeviceWriteIOps":null,"CpuPeriod":0,"CpuQuota":0,"CpuRealtimePeriod":0,"CpuRealtimeRuntime":0,"CpusetCpus":"","CpusetMems":"","Devices":null,"DeviceCgroupRules":null,"DeviceRequests":null,"KernelMemory":0,"KernelMemoryTCP":0,"MemoryReservation":0,"MemorySwap":0,"MemorySwappiness":null,"OomKillDisable":false,"PidsLimit":null,"Ulimits":null,"CpuCount":0,"CpuPercent":0,"IOMaximumIOps":0,"IOMaximumBandwidth":0,"MaskedPaths":["/proc/asound","/proc/acpi","/proc/kcore","/proc/keys","/proc/latency_stats","/proc/timer_list","/proc/timer_stats","/proc/sched_debug","/proc/scsi","/sys/firmware"],"ReadonlyPaths":["/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]},"GraphDriver":{"Data":{"LowerDir":"/home/candrews/.local/share/docker/fuse-overlayfs/cd7e0f0c38ba0d3dc820455530011c7ed4edeb1187b9f45036c1278d5d4e1357-init/diff:/home/candrews/.local/share/docker/fuse-overlayfs/99733141ac819c712568ef3a81acd32516d4e8d9301f7d200f56444c4824b752/diff:/home/candrews/.local/share/docker/fuse-overlayfs/04d92d3903813df43420892405024c5e878ccb7d3f6c148b53eda6f071333a06/diff","MergedDir":"/home/candrews/.local/share/docker/fuse-overlayfs/cd7e0f0c38ba0d3dc820455530011c7ed4edeb1187b9f45036c1278d5d4e1357/merged","UpperDir":"/home/candrews/.local/share/docker/fuse-overlayfs/cd7e0f0c38ba0d3dc820455530011c7ed4edeb1187b9f45036c1278d5d4e1357/diff","WorkDir":"/home/candrews/.local/share/docker/fuse-overlayfs/cd7e0f0c38ba0d3dc820455530011c7ed4edeb1187b9f45036c1278d5d4e1357/work"},"Name":"fuse-overlayfs"},"Mounts":[],"Config":{"Hostname":"08f712e13681","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"ExposedPorts":{"22/tcp":{}},"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PASSWORD=db293f00-0100-45dd-8b17-843fc08ad5e5","PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":["sh","-c","echo \\"root:$PASSWORD\\" | chpasswd && /usr/sbin/sshd -D -o PermitRootLogin=yes -o AddressFamily=inet -o GatewayPorts=yes"],"Image":"testcontainers/sshd:1.0.0","Volumes":null,"WorkingDir":"","Entrypoint":null,"OnBuild":null,"Labels":{"org.testcontainers":"true","org.testcontainers.sessionId":"41f804e9-5725-4034-849d-af70665a9943"}},"NetworkSettings":{"Bridge":"","SandboxID":"2b831fb77fc10d4a216362a8e2c9aedc2fa61b2de953da6412c9f9de6db8c7bf","HairpinMode":false,"LinkLocalIPv6Address":"","LinkLocalIPv6PrefixLen":0,"Ports":{"22/tcp":[{"HostIp":"0.0.0.0","HostPort":"49153"},{"HostIp":"::","HostPort":"49153"}]},"SandboxKey":"/run/user/1000/docker/netns/2b831fb77fc1","SecondaryIPAddresses":null,"SecondaryIPv6Addresses":null,"EndpointID":"7e73cbc45075ccb7aaaf9ee68c713970e711917a7b9c5f683b3fde28b2b7bb42","Gateway":"172.17.0.1","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"IPAddress":"172.17.0.2","IPPrefixLen":16,"IPv6Gateway":"","MacAddress":"02:42:ac:11:00:02","Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"b22bf5f34e03b7912ce2e3e559b9c77cbd289ff1b0fd1ac78b0668dd997e6d01","EndpointID":"7e73cbc45075ccb7aaaf9ee68c713970e711917a7b9c5f683b3fde28b2b7bb42","Gateway":"172.17.0.1","IPAddress":"172.17.0.2","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:11:00:02","DriverOpts":null}}}}

Note that NetworkSettings.Networks is populated:

	"NetworkSettings": {
		"Bridge": "",
		"SandboxID": "2b831fb77fc10d4a216362a8e2c9aedc2fa61b2de953da6412c9f9de6db8c7bf",
		"HairpinMode": false,
		"LinkLocalIPv6Address": "",
		"LinkLocalIPv6PrefixLen": 0,
		"Ports": {
			"22/tcp": [{
				"HostIp": "0.0.0.0",
				"HostPort": "49153"
			}, {
				"HostIp": "::",
				"HostPort": "49153"
			}]
		},
		"SandboxKey": "/run/user/1000/docker/netns/2b831fb77fc1",
		"SecondaryIPAddresses": null,
		"SecondaryIPv6Addresses": null,
		"EndpointID": "7e73cbc45075ccb7aaaf9ee68c713970e711917a7b9c5f683b3fde28b2b7bb42",
		"Gateway": "172.17.0.1",
		"GlobalIPv6Address": "",
		"GlobalIPv6PrefixLen": 0,
		"IPAddress": "172.17.0.2",
		"IPPrefixLen": 16,
		"IPv6Gateway": "",
		"MacAddress": "02:42:ac:11:00:02",
		"Networks": {
			"bridge": {
				"IPAMConfig": null,
				"Links": null,
				"Aliases": null,
				"NetworkID": "b22bf5f34e03b7912ce2e3e559b9c77cbd289ff1b0fd1ac78b0668dd997e6d01",
				"EndpointID": "7e73cbc45075ccb7aaaf9ee68c713970e711917a7b9c5f683b3fde28b2b7bb42",
				"Gateway": "172.17.0.1",
				"IPAddress": "172.17.0.2",
				"IPPrefixLen": 16,
				"IPv6Gateway": "",
				"GlobalIPv6Address": "",
				"GlobalIPv6PrefixLen": 0,
				"MacAddress": "02:42:ac:11:00:02",
				"DriverOpts": null
			}
		}
	}

@mheon
Copy link
Member

mheon commented May 9, 2021 via email

@candrews
Copy link
Author

I ran podman with ‘—log-level=debug’, then ran ./test-podman.sh from the test project at https://github.com/candrews/podman-testcontainers-issue

Here's the resulting log output:
podman-debug.log

@Luap99
Copy link
Member

Luap99 commented Jun 3, 2021

Can you retry with latest master. I think it should work now.

@rhatdan
Copy link
Member

rhatdan commented Jun 7, 2021

Reopen if we are mistaken.

@rhatdan rhatdan closed this as completed Jun 7, 2021
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

No branches or pull requests

5 participants