TestFunctional/parallel/MountCmd fails in GitHub actions with podman driver

[ilya-zuyev]

Mount comand fails:

        I0505 04:52:14.911379   21117 out.go:170] * Mounting host path /tmp/mounttest079057694 into VM as /mount-9p ...
        I0505 04:52:14.912059   21117 out.go:170]   - Mount type:   
        I0505 04:52:14.912443   21117 out.go:170]   - User ID:      docker
        I0505 04:52:14.912896   21117 out.go:170]   - Group ID:     docker
        I0505 04:52:14.913952   21117 out.go:170]   - Version:      9p2000.L
        I0505 04:52:14.914529   21117 out.go:170]   - Message Size: 262144
        I0505 04:52:14.915176   21117 out.go:170]   - Permissions:  755 (-rwxr-xr-x)
        I0505 04:52:14.915610   21117 out.go:170]   - Options:      map[]
        I0505 04:52:14.916120   21117 out.go:170]   - **Bind Address: <nil>:43651**
...
       I0505 04:52:15.388921   21117 ssh_runner.go:149] Run: /bin/bash -c "sudo mount -t 9p -o dfltgid=$(grep ^docker: /etc/group | cut -d: -f3),dfltuid=$(id -u docker),msize=262144,port=43651,trans=tcp,version=9p2000.L <nil> /mount-9p"
        I0505 04:52:15.407716   21117 out.go:170] 
        W0505 04:52:15.407873   21117 out.go:235] X Exiting due to GUEST_MOUNT: mount with cmd /bin/bash -c "sudo mount -t 9p -o dfltgid=$(grep ^docker: /etc/group | cut -d: -f3),dfltuid=$(id -u docker),msize=262144,port=43651,trans=tcp,version=9p2000.L <nil> /mount-9p" : /bin/bash -c "sudo mount -t 9p -o dfltgid=$(grep ^docker: /etc/group | cut -d: -f3),dfltuid=$(id -u docker),msize=262144,port=43651,trans=tcp,version=9p2000.L <nil> /mount-9p": Process exited with status 1

Please not, the Bind Address has "nil" host part.
Comes from the result of oci.containerGatewayIP which runs podman container inspect --format {{.NetworkSettings.Gateway}}, and inspect returns

{
 ...
 NetworkSettings": {
      "EndpointID": "",
      "Gateway": "",
      "IPAddress": "",
      "IPPrefixLen": 0,
      "IPv6Gateway": "",
      "GlobalIPv6Address": "",
      "GlobalIPv6PrefixLen": 0,
      "MacAddress": "",
      "Bridge": "",
      "SandboxID": "",
      "HairpinMode": false,
      "LinkLocalIPv6Address": "",
      "LinkLocalIPv6PrefixLen": 0, ... 
}

7_Run Integration Test.txt

[afbjorklund]

It seems that the custom network doesn't have this. A regular container will happily reply 10.88.0.1

        "NetworkSettings": {
            "EndpointID": "",
            "Gateway": "10.88.0.1",
            "IPAddress": "10.88.0.166",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "MacAddress": "9a:79:8f:ba:69:ce",
            "Bridge": "",
            "SandboxID": "",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/run/netns/cni-14484bc4-43d8-6793-52ae-3c144bffc363",
            "Networks": {
                "podman": {
                    "EndpointID": "",
                    "Gateway": "10.88.0.1",
                    "IPAddress": "10.88.0.166",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "9a:79:8f:ba:69:ce",
                    "NetworkID": "podman",
                    "DriverOpts": null,
                    "IPAMConfig": null,
                    "Links": null
                }
            }
        },

        "NetworkSettings": {
            "EndpointID": "",
            "Gateway": "",
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "MacAddress": "",
            "Bridge": "",
            "SandboxID": "",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/run/netns/cni-a96b5a52-8393-70da-b0e5-54ec25cd0708",
            "Networks": {
                "test": {
                    "EndpointID": "",
                    "Gateway": "10.89.0.1",
                    "IPAddress": "10.89.0.2",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "7e:5c:18:be:67:c9",
                    "NetworkID": "test",
                    "DriverOpts": null,
                    "IPAMConfig": null,
                    "Links": null
                }
            }
        },

So apparently we have to get the gateway from the nested Networks document ?

[afbjorklund]

Yes, there is a bug in the podman code that only sets the basic network config for the default network...

https://github.com/containers/podman/blob/v3.1.2/libpod/networking_linux.go#L1040

 settings.InspectBasicNetworkConfig = basicConfig

We will need to get it from the Networks, for compatibility with older Podman version (before any fix).

[afbjorklund]

Apparently it was left empty on purpose, for Docker compatibility. Sadly we used different code for them.

        if ociBin == Docker {
                return dockerNetworkInspect(name)
        }
        if ociBin == Podman {
                return podmanNetworkInspect(name)
        }