ipv6 neighbor proxy does not work

[fansari]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

I use netavark as network backend.
I don't want to use IPv6 NAT.
Instead I proxy an IPv6 adresss but cannot reach it from external. Neither the container IPs are reachable nor is the IPv6 address of the podman interface.

Steps to reproduce the issue:

Make sure you have:

net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.all.proxy_ndp = 1

Create a podman1 interface with an IPv6 network (e.g. as /96 if you only have one /64 from your ISP).

/etc/containers/networks/podman1.json

{
     "name": "podman1",
     "id": "8a61d27afd9be7ea11073fb6dfe003ed11a9c7c0b5e70ccd087fb075847bd788",
     "driver": "bridge",
     "network_interface": "podman1",
     "created": "2022-05-26T14:45:30.397155928Z",
     "subnets": [
          {
               "subnet": "10.90.0.0/16",
               "gateway": "10.90.0.1"
          },
          {
              "subnet": "2a03:xxxx:xxxx:xxxx:xxxx:xxxx::/96",
              "gateway": "2a03:xxx:xxxx:xxxx:xxxx:xxxx::1"
          }
     ],
     "ipv6_enabled": true,
     "internal": false,
     "dns_enabled": true,
     "ipam_options": {
          "driver": "host-local"
     }
}

Proxy the IP of this network to your interface which is connected to the Internet.

ip -6 neigh add proxy 2a03:xxxx:xxxx:xxxx::1 dev ens3

Check the result:

ip -6 neigh show proxy

Try to ping this IP from internal. Works? Then next:

Try to ping this IP from external.

Create one container and assign to it an IPv6 address of our /96 range. Then proxy the IP as explained above.

Try to ping an external IP (e.g. Google DNS) from within such a container.

Describe the results you received:
Ping from external does not answer.
Containers are not able to ping external IPv6 addresses.

Describe the results you expected:
Ping from external should answer.
Containers should be able to reach the Internet with IPv6 without NAT.

Additional information you deem important (e.g. issue happens only occasionally):

I also tried this with stopped firewall (nftables) and this also did not work.
Also I tried the same thing with containers themselves (assiging IPv6 addresses to them and testing).

Output of podman version:

Client:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.18beta2

Built:      Thu Mar  3 14:56:09 2022
OS/Arch:    linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.24.1
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.0-2.fc36.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: '
  cpus: 2
  distribution:
    distribution: fedora
    variant: coreos
    version: "36"
  eventLogger: journald
  hostname: magpie.frank-ansari.de
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.17.5-300.fc36.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 7738515456
  memTotal: 8331010048
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.4.4-1.fc36.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.4.4
      commit: 6521fcc5806f20f6187eb933f9f45130c86da230
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
 remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-0.2.beta.0.fc36.x86_64
    version: |-
      slirp4netns version 1.2.0-beta.0
      commit: 477db14a24ff1a3de3a705e51ca2c4c1fe3dda64
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 55m 55.34s
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 5
    paused: 0
    running: 5
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 28
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.0.2
  Built: 1646319369
  BuiltTime: Thu Mar  3 14:56:09 2022
  GitCommit: ""
  GoVersion: go1.18beta2
  OsArch: linux/amd64
  Version: 4.0.2

Package info (e.g. output of rpm -q podman or apt list podman):

podman-4.0.2-1.fc36.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes/No

Additional environment details (AWS, VirtualBox, physical, etc.):

This was the same issue I had with the old network stack (CNI).

#11999

I am not really sure whether podman is the issue. On my PC I have a similar scenario (Fedora 36, podman 4.1.0) and there the containers can reach the Internet without NAT.

The described issue occurs on a KVM VM of my hoster. So maybe this occurs only in certain scenarios but I have no idea what the reason might be.

[fansari]

I run a tcpdump on the target host.

When I ping the IP of the external interface from my PC

tcpdump -i ens3 host <my_host_ip> and icmp6

I see echos and echo replies on the target host.

On the other hand when I do this:

tcpdump -i ens3 host <my_container_ip> and icmp6

I don't see anything incoming on the target host (even when the firewall is off).

So maybe this is some issue with my hoster.

[nivekuil]

Could you try assigning your host a different ip? Maybe there's a whitelist?

[fansari]

My webhoster is netcup.de. As it turned out you can only use IPv6 fully (which includes proxy ndp) if you buy an additioinal IPv6 net. The net which they deliver does not work for such a scenario.

https://forum.netcup.de/administration-eines-server-vserver/vserver-server-kvm-server/p181533-ipv6-trotz-proxy-ndp-kommt-kein-traffic-an/#post181585

I bought an additional /64 net an this worked instantly.

[nivekuil]

odd, my netcup rootserver works fine with the kernel's proxy_ndp. I'm pretty sure this should work from the support's last post and linked post, the additional subnet you can buy is routed, the default one is on-link (as is typical). I'd like to see this built in to netavark too (containers/netavark#270)

[rhatdan]

So is this actually an issue Podman can solve?

[fansari]

I think this has nothing to do with podman. Seems to be something the user has to care about.