ipv6 connectivity -- managing NDP entries

[nivekuil]

/kind feature

Description

Podman 4.0 has added support for ipv6 static addresses (really appreciating the perf improvements with netavark, btw). Podman also has facilities for routing traffic to containers (e.g. managing firewall rules). With this in mind, it could be nice if Podman also took the step of ensuring ipv6 connectivity through managing the host's NDP table.

Linux has some support for responding to NDP requests, but this must manually set with iproute2 per address and not an entire subnet. It is therefore more practical to run a daemon (ndppd, ndpresponder) to do this mapping dynamically. Could Podman remove the need for this extra dependency by managing the host's NDP table to automatically ensure connectivity to containers?

[Luap99]

Moved to netavark since we would need to implement it here.
I assume you want to use this with rootful podman because rootless would be another story.

I think netavark does only ipv6 nat at the moment and I think you want global routeable addresses?
Why would need to manually manage ndp entries? Shouldn't the kernel handle this automatically?

[mheon]

I think we do support globally-routable addresses, but I don't know if we documented the "how" of it particularly well, and we definitely aren't testing them in CI on account of not having any to work with.

[Luap99]

I do not see any conditionals in the ipv6 iptables code so i think we always NAT them.

[mheon]

Well, damn. I know we talked about it, evidently it wasn't implemented.

Should definitely get on this, as it's a regression from CNI's v6 support; @baude PTAL

[Luap99]

I don't think cni supports this either

[mheon]

We've definitely been instructing people to do this in CNI land for several years. I have to imagine at least some of them were successful?

[nivekuil]

I think netavark does only ipv6 nat at the moment and I think you want global routeable addresses? Why would need to manually manage ndp entries? Shouldn't the kernel handle this automatically?

Vanishingly few hosting providers will route the entire ipv6 subnet they assign you. Usually there needs to be
something on your host responding to the upstream NDP solicitations for each pod. Linux can proxy ndp natively but does not automatically know how to respond
to these and needs to be configured manually (see https://github.com/DanielAdolfsson/ndppd/blob/0.2.5/README,
https://yoursunny.com/t/2021/ndpresponder/)

[nivekuil]

This can be done with an OCI hook. I think the kernel will GC stale NDP entries automatically.

#!/bin/sh
in=$(</dev/stdin)
ip=$(echo $in | jq -r '.annotations | .[\"hatchery.pod.ip\"]')
if=$(/usr/sbin/ip -6 r | awk '/^default/ {printf $5}')
/usr/sbin/ip -6 neigh add proxy $ip dev $if

[nivekuil]

Yes, rootful. Vanishingly few hosting providers will actually route the entire ipv6 subnet they give you, so usually there needs to be something on your host responding to the upstream NDP solicitations for each pod. Linux does not automatically know how to respond to these and needs to be configured manually (see https://github.com/DanielAdolfsson/ndppd/blob/0.2.5/README, https://yoursunny.com/t/2021/ndpresponder/) Mar 28, 2022 4:23:32 AM Paul Holzinger ***@***.***>:

…

Moved to netavark since we would need to implement it here. I assume you want to use this with rootful podman because rootless would be another story. I think netavark does only ipv6 nat at the moment and I think you want global routeable addresses? Why would need to manually manage ndp entries? Shouldn't the kernel handle this automatically? — Reply to this email directly, view it on GitHub[#270 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/ABDLDWDIKS5OD4Q4N52PE2DVCGJDFANCNFSM5R2ZPVMQ]. You are receiving this because you authored the thread. [data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAFQAAABUCAYAAAAcaxDBAAAAAXNSR0IArs4c6QAAAARzQklUCAgICHwIZIgAAAAySURBVHic7cEBDQAAAMKg909tDjegAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAeDVulAABbzDScQAAAABJRU5ErkJggg==###24x24:true###][Tracking image][https://github.com/notifications/beacon/ABDLDWEQ2ZZZYRACAUPBUFLVCGJDFA5CNFSM5R2ZPVM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIBTZAGY.gif]

[ruabmbua]

Is there any update on this? I want to use public routable ipv6 addresses for my containers, but I can not find any documentation on how to do this with podman.

[Luap99]

Not NAt-ed global ipv6 address sounds easy enough to implement, contributions welcome.
However my ipv6 knowledge is very limited, if managing NDP entries for this is required than it looks much more complex and I don't think I have time to look into it anytime soon.

[ruabmbua]

Got it working by fiddling with network namespaces and interfaces by hand. Should be easy to implement, but e.g. integrating with systemd-networkd for prefix delegation from host interface will be more difficult.

Might start working on a patch for systemd to add an API for container runtimes if it does not exist