ip6tables error

[bstegmaier75]

I tried to use ipv6nat with docker-ce 18.09.63-0debian-buster on a current Debian Testing.
A sample nginx container is running on an IPv6 enabled network configured as fd01::/64.

When I start ipv6nat (using docker-compose up) I get the following error:

ipv6nat_1 | 2019/05/29 19:52:44 running [/usr/local/bin/ip6tables -t filter -I DOCKER 1 -d fd01::2 ! -i br-4895d4b90f94 -o br-4895d4b90f94 -p tcp -m tcp --dport 80 -j ACCEPT --wait]: exit status 1: iptables: Invalid argument. Run `dmesg' for more information.

dmesg shows:

[613582.357457] x_tables: ip6_tables: tcp match: only valid for protocol 6

I am running nftables on the host machine.
Stock docker IPv4 NAT works fine.

[robbertkl]

Hi @bstegmaier75, thanks for reporting. I'm aware of the issue (see discussion at #33) and working on a fix. I'll leave this bug open until it's fixed.

[bstegmaier75]

OK, sorry, didn't read #33 until the end...
Anyway... thanks!

[robbertkl]

Yeah, sorry, #33 was closed prematurely 😉

[robbertkl]

@bstegmaier75 could you pull the latest image and try again? Thanks!

[bstegmaier75]

Seems to work fine now, thanks!

It doesn't work to connect to the webserver, but from what I see IPv6 config of my local interfaces seems to be completely broken (I tried various things)... I have to fix that first.

That's what has been generated for a single port 8000:80 mapping:

table ip6 filter {
chain INPUT {
type filter hook input priority 0; policy accept;
}

chain FORWARD {
type filter hook forward priority 0; policy accept;
counter packets 8 bytes 448 jump DOCKER-ISOLATION
oifname "br-abc0cbaf4052" counter packets 0 bytes 0 jump DOCKER
oifname "br-abc0cbaf4052" ct state related,established counter packets 0 bytes 0 accept
iifname "br-abc0cbaf4052" oifname != "br-abc0cbaf4052" counter packets 0 bytes 0 accept
iifname "br-abc0cbaf4052" oifname "br-abc0cbaf4052" counter packets 0 bytes 0 accept
}

chain OUTPUT {
type filter hook output priority 0; policy accept;
}

chain DOCKER {
iifname != "br-abc0cbaf4052" oifname "br-abc0cbaf4052" meta l4proto tcp ip6 daddr fd01::2 tcp dport 80 counter packets 0 bytes 0 accept
}

chain DOCKER-ISOLATION {
counter packets 8 bytes 448 return
}
}
table ip6 nat {
chain PREROUTING {
type nat hook prerouting priority -100; policy accept;
fib daddr type local counter packets 0 bytes 0 jump DOCKER
}

chain INPUT {
type nat hook input priority 100; policy accept;
}

chain POSTROUTING {
type nat hook postrouting priority 100; policy accept;
oifname != "br-abc0cbaf4052" ip6 saddr fd01::/64 counter packets 0 bytes 0 masquerade
meta l4proto tcp ip6 saddr fd01::2 ip6 daddr fd01::2 tcp dport 80 counter packets 0 bytes 0 masquerade
}

chain OUTPUT {
type nat hook output priority -100; policy accept;
fib daddr type local counter packets 0 bytes 0 jump DOCKER
}

chain DOCKER {
meta l4proto tcp tcp dport 8000 counter packets 0 bytes 0 dnat to [fd01::2]:80
}
}

[bstegmaier75]

Reboot seems to have fixed IPv6 config for all interfaces.
But, still can't connect to webserver via IPv6 address.
I can ping the host with its IPv6, so in general IPv6 setup should be OK.

I am no expert on nftables and I still have find out yet how to setup the NAT rules manually, but there is one thing I am not sure about:

In several places there is something like that:

  meta l4proto tcp ip6 saddr fd01::2 ip6 daddr fd01::2 tcp dport 80 counter packets 0 bytes 0 masquerade  

Interesting is "... meta l4proto tcp ip6 saddr fd01::2 ..."

In DOCKER chain there is

  meta l4proto tcp tcp dport 8000 counter packets 10 bytes 840 dnat to [fd01::2]:80

Here there is "... meta l4proto tcp tcp dport 8000 ..." instead of tcp ip6

Might that be a problem?

[robbertkl]

I don't think that's the problem, those are matches and they're split up a bit differently:

meta l4proto tcp ip6 saddr fd01::2 ip6 daddr fd01::2 tcp dport 80 counter packets 0 bytes 0 masquerade:

• meta l4proto tcp
• ip6 saddr fd01::2
• ip6 daddr fd01::2
• tcp dport 80
• ...

meta l4proto tcp tcp dport 8000 counter packets 10 bytes 840 dnat to [fd01::2]:80:

• meta l4proto tcp
• tcp dport 8000
• `...

What's the output of ip6tables-save? (this translates the underlying nft rules back to iptables format)

[bstegmaier75]

ip6tables-save on host:

# Generated by xtables-save v1.8.2 on Thu May 30 13:56:18 2019
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-ISOLATION - [0:0]
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o br-86624d2f14b9 -j DOCKER
-A FORWARD -o br-86624d2f14b9 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-86624d2f14b9 ! -o br-86624d2f14b9 -j ACCEPT
-A FORWARD -i br-86624d2f14b9 -o br-86624d2f14b9 -j ACCEPT
-A DOCKER -d fd01::2/128 ! -i br-86624d2f14b9 -o br-86624d2f14b9 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER-ISOLATION -j RETURN
COMMIT
# Completed on Thu May 30 13:56:18 2019
# Generated by xtables-save v1.8.2 on Thu May 30 13:56:18 2019
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s fd01::/64 ! -o br-86624d2f14b9 -j MASQUERADE
-A POSTROUTING -s fd01::2/128 -d fd01::2/128 -p tcp -m tcp --dport 80 -j MASQUERADE
-A OUTPUT -m addrtype --dst-type LOCAL -j DOCKER
-A DOCKER -p tcp -m tcp --dport 8000 -j DNAT --to-destination [fd01::2]:80
COMMIT
# Completed on Thu May 30 13:56:18 2019

Looks pretty much the same as the IPv4 stuff, as far as I can see.

Route should be also OK:

fd01::/64                      [::]                       U    256 3     0 br-86624d2f14b9
fd01::/128                     [::]                       Un   0   3     0 br-86624d2f14b9
fd01::1/128                    [::]                       Un   0   4     0 br-86624d2f14b9

[robbertkl]

Yeah, this looks OK. Are you sure the problem is with docker-ipv6nat or could it be something else in your IPv6 setup? Was everything working properly before you upgraded to buster?

[bstegmaier75]

No, not sure that it is with ipv6nat.
I (tried to) set it up for the first time yesterday.

I checked the counters of the various rules and they don't change when I try to connect, so it seems as if I don't even reach the host and I probably have some problem with my IPv6 setup.

Just to be sure: I have my router configured to announce a fd00:: ULA to my local network and the docker network to be fd01::. That's how it is supposed to be?

[robbertkl]

I'm not familiar with a setup like that, but it should be good as long as the ULA prefixes are different. I (and most users) have a publicly available IPv6 address on a server and use a ULA for the Docker with NAT.

[robbertkl]

Try to get basic IPv6 connectivity working first, can you ping6 from the host?

I'll close the issue now, since the original error was resolved. I've got confirmation in #33 that everything is working now with buster/nftables. Feel free to continue discussion here to get your setup working.

[bstegmaier75]

Silly me.
Some leftover firewall rules from my before-docker setup caught traffic from docker network to host.
Removed them and everything is working as it should.

Thanks for the support!

[robbertkl]

Great! Have a good day.