Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support dual-stack in klipper-lb #4021

Closed
manuelbuil opened this issue Sep 15, 2021 · 6 comments
Closed

Support dual-stack in klipper-lb #4021

manuelbuil opened this issue Sep 15, 2021 · 6 comments
Assignees
@manuelbuil @ShylajaDevadiga
Milestone
v1.22.3+k3s1

Comments

@manuelbuil
Copy link
Contributor

Is your feature request related to a problem? Please describe.

Dual-stack services in load-balanced mode are not working because klipper-lb does not support it:

+ iptables -t nat -I PREROUTING '!' -s 2001:cafe:42:1::def/32 -p TCP --dport 8080 -j DNAT --to 2001:cafe:42:1::def:8080
iptables v1.6.2: Port `cafe:42:1::def:8080' not valid

Describe the solution you'd like

klipper-lb is able to apply an ipv6 config

Additional context

#1405
@manuelbuil manuelbuil added this to the v1.22.2+k3s1 milestone Sep 15, 2021
@manuelbuil manuelbuil self-assigned this Sep 15, 2021
@manuelbuil manuelbuil mentioned this issue Sep 16, 2021
Merged
@manuelbuil
Copy link
Contributor Author

This task requires two things:
1 - Update klipper-lb so that it creates an iptable rule for ipv6 addresses as well
2 - Update k3s so that nodes get the ipv6 address of the main interface as internal ip too. Otherwise, the loadBalancer service will always have the ipv4 as the external ip

@brandond
Copy link
Member

Upstream requires that users manually specify the desired ipv6 address: they do not plan to have the kubelet or kubeadm attempt to autodetect it. I think we should follow suit.

@manuelbuil
Copy link
Contributor Author

manuelbuil commented Sep 16, 2021 •
edited
Loading

Upstream requires that users manually specify the desired ipv6 address: they do not plan to have the kubelet or kubeadm attempt to autodetect it. I think we should follow suit.

* [Dual-Stack support for Kubeadm kubernetes/kubeadm#1612 (comment)](https://github.com/kubernetes/kubeadm/issues/1612#issuecomment-786540060)

Yes, via node-ip. I discussed this with Antonio a few hours ago and I was testing this:

status:
  addresses:
  - address: 10.0.10.5
    type: InternalIP
  - address: 2a05:d012:c6f:4611:89e7:e55f:c55d:aa6
    type: InternalIP

Note that in case of no ipv4, it autodetecs ipv6. But it does not autodetect for dual-stack, which is what we want:

IP address (or comma-separated dual-stack IP addresses) of the node. If unset, kubelet will use the node's default IPv4 address, if any, or its default IPv6 address if it has no IPv4 addresses. You can pass '::' to make it prefer the default IPv6 address rather than the default IPv4 address.

@cwayne18 cwayne18 modified the milestones: v1.22.2+k3s1, v1.21.6+k3s1 Sep 16, 2021
@brandond brandond modified the milestones: v1.21.6+k3s1, v1.22.3+k3s1 Sep 21, 2021
@brandond
Copy link
Member

/backport v1.21.6+k3s1

This was referenced Sep 21, 2021
Closed
Closed
This was referenced Sep 28, 2021
Merged
Closed
Merged
Merged
@manuelbuil
Copy link
Contributor Author

How to test:

k apply -f https://gist.githubusercontent.com/aojea/90768935ab71cb31950b6a13078a7e92/raw/99ceac308f2b2658c7313198a39fbe24b155ae68/dual-stack.yaml

And then change the Type of the four services to LoadBalancer. You should see:

default       my-service-v6             LoadBalancer   2001:cafe:42:1::a62f   2a05:d012:c6f:4611:5c2:5602:eed2:898c             8080:30111/TCP               6m26s   app=MyDualApp
default       my-service-v4             LoadBalancer   10.43.218.132          10.0.10.7                                         8081:30541/TCP               6m26s   app=MyDualApp
default       my-service-prefer-dual    LoadBalancer   10.43.104.103          10.0.10.7,2a05:d012:c6f:4611:5c2:5602:eed2:898c   8082:31258/TCP               6m26s   app=MyDualApp
default       my-service-require-dual   LoadBalancer   2001:cafe:42:1::5f60   10.0.10.7,2a05:d012:c6f:4611:5c2:5602:eed2:898c   8083:30520/TCP               99s     app=MyDualApp

And be able to curl on both ipv4 and ipv6 ips. For example

curl http://[2a05:d012:c6f:4611:5c2:5602:eed2:898c]:8083
curl http://10.0.10.7:8083

@ShylajaDevadiga
Copy link
Contributor

Validated on k3s version v1.22.3-rc3+k3s1 following the steps to test on a multi-node cluster.

$ kubectl get svc -A |grep my-service
default       my-service-prefer-dual    LoadBalancer   10.43.166.221          192.168.30.181,192.168.9.14,2600:REDACTED,2600:REDACTED   8084:31224/TCP               56m
default       my-service-require-dual   LoadBalancer   10.43.202.227          192.168.30.181,192.168.9.14,2600:REDACTED,2600:REDACTED  8083:32187/TCP               56m
default       my-service-v4             LoadBalancer   10.43.38.16            192.168.30.181,192.168.9.14                               8081:31124/TCP               56m
default       my-service-v6             LoadBalancer   2001:cafe:42:1::de5a   2600:REDACTED,2600:REDACTED                               8082:31527/TCP               56m


$ curl http://192.168.30.181:8084
<html><body><h1>It works!</h1></body></html>
$ curl http://192.168.9.14:8084
<html><body><h1>It works!</h1></body></html>
$ curl http://[2600:REDACTED]:8084
<html><body><h1>It works!</h1></body></html>
$ curl http://[2600:REDACTED]:8084
<html><body><h1>It works!</h1></body></html>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@manuelbuil manuelbuil

@ShylajaDevadiga ShylajaDevadiga

Labels
None yet
Projects
None yet
Milestone
v1.22.3+k3s1
Development

No branches or pull requests
4 participants
@brandond @cwayne18 @manuelbuil @ShylajaDevadiga