[BUG] cannot map port with protocol information

[silme]

Load Balancer crashes when a port is exposed by specifying the protocol. The reason is that the generated configuration is invalid. Here is an example with UDP (same error occurs with TCP).

$ k3d version
k3d version v3.0.0-rc.3
k3s version v1.18.3-k3s1 (default)

$ k3d create cluster --port 1234/UDP@loadbalancer
INFO[0000] Created network 'k3d-k3s-default'            
INFO[0000] Created volume 'k3d-k3s-default-images'      
INFO[0001] Creating node 'k3d-k3s-default-master-0'     
INFO[0003] Creating LoadBalancer 'k3d-k3s-default-masterlb' 
INFO[0006] Cluster 'k3s-default' created successfully!  
INFO[0006] You can now use it like this:                
export KUBECONFIG=$(k3d get kubeconfig k3s-default)
kubectl cluster-info

$ docker logs k3d-k3s-default-masterlb
2020-06-17T13:33:01Z k3d-k3s-default-masterlb confd[7]: INFO Backend set to env
2020-06-17T13:33:01Z k3d-k3s-default-masterlb confd[7]: INFO Starting confd
2020-06-17T13:33:01Z k3d-k3s-default-masterlb confd[7]: INFO Backend source(s) set to 
2020-06-17T13:33:01Z k3d-k3s-default-masterlb confd[7]: INFO /etc/nginx/nginx.conf has md5sum 9c9065b8e74f4b01f6eed5a7af1141b6 should be 36a3e18a5974521b0df47718102c58f8
2020-06-17T13:33:01Z k3d-k3s-default-masterlb confd[7]: INFO Target config /etc/nginx/nginx.conf out of sync
2020-06-17T13:33:01Z k3d-k3s-default-masterlb confd[7]: INFO Target config /etc/nginx/nginx.conf has been updated
===== Initial nginx configuration =====
error_log stderr notice;

worker_processes auto;
events {
  multi_accept on;
  use epoll;
  worker_connections 1024;
}

stream {
  upstream master_nodes_6443 {
    server k3d-k3s-default-master-0:6443 max_fails=1 fail_timeout=10s;
  }

  server {
    listen        6443;
    proxy_pass    master_nodes_6443;
    proxy_timeout 600;
    proxy_connect_timeout 2s;
  }
  upstream master_nodes_1234/UDP {
    server k3d-k3s-default-master-0:1234/UDP max_fails=1 fail_timeout=10s;
  }

  server {
    listen        1234/UDP;
    proxy_pass    master_nodes_1234/UDP;
    proxy_timeout 600;
    proxy_connect_timeout 2s;
  }
}=======================================
2020/06/17 13:33:01 [emerg] 14#14: invalid host in upstream "k3d-k3s-default-master-0:1234/UDP" in /etc/nginx/nginx.conf:22
nginx: [emerg] invalid host in upstream "k3d-k3s-default-master-0:1234/UDP" in /etc/nginx/nginx.conf:22

It should be possible to specify the protocol, at least according to k3d create cluster --help. Am I missing something?

[iwilltry42]

Hi @silme , thanks for opening this issue :)
Good catch!
Actually, the protocol definition as mentioned in the help text is only valid for the docker level (as in mapping a container port to the host with this protocol).
There's no logic (yet) to actually push such a configuration to the loadbalancer.

[iwilltry42]

Seems like this was never closed even though it's been fixed for quite some time now (#488).
The output now would look like this:

$ k3d cluster create --port 1234/UDP@loadbalancer
INFO[0000] Prep: Network                                
INFO[0000] Created network 'k3d-k3s-default' (a1df9cc939e166e269bff0068b4f93ace0e6688955146ea8ed25aefe7ca353fd) 
INFO[0000] Created volume 'k3d-k3s-default-images'      
INFO[0001] Creating node 'k3d-k3s-default-server-0'     
INFO[0002] Helper image tag set from env var            
INFO[0002] Creating LoadBalancer 'k3d-k3s-default-serverlb' 
INFO[0002] Starting cluster 'k3s-default'               
INFO[0002] Starting servers...                          
INFO[0002] Starting Node 'k3d-k3s-default-server-0'     
INFO[0007] Starting agents...                           
INFO[0007] Starting helpers...                          
INFO[0007] Starting Node 'k3d-k3s-default-serverlb'     
INFO[0008] (Optional) Trying to get IP of the docker host and inject it into the cluster as 'host.k3d.internal' for easy access 
INFO[0011] Successfully added host record to /etc/hosts in 2/2 nodes and to the CoreDNS ConfigMap 
INFO[0011] Cluster 'k3s-default' created successfully!  
INFO[0011] --kubeconfig-update-default=false --> sets --kubeconfig-switch-context=false 
INFO[0011] You can now use it like this:                
kubectl config use-context k3d-k3s-default
kubectl cluster-info

$ docker ps    
CONTAINER ID   IMAGE                      COMMAND                  CREATED          STATUS          PORTS                                                                          NAMES
1fd902090059   rancher/k3d-proxy:v4.4.4   "/bin/sh -c nginx-pr…"   11 seconds ago   Up 5 seconds    80/tcp, 0.0.0.0:49153->1234/udp, :::49153->1234/udp, 0.0.0.0:40603->6443/tcp   k3d-k3s-default-serverlb
ee3c13cf1d64   rancher/k3s:v1.21.1-k3s1   "/bin/k3s server --t…"   12 seconds ago   Up 10 seconds                                                                                  k3d-k3s-default-server-0

$ docker logs 1f                                        
2021-06-11T10:48:39Z k3d-k3s-default-serverlb confd[8]: INFO Backend set to env
2021-06-11T10:48:39Z k3d-k3s-default-serverlb confd[8]: INFO Starting confd
2021-06-11T10:48:39Z k3d-k3s-default-serverlb confd[8]: INFO Backend source(s) set to 
2021-06-11T10:48:39Z k3d-k3s-default-serverlb confd[8]: INFO /etc/nginx/nginx.conf has md5sum 9c9065b8e74f4b01f6eed5a7af1141b6 should be 04d3731aaada60d6407ca848a0a959fa
2021-06-11T10:48:39Z k3d-k3s-default-serverlb confd[8]: INFO Target config /etc/nginx/nginx.conf out of sync
2021-06-11T10:48:39Z k3d-k3s-default-serverlb confd[8]: INFO Target config /etc/nginx/nginx.conf has been updated
===== Initial nginx configuration =====
error_log stderr notice;

worker_processes auto;
events {
 multi_accept on;
 use epoll;
 worker_connections 1025;
}

stream {

 #######
 # TCP #
 #######
 upstream server_nodes_6443 {
   server k3d-k3s-default-server-0:6443 max_fails=1 fail_timeout=10s;
 }

 server {
   listen        6443;
   proxy_pass    server_nodes_6443;
   proxy_timeout 600;
   proxy_connect_timeout 2s;
 }

 #######
 # UDP #
 #######
 upstream server_nodes_udp_1234 {
   server k3d-k3s-default-server-0:1234 max_fails=1 fail_timeout=10s;
 }

 server {
   listen        1234 udp;
   proxy_pass    server_nodes_udp_1234;
   proxy_timeout 600;
   proxy_connect_timeout 2s;
 }
}
=====
....