Missing Cilium Support (?)

[Gerrit91]

Route injection appears not to be required with Cilium CNI.

Program is crashing on nodes configured with Cilium CNI during route injection:

❯ k logs node-init-74427
I0405 12:32:35.576475       1 init.go:51] starting node-init                                                                                                                                                                                      
F0405 12:32:35.619115       1 init.go:77] error during reconciliation, dying: %vfile exists                                                                                                                                                       
goroutine 1 [running]:                                                                                                                                                                                                                            
k8s.io/klog/v2.stacks(0x1)                                                                                                                                                                                                                        
        /go/pkg/mod/k8s.io/klog/[email protected]/klog.go:1026 +0x8a                                                                                                                                                                                     
k8s.io/klog/v2.(*loggingT).output(0x22330e0, 0x3, {0x0, 0x0}, 0xc0003b2310, 0x1, {0x1a75300, 0xc000700000}, 0xc00012e8a0, 0x0)                                                                                                                    
        /go/pkg/mod/k8s.io/klog/[email protected]/klog.go:975 +0x63d                                                                                                                                                                                     
k8s.io/klog/v2.(*loggingT).printDepth(0x0, 0x0, {0x0, 0x0}, {0x0, 0x0}, 0x0, {0xc00012e8a0, 0x2, 0x2})                                                                                                                                            
        /go/pkg/mod/k8s.io/klog/[email protected]/klog.go:735 +0x1ba                                                                                                                                                                                     
k8s.io/klog/v2.(*loggingT).print(...)                                                                                                                                                                                                             
        /go/pkg/mod/k8s.io/klog/[email protected]/klog.go:717                                                                                                                                                                                            
k8s.io/klog/v2.Fatal(...)                                                                                                                                                                                                                         
        /go/pkg/mod/k8s.io/klog/[email protected]/klog.go:1494                                                                                                                                                                                           
github.com/metal-stack/node-init/cmd.initNetwork({0xc000157d10, 0x0, 0x0})                                                                                                                                                                        
        /work/cmd/init.go:77 +0x30e                                                                                                                                                                                                               
github.com/metal-stack/node-init/cmd.glob..func1(0x221e720, {0x2263e48, 0x0, 0x0})                                                                                                                                                                
        /work/cmd/init.go:32 +0x27
github.com/spf13/cobra.(*Command).execute(0x221e720, {0x2263e48, 0x0, 0x0})
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:856 +0x60e
github.com/spf13/cobra.(*Command).ExecuteC(0x221e9a0)
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3bc
github.com/spf13/cobra.(*Command).Execute(...)
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:902
github.com/metal-stack/node-init/cmd.Execute()
        /work/cmd/root.go:27 +0x25
main.main()
        /work/main.go:16 +0x85

Turns out Cilium already puts the pod cidr on a cilium_host interface:

[mwennrich]

hm, cannot reproduce:

$ k logs node-init-vxqnn
I0414 11:58:32.878301       1 init.go:51] starting node-init
I0414 11:58:32.959042       1 init.go:120] route for 10.244.0.0/22 already exists
I0414 11:58:32.959055       1 init.go:81] waiting 2m0s for next reconciliation

but if such a route is already added by cilium, then node-init is probably not necessary anyway

[mwennrich]

$ k logs -n kube-system node-init-b29gq
I0414 13:37:45.797181       1 init.go:51] starting node-init
I0414 13:37:45.856089       1 init.go:130] route for 10.244.12.0/22 successfully added

looks like the route isn't provided by cilium

[robertvolkmann]

Cilium 1.14+ already adds the POD_CIDR as a route:

10.240.0.0/24 via 10.240.0.32 dev cilium_host proto kernel src 10.240.0.32 

Unfortunately, FRR cannot announce this route: metal-stack/releases#178