problem in enabling vpp dataplane on calico with kubeadm

[H4xor-hash]

Expected Behavior

I am using kubeadm and I want to enable vpp dataplane, I am using the guide on the website on the following link:
https://projectcalico.docs.tigera.io/getting-started/kubernetes/vpp/getting-started
I am expecting all the pods on kube-system and calico namespace to work properly.

Current Behavior

when I am deploying calico normally everything works perfectly BUT when I use the vpp guide some pods in the system on kube-system and calico namespace don't work. Therefore, whole system does not work

Possible Solution

I have tried on serveral different kubeadm versions such as 1.18, 1.19 , 1.20, 1.23, 1.24
I have also tried different versions of docker as well but usually using the latest docker version which doesn't seem to have a difference in the final result.
so I think maybe is there anyone who tested on one of the version of 1.21 , 1,22 , 1.23 of kubeadm and get some result back?

Steps to Reproduce (for bugs)

1. bring up 2 VMs using vagrant, 2gb ram, 2 cpu each of them
2. these 2 vms have ubnutu 18.04
3. I disable the firewall on all machines with command #ufw disable
4. and then sudo modprobe br_netfilter
5. cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
   br_netfilter
   EOF
   cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
   net.bridge.bridge-nf-call-ip6tables = 1
   net.bridge.bridge-nf-call-iptables = 1
   EOF
   net.bridge.bridge-nf-call-ip6tables = 1

6.swapoff -a; sed -i '/swap/d' /etc/fstab

7.kube int command and join worker nodes
for example
sudo kubeadm init --apiserver-advertise-address=10.244.0.102 --apiserver-cert-extra-sans=10.244.0.102 --pod-network-cidr=192.168.0.0/16
and everything work up to this step and worker node join to the master perfectly
and if I just deploy calico normally everything works but when I deploy it with the use of the guide on the website(for enabling vpp dataplane ) it does not work and some pods of the system and calico stay in a crash loop or error. I do step by step as the website tells.

8. I assign the huge pages with
   echo "vfio_pci" > /etc/modules-load.d/95-vpp.conf
   modprobe vfio_pci
   echo "vm.nr_hugepages = 512" >> /etc/sysctl.conf
   sysctl -p
   systemctl restart kubelet

9.kubectl apply -f https://projectcalico.docs.tigera.io/manifests/tigera-operator.yaml

10.kubectl apply -f https://raw.githubusercontent.com/projectcalico/vpp-dataplane/v3.23.0/yaml/calico/installation-default.yaml

11.curl -o calico-vpp.yaml https://raw.githubusercontent.com/projectcalico/vpp-dataplane/v3.23.0/yaml/generated/calico-vpp.yaml
leave every configuration as it is just change the vpp_dataplane_interface on the config file.
other configuration is the same as the configuration of my cluster such as default value of 10.96.0.0/12 for service_prefix
so I get something like
kind: ConfigMap
apiVersion: v1
metadata:
name: calico-config
namespace: calico-vpp-dataplane
data:
service_prefix: 10.96.0.0/12
vpp_dataplane_interface: enp0s8
vpp_uplink_driver: ""
...

12. I will be grateful if someone has tested vpp-dateplane of calico in some versions of kubeadm just let me know the exact version that works with calico 3.23

Context

I want to use vpp dataplane in order to implement SRv6 for communication between the pods of different worker nodes

Your Environment

• Calico version 3.23.0
• Orchestrator version (e.g. kubernetes, mesos, rkt): kubernetes version 1.18, 1.19, 1.20, 1.23, 1.24,
• Operating System and version: ubuntu server 18.04
• Link to your project (optional):

[Josh-Tigera]

You mention that some of the pods within the system and Calico crashloop or error. Can you share the error message?

[lwr20]

@sknat perhaps one for you?

[lwr20]

I will be grateful if someone has tested vpp-dateplane of calico in some versions of kubeadm just let me know the exact version that works with calico 3.23

We have overnight runs that run VPP in AWS and GCP on ubuntu 1804, k8s 1.21, calico v3.23.

If this isn't working for you, we're going to need logs from the crashing pods.

[H4xor-hash]

the problem got solved with use of following command
kubectl -n calico-vpp-dataplane logs calico-vpp-node-XXXXX -c vpp
kubectl -n calico-vpp-dataplane logs calico-vpp-node-XXXXX -c agent

I use the wrong interface for vpp

[lwr20]

Ah OK. Thanks for the update!