use usespace vdpa can't find the pfName because the net path is wrong #507

JasinlikeEatingOrange · 2023-11-08T14:46:48Z

What happened?

There is no net path in Virtio-net is the net layer of virtio, and the standard kernel driver places this layer separately into the virtio1 path

What did you expect to happen?

If the net folder is not found in the /sys/bus/pci/devices/pciAddr/physfn/net path, I hope you can find the folder starting with virtio again

What are the minimal steps needed to reproduce the bug?

No**/sys/bus/pci/devices/pciAddr/physfn/net**
Only the /sys/bus/pci/devices/pciAddr virtioxx/physfn/net

Anything else we need to know?

Component Versions

Please fill in the below table with the version numbers of components used.

Component	Version
SR-IOV Network Device Plugin	`0bf8f4f`
SR-IOV CNI Plugin	`0bf8f4f`
Multus
Kubernetes	v1.24.0
OS	ubuntu 22.04

Config Files

Config file locations may be config dependent.

Device pool config file location (Try '/etc/pcidp/config.json')

Multus config (Try '/etc/cni/multus/net.d')

CNI config (Try '/etc/cni/net.d/')

Kubernetes deployment type ( Bare Metal, Kubeadm etc.)

Kubeconfig file

SR-IOV Network Custom Resource Definition

Logs

SR-IOV Network Device Plugin Logs (use `kubectl logs $PODNAME`)

root@master:~/temp/kubevirt# I1108 14:43:33.442520 I1108 14:43:33.442648 I1108 14:43:33.442685 "resourceList": [{
"resourceName": "sriov_dpu",
"resourcePrefix": "yusur.tech",
"selectors": {
"rootDevices": ["0000:01:00.0#1-6"]
}
}
]
}
I1108 14:43:33.442759 I1108 14:43:33.442781 I1108 14:43:33.442815 I1108 14:43:33.442818 I1108 14:43:33.469296 I1108 14:43:33.469535 I1108 14:43:33.469605 I1108 14:43:33.469659 I1108 14:43:33.469719 I1108 14:43:33.469767 I1108 14:43:33.469814 I1108 14:43:33.469875 I1108 14:43:33.469931 I1108 14:43:33.469990 I1108 14:43:33.470009 I1108 14:43:33.470035 I1108 14:43:33.470045 I1108 14:43:33.470054 I1108 14:43:33.470064 I1108 14:43:33.470075 I1108 14:43:33.470086 I1108 14:43:33.470097 I1108 14:43:33.470106 I1108 14:43:33.470127 I1108 14:43:33.470151 I1108 14:43:33.470157 I1108 14:43:33.470160 I1108 14:43:33.470162 I1108 14:43:33.470166 I1108 14:43:33.470339 I1108 14:43:33.470511 I1108 14:43:33.470694 I1108 14:43:33.471309 I1108 14:43:33.471644 I1108 14:43:33.471959 I1108 14:43:33.472276 I1108 14:43:33.472473 I1108 14:43:33.472583 W1108 14:43:33.472592 I1108 14:43:33.472827 W1108 14:43:33.472837 I1108 14:43:33.472977 W1108 14:43:33.472984 I1108 14:43:33.473145 W1108 14:43:33.473154 I1108 14:43:33.473338 W1108 14:43:33.473347 I1108 14:43:33.473572 W1108 14:43:33.473580 I1108 14:43:33.473825 W1108 14:43:33.473834 I1108 14:43:33.474096 W1108 14:43:33.474104 I1108 14:43:33.474171 W1108 14:43:33.474178 I1108 14:43:33.474239 W1108 14:43:33.474246 I1108 14:43:33.474261 I1108 14:43:33.474263 I1108 14:43:33.474265 I1108 14:43:33.474267 I1108 14:43:33.474271 I1108 14:43:33.474273 I1108 14:43:33.474281 I1108 14:43:33.474285 I1108 14:43:33.474446 I1108 14:43:33.474818 I1108 14:43:33.474842 I1108 14:43:33.474848 I1108 14:43:33.964954 I1108 14:43:33.964962 I1108 14:43:33.964969 kubectl logs -f -n kube-system kube-sriov-device-plugin-amd64-d6vsq
1 manager.go:51] Using Kubelet Plugin Registry Mode
1 main.go:44] resource manager reading configs
1 manager.go:79] raw ResourceList: {
1 factory.go:166] net device selector for resource sriov_dpu is &{DeviceSelectors:{Vendors:[] Devices:[] Drivers:[] PciAddresses:[]} PfNames:[] RootDevices:[0000:01:00.0#1-6] LinkTypes:[] DDPProfiles:[] IsRdma:false NeedVhostNet:false}
1 manager.go:99] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice Selectors:0xc00000cd38 SelectorObj:0xc0001b32b0}]
1 manager.go:200] validating resource name "yusur.tech/sriov_dpu"
1 main.go:60] Discovering host devices
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02 unknown unknown
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02 unknown unknown
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02 unknown unknown
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02 unknown unknown
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02 unknown unknown
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02 unknown unknown
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02 unknown unknown
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02 unknown unknown
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02 unknown unknown
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02 Red Hat, Inc. Virtio network device
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.0 02 Red Hat, Inc. Virtio network device
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.1 02 Red Hat, Inc. Virtio network device
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.2 02 Red Hat, Inc. Virtio network device
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.3 02 Red Hat, Inc. Virtio network device
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.4 02 Red Hat, Inc. Virtio network device
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.5 02 Red Hat, Inc. Virtio network device
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.6 02 Red Hat, Inc. Virtio network device
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.7 02 Red Hat, Inc. Virtio network device
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:02:00.0 02 Red Hat, Inc. Virtio network device
1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:10:00.0 02 Red Hat, Inc. Virtio network device
1 main.go:66] Initializing resource servers
1 manager.go:105] number of config: 1
1 manager.go:109]
1 manager.go:110] Creating new ResourcePool: sriov_dpu
1 manager.go:111] DeviceType: netDevice
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0b.0. error getting devlink device attributes for net device 0000:00:0b.0 no such device
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0c.0. error getting devlink device attributes for net device 0000:00:0c.0 no such device
1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:00:0c.0/net: no such file or directory"
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.1/net: no such file or directory"
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.2/net: no such file or directory"
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.3/net: no such file or directory"
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.4/net: no such file or directory"
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.6/net: no such file or directory"
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.7/net: no such file or directory"
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:02:00.0. error getting devlink device attributes for net device 0000:02:00.0 no such device
1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:02:00.0/net: no such file or directory"
1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:10:00.0. error getting devlink device attributes for net device 0000:10:00.0 no such device
1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:10:00.0/net: no such file or directory"
1 factory.go:106] device added: [pciAddr: 0000:01:00.2, vendor: 1af4, device: 1041, driver: virtio-pci]
1 factory.go:106] device added: [pciAddr: 0000:01:00.3, vendor: 1af4, device: 1041, driver: virtio-pci]
1 factory.go:106] device added: [pciAddr: 0000:01:00.4, vendor: 1af4, device: 1041, driver: virtio-pci]
1 factory.go:106] device added: [pciAddr: 0000:01:00.5, vendor: 1af4, device: 1041, driver: virtio-pci]
1 factory.go:106] device added: [pciAddr: 0000:01:00.6, vendor: 1af4, device: 1041, driver: virtio-pci]
1 factory.go:106] device added: [pciAddr: 0000:01:00.7, vendor: 1af4, device: 1041, driver: virtio-pci]
1 manager.go:139] New resource server is created for sriov_dpu ResourcePool
1 main.go:72] Starting all servers...
1 server.go:199] starting sriov_dpu device plugin endpoint at: yusur.tech_sriov_dpu.sock
1 server.go:226] sriov_dpu device plugin endpoint started serving
1 main.go:77] All servers started.
1 main.go:78] Listening for term signals
1 server.go:110] Plugin: yusur.tech_sriov_dpu.sock gets registered successfully at Kubelet
1 server.go:134] ListAndWatch(sriov_dpu) invoked
1 server.go:142] ListAndWatch(sriov_dpu): send devices &ListAndWatchResponse{Devices:[]*Device{&Device{ID:0000:01:00.6,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.7,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.2,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.3,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.4,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.5,Health:Healthy,Topology:nil,},},}

Multus logs (If enabled. Try '/var/log/multus.log' )

Kubelet logs (journalctl -u kubelet)

The text was updated successfully, but these errors were encountered:

adrianchiris · 2023-11-09T09:28:20Z

I dont understand the issue,

device plugin is configured using rootDevices selector and is reporting 6 devices under ysur.tech/sriov_dpu
it emits warning in logs because it did not find netdevice for the PFs which is expected in your case since the PF driver did not create a netdevice.

CNI is failing since it failed to find netdevice.

JasinlikeEatingOrange · 2023-11-16T06:26:58Z

I dont understand the issue,

device plugin is configured using rootDevices selector and is reporting 6 devices under ysur.tech/sriov_dpu it emits warning in logs because it did not find netdevice for the PFs which is expected in your case since the PF driver did not create a netdevice.

CNI is failing since it failed to find netdevice.

Sorry, let me rephrase.My current problem is that pod can’t find pfname

1. Situation

Now the directory of our virtio device on this machine is as follows.
/sys/bus/pci/devices/0000:01:00.5/physfn/virtio5/net

The current version of sriov reads the pfname of this machine as shown below.

it such as /sys/bus/pci/devices/0000:01:00.5/physfn/net
It cannot find the net directory of this device, so it cannot be placed in the resource pool.

2. Solution

I added an extra logical judgment to the path here, and now it can be successfully adapted to our hardware equipment. We are a Chinese DPU equipment company YUSUR. Could you please evaluate whether it’s necessary to integrate this into the community?

adrianchiris · 2023-11-16T10:57:33Z

PF name the the PF netdev name of the PCI device.
what you suggest is to support in device plugin to take the name of the PF netdev is from a virtio device

saying in the topic the path is "wrong" sort of hints that device plugin behaves incorrectly, which is not the case.

I still dont understand what it is that you are seeking. currently device plugin will indeed fail to get PF name, but would still expose the resources.

if you want to make this virtio netdev available in workload pod then you need support in CNI not device plugin.

can you join one of our community meetings and present your use-case ? this would help

JasinlikeEatingOrange · 2023-11-23T05:05:39Z

Certainly, I can attend the meeting. Meanwhile, please review my reproduction on your end. Please refrain from referring to my previous responses; focus on the replies I provide below.

1.Current original version:

#Local situation, where the node node1vdpa is a DPU node. I want to use our SR-IOV plugin to obtain the corresponding VF resources.
root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl  get pods -A -owide
NAMESPACE        NAME                                   READY   STATUS    RESTARTS       AGE     IP               NODE        NOMINATED NODE   READINESS GATES
kube-system      coredns-bcb458c48-kxlbh                1/1     Running   0              7m10s   10.244.1.3       master      <none>           <none>
kube-system      coredns-bcb458c48-qwqfn                1/1     Running   0              7m10s   10.244.1.4       master      <none>           <none>
kube-system      etcd-master                            1/1     Running   23 (60m ago)   45h     192.168.201.20   master      <none>           <none>
kube-system      kube-apiserver-master                  1/1     Running   23 (60m ago)   45h     192.168.201.20   master      <none>           <none>
kube-system      kube-controller-manager-master         1/1     Running   23 (60m ago)   45h     192.168.201.20   master      <none>           <none>
kube-system      kube-multus-ds-cd7nn                   1/1     Running   0              7m9s    192.168.201.20   master      <none>           <none>
kube-system      kube-multus-ds-nntrh                   1/1     Running   0              7m9s    192.168.201.21   node1vdpa   <none>           <none>
kube-system      kube-scheduler-master                  1/1     Running   22 (60m ago)   45h     192.168.201.20   master      <none>           <none>
kube-system      kube-sriov-device-plugin-amd64-5p72l   1/1     Running   0              27s     192.168.201.21   node1vdpa   <none>           <none>
kube-system      kube-sriov-device-plugin-amd64-vgd9j   1/1     Running   0              7m9s    192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-db-77b8bbd66f-ccwpk            2/2     Running   0              11m     192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-master-78b7865c59-xntjb        2/2     Running   0              10m     192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-node-4vs7s                     3/3     Running   0              9m12s   192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-node-dpu-host-vdpa-cv7m2       1/1     Running   0              8m11s   192.168.201.21   node1vdpa   <none>           <none>
ovn-kubernetes   ovs-node-2w6qr                         1/1     Running   0              12m     192.168.201.20   master      <none>           <none>

Let's first take a look at the log information for the SR-IOV pod.

#SR-IOV log information

root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl logs -f -n kube-system      kube-sriov-device-plugin-amd64-5p72l
I1123 02:59:05.191954       1 manager.go:51] Using Kubelet Plugin Registry Mode
I1123 02:59:05.192234       1 main.go:44] resource manager reading configs
I1123 02:59:05.192262       1 manager.go:79] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
           "pfNames": ["enp1s0#1-6"]
        }
      }
  ]
}
I1123 02:59:05.192348       1 factory.go:166] net device selector for resource sriov_dpu is &{DeviceSelectors:{Vendors:[] Devices:[] Drivers:[] PciAddresses:[]} PfNames:[enp1s0#1-6] RootDevices:[] LinkTypes:[] DDPProfiles:[] IsRdma:false NeedVhostNet:false}
I1123 02:59:05.192370       1 manager.go:99] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice Selectors:0xc00000cd38 SelectorObj:0xc000243110}]
I1123 02:59:05.192408       1 manager.go:200] validating resource name "yusur.tech/sriov_dpu"
I1123 02:59:05.192412       1 main.go:60] Discovering host devices
I1123 02:59:05.269156       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1123 02:59:05.275501       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1123 02:59:05.275589       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1123 02:59:05.275653       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1123 02:59:05.275702       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1123 02:59:05.275752       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1123 02:59:05.275799       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1123 02:59:05.275864       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1123 02:59:05.275920       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1123 02:59:05.275984       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276007       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276029       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276042       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276052       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276061       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276072       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276082       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276093       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276210       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276239       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276291       1 main.go:66] Initializing resource servers
I1123 02:59:05.276435       1 manager.go:105] number of config: 1
I1123 02:59:05.276441       1 manager.go:109]
I1123 02:59:05.276444       1 manager.go:110] Creating new ResourcePool: sriov_dpu
I1123 02:59:05.276446       1 manager.go:111] DeviceType: netDevice
I1123 02:59:05.276764       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.276982       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.277159       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.277376       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.277596       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.277821       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.278080       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 02:59:05.278169       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0b.0. error getting devlink device attributes for net device 0000:00:0b.0 no such device
I1123 02:59:05.278261       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0c.0. error getting devlink device attributes for net device 0000:00:0c.0 no such device
W1123 02:59:05.278273       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:00:0c.0/net: no such file or directory"
I1123 02:59:05.278479       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.278489       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.1/net: no such file or directory"
I1123 02:59:05.278618       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.278627       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.2/net: no such file or directory"
I1123 02:59:05.278768       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.278777       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.3/net: no such file or directory"
I1123 02:59:05.278941       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.278952       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.4/net: no such file or directory"
I1123 02:59:05.279140       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.279149       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"
I1123 02:59:05.279351       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.279359       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.6/net: no such file or directory"
I1123 02:59:05.279603       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1123 02:59:05.279612       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.7/net: no such file or directory"
I1123 02:59:05.279670       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:02:00.0. error getting devlink device attributes for net device 0000:02:00.0 no such device
W1123 02:59:05.279679       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:02:00.0/net: no such file or directory"
I1123 02:59:05.279738       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:10:00.0. error getting devlink device attributes for net device 0000:10:00.0 no such device
W1123 02:59:05.279747       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:10:00.0/net: no such file or directory"
I1123 02:59:05.279755       1 manager.go:125] no devices in device pool, skipping creating resource server for sriov_dpu
I1123 02:59:05.279758       1 main.go:72] Starting all servers...
I1123 02:59:05.279760       1 main.go:77] All servers started.
I1123 02:59:05.279763       1 main.go:78] Listening for term signals

it will throw error "unable to get PF name "open /sys/bus/pci/devices/0000:02:00.0/net: no such file or directory"
Below is the node resource information. We can see that the node has not obtained SR-IOV resource information: yusur.tech/sriov_dpu: 0

#Node Resource Information
root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl describe node node1vdpa
Name:               node1vdpa
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    k8s.ovn.org/dpu-host=
                    k8s.ovn.org/vdpa-dpu-host=
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=node1vdpa
                    kubernetes.io/os=linux
                    node=node1
                    yusur-type=dpu
Annotations:        k8s.ovn.org/host-addresses: ["172.17.0.1","192.168.124.1","192.168.200.35","192.168.201.101"]
                    k8s.ovn.org/l3-gateway-config:
                      {"default":{"mode":"shared","interface-id":"brp0_node1vdpa","mac-address":"00:15:6d:ee:ce:cb","ip-addresses":["192.168.201.21/24"],"ip-add...
                    k8s.ovn.org/node-chassis-id: 58674eff-bed3-4cec-837d-afc88df23dc4
                    k8s.ovn.org/node-gateway-router-lrp-ifaddr: {"ipv4":"100.64.0.3/16"}
                    k8s.ovn.org/node-mgmt-port-mac-address: 0a:58:0a:f4:00:02
                    k8s.ovn.org/node-primary-ifaddr: {"ipv4":"192.168.201.21/24"}
                    k8s.ovn.org/node-subnets: {"default":"10.244.0.0/24"}
                    kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/containerd/containerd.sock
                    node.alpha.kubernetes.io/ttl: 0
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Tue, 21 Nov 2023 13:51:38 +0800
Taints:             <none>
Unschedulable:      false
Lease:
  HolderIdentity:  node1vdpa
  AcquireTime:     <unset>
  RenewTime:       Thu, 23 Nov 2023 11:04:08 +0800
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Thu, 23 Nov 2023 10:59:16 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Thu, 23 Nov 2023 10:59:16 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Thu, 23 Nov 2023 10:59:16 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Thu, 23 Nov 2023 10:59:16 +0800   Thu, 23 Nov 2023 10:51:27 +0800   KubeletReady                 kubelet is posting ready status. AppArmor enabled
Addresses:
  InternalIP:  192.168.201.21
  Hostname:    node1vdpa
Capacity:
  cpu:                   8
  ephemeral-storage:     24590672Ki
  hugepages-1Gi:         0
  hugepages-2Mi:         1200Mi
  memory:                8127040Ki
  pods:                  110
  yusur.tech/sriov_dpu:  6
Allocatable:
  cpu:                   8
  ephemeral-storage:     22662763278
  hugepages-1Gi:         0
  hugepages-2Mi:         1200Mi
  memory:                6795840Ki
  pods:                  110
  yusur.tech/sriov_dpu:  0
System Info:
  Machine ID:                 eabba479306945cfb24292224ef23d1a
  System UUID:                da7512b0-6152-49be-9bf6-de13d986c2c7
  Boot ID:                    a2a0a5dc-308e-4db6-a362-8c3d7a855a3a
  Kernel Version:             5.15.0-87-generic
  OS Image:                   Ubuntu 20.04.6 LTS
  Operating System:           linux
  Architecture:               amd64
  Container Runtime Version:  containerd://1.6.19
  Kubelet Version:            v1.24.2
  Kube-Proxy Version:         v1.24.2
PodCIDR:                      10.244.1.0/24
PodCIDRs:                     10.244.1.0/24
Non-terminated Pods:          (3 in total)
  Namespace                   Name                                    CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------                   ----                                    ------------  ----------  ---------------  -------------  ---
  kube-system                 kube-multus-ds-nntrh                    100m (1%)     100m (1%)   50Mi (0%)        50Mi (0%)      11m
  kube-system                 kube-sriov-device-plugin-amd64-5p72l    250m (3%)     1 (12%)     40Mi (0%)        200Mi (3%)     5m7s
  ovn-kubernetes              ovnkube-node-dpu-host-vdpa-cv7m2        100m (1%)     0 (0%)      300Mi (4%)       0 (0%)         12m
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource              Requests     Limits
  --------              --------     ------
  cpu                   450m (5%)    1100m (13%)
  memory                390Mi (5%)   250Mi (3%)
  ephemeral-storage     0 (0%)       0 (0%)
  hugepages-1Gi         0 (0%)       0 (0%)
  hugepages-2Mi         300Mi (25%)  300Mi (25%)
  yusur.tech/sriov_dpu  0            0
Events:
  Type    Reason        Age                From     Message
  ----    ------        ----               ----     -------
  Normal  NodeNotReady  27m (x2 over 15h)  kubelet  Node node1vdpa status is now: NodeNotReady
  Normal  NodeReady     12m (x4 over 45h)  kubelet  Node node1vdpa status is now: NodeReady

In other words, before modifying the code version, our SR-IOV cannot obtain node information in the current situation when using selectors for pfNames. At the same time, it will throw an error, "unable to get PF name "open /sys/bus/pci/devices/0000:01:00.3/net: no such file or directory". To understand the error situation, let's go to the node node1vdpa and investigate what's going on.

2. Node information for node1vdpa

Let's first take a look at the top-level directory. Currently, we are on the node1vdpa node.

root@node1vdpa:/sys/bus/pci/devices# ls
0000:00:00.0  0000:00:01.2  0000:00:01.5  0000:00:02.0  0000:00:02.3  0000:00:02.6  0000:00:0a.0  0000:00:0a.3  0000:00:0a.6  0000:00:0c.0  0000:00:1f.2  0000:01:00.1  0000:01:00.4  0000:01:00.7  0000:0b:00.0  0000:0e:00.0
0000:00:01.0  0000:00:01.3  0000:00:01.6  0000:00:02.1  0000:00:02.4  0000:00:02.7  0000:00:0a.1  0000:00:0a.4  0000:00:0a.7  0000:00:1b.0  0000:00:1f.3  0000:01:00.2  0000:01:00.5  0000:02:00.0  0000:0c:00.0  0000:10:00.0
0000:00:01.1  0000:00:01.4  0000:00:01.7  0000:00:02.2  0000:00:02.5  0000:00:03.0  0000:00:0a.2  0000:00:0a.5  0000:00:0b.0  0000:00:1f.0  0000:01:00.0  0000:01:00.3  0000:01:00.6  0000:09:00.0  0000:0d:00.0


root@node1vdpa:/sys/bus/pci/devices# cd  /sys/bus/pci/devices/0000:01:00.3/

root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3# ls
ari_enabled           config                    current_link_width  dma_mask_bits    enable       irq            local_cpus      modalias  numa_node  power_state   resource   resource4_wc         subsystem         uevent
broken_parity_status  consistent_dma_mask_bits  d3cold_allowed      driver           iommu        link           max_link_speed  msi_bus   physfn     reset         resource1  revision             subsystem_device  vendor
class                 current_link_speed        device              driver_override  iommu_group  local_cpulist  max_link_width  msi_irqs  power      reset_method  resource4  sriov_vf_msix_count  subsystem_vendor  virtio10

After entering the directory /sys/bus/pci/devices/0000:01:00.3, we can indeed see that the net folder is not present. Let's take a look at how the code is implemented in the project to understand this.

We can see that the current code navigates into the physfn directory within /sys/bus/pci/devices/0000:01:00.3 to look for the net folder.

#let's see physfn dir
root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3# cd physfn

# we cant find the net dir meanwhile we can find virtio1
root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3/physfn# ls
ari_enabled           consistent_dma_mask_bits  device           enable         irq            max_link_speed  msi_irqs     remove        resource      revision                 sriov_offset     sriov_vf_total_msix  uevent   virtfn2  virtfn6
broken_parity_status  current_link_speed        dma_mask_bits    firmware_node  link           max_link_width  numa_node    rescan        resource1     rom                      sriov_stride     subsystem            vendor   virtfn3  virtio1
class                 current_link_width        driver           iommu          local_cpulist  modalias        power        reset         resource4     sriov_drivers_autoprobe  sriov_totalvfs   subsystem_device     virtfn0  virtfn4
config                d3cold_allowed            driver_override  iommu_group    local_cpus     msi_bus         power_state  reset_method  resource4_wc  sriov_numvfs             sriov_vf_device  subsystem_vendor     virtfn1  virtfn5

It's evident that the net folder is not present in the current directory, so it cannot find the corresponding resources. As a result, the local SR-IOV resources are reported as 0.

root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3/physfn# ethtool -i enp1s0
driver: virtio_net
version: 1.0.0
firmware-version:
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

3. Solution

When the driver is virtio, we can observe that, in the directory /sys/bus/pci/devices/0000:01:00.3/physfn, an additional folder named virtioxx is present. Let's enter that folder.

root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3/physfn# ls
ari_enabled           consistent_dma_mask_bits  device           enable         irq            max_link_speed  msi_irqs     remove        resource      revision                 sriov_offset     sriov_vf_total_msix  uevent   virtfn2  virtfn6
broken_parity_status  current_link_speed        dma_mask_bits    firmware_node  link           max_link_width  numa_node    rescan        resource1     rom                      sriov_stride     subsystem            vendor   virtfn3  virtio1
class                 current_link_width        driver           iommu          local_cpulist  modalias        power        reset         resource4     sriov_drivers_autoprobe  sriov_totalvfs   subsystem_device     virtfn0  virtfn4
config                d3cold_allowed            driver_override  iommu_group    local_cpus     msi_bus         power_state  reset_method  resource4_wc  sriov_numvfs             sriov_vf_device  subsystem_vendor     virtfn1  virtfn5

# virtio1 is our focus
root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3/physfn# cd virtio1/

root@node1vdpa:/sys/bus/pci/devices/0000:01:00.3/physfn/virtio1# ls
device  driver  features  modalias  net  power  status  subsystem  uevent  vendor

root@node1vdpa:/sys/bus/pci/devices/0000:01:00.6/physfn/virtio1# ls net/
enp1s0

Great, we can now see the net folder with enp1s0 inside, which is the pfName we need. So, the code can be modified as follows:

4. The effect after the modification.

#I have already switched to the image with the modified code on node1vdpa. Now, let's delete the existing pod 
root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl delete po -n kube-system      kube-sriov-device-plugin-amd64-5p72l --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "kube-sriov-device-plugin-amd64-5p72l" force deleted

#get pods
root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl get pods -A -owide
NAMESPACE        NAME                                   READY   STATUS    RESTARTS       AGE   IP               NODE        NOMINATED NODE   READINESS GATES
kube-system      coredns-bcb458c48-kxlbh                1/1     Running   0              45m   10.244.1.3       master      <none>           <none>
kube-system      coredns-bcb458c48-qwqfn                1/1     Running   0              45m   10.244.1.4       master      <none>           <none>
kube-system      etcd-master                            1/1     Running   23 (98m ago)   45h   192.168.201.20   master      <none>           <none>
kube-system      kube-apiserver-master                  1/1     Running   23 (98m ago)   45h   192.168.201.20   master      <none>           <none>
kube-system      kube-controller-manager-master         1/1     Running   23 (98m ago)   45h   192.168.201.20   master      <none>           <none>
kube-system      kube-multus-ds-cd7nn                   1/1     Running   0              45m   192.168.201.20   master      <none>           <none>
kube-system      kube-multus-ds-nntrh                   1/1     Running   0              45m   192.168.201.21   node1vdpa   <none>           <none>
kube-system      kube-scheduler-master                  1/1     Running   22 (98m ago)   45h   192.168.201.20   master      <none>           <none>
kube-system      kube-sriov-device-plugin-amd64-8g52f   1/1     Running   0              3s    192.168.201.21   node1vdpa   <none>           <none>
kube-system      kube-sriov-device-plugin-amd64-vgd9j   1/1     Running   0              45m   192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-db-77b8bbd66f-ccwpk            2/2     Running   0              49m   192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-master-78b7865c59-xntjb        2/2     Running   0              48m   192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-node-4vs7s                     3/3     Running   0              47m   192.168.201.20   master      <none>           <none>
ovn-kubernetes   ovnkube-node-dpu-host-vdpa-cv7m2       1/1     Running   0              46m   192.168.201.21   node1vdpa   <none>           <none>
ovn-kubernetes   ovs-node-2w6qr                         1/1     Running   0              50m   192.168.201.20   master      <none>           <none>

#check the pod logs.
root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl logs -f -n kube-system      kube-sriov-device-plugin-amd64-8g52f
I1123 03:37:46.852262       1 manager.go:57] Using Kubelet Plugin Registry Mode
I1123 03:37:46.852408       1 main.go:46] resource manager reading configs
I1123 03:37:46.852435       1 manager.go:86] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
           "pfNames": ["enp1s0#1-6"]
        }
      }
  ]
}
I1123 03:37:46.852517       1 factory.go:209] *types.NetDeviceSelectors for resource sriov_dpu is [0xc0002a0e00]
I1123 03:37:46.852526       1 manager.go:106] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice ExcludeTopology:false Selectors:0xc00041a840 AdditionalInfo:map[] SelectorObjs:[0xc0002a0e00]}]
I1123 03:37:46.852554       1 manager.go:217] validating resource name "yusur.tech/sriov_dpu"
I1123 03:37:46.852562       1 main.go:62] Discovering host devices
I1123 03:37:46.891928       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1123 03:37:46.892463       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1123 03:37:46.892618       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1123 03:37:46.892702       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1123 03:37:46.892775       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1123 03:37:46.892840       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1123 03:37:46.892903       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1123 03:37:46.892966       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1123 03:37:46.893028       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1123 03:37:46.893111       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1123 03:37:46.893135       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893164       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893176       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893186       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893197       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893208       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893221       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893234       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893248       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893276       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893304       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.0   02              unknown                 unknown
I1123 03:37:46.893321       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.1   02              unknown                 unknown
I1123 03:37:46.893324       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.2   02              unknown                 unknown
I1123 03:37:46.893327       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.3   02              unknown                 unknown
I1123 03:37:46.893330       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.4   02              unknown                 unknown
I1123 03:37:46.893335       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.5   02              unknown                 unknown
I1123 03:37:46.893338       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.6   02              unknown                 unknown
I1123 03:37:46.893341       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.7   02              unknown                 unknown
I1123 03:37:46.893346       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0b.0   02              unknown                 unknown
I1123 03:37:46.893349       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0c.0   02              Red Hat, Inc.           Virtio network device
I1123 03:37:46.893352       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893357       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.1   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893363       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.2   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893367       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.3   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893371       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.4   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893373       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.5   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893378       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.6   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893381       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.7   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893384       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:02:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893392       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:10:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1123 03:37:46.893395       1 main.go:68] Initializing resource servers
I1123 03:37:46.893402       1 manager.go:117] number of config: 1
I1123 03:37:46.893407       1 manager.go:121] Creating new ResourcePool: sriov_dpu
I1123 03:37:46.893410       1 manager.go:122] DeviceType: netDevice
I1123 03:37:46.893505       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.893752       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.893982       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.894218       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.894488       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.894771       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.895066       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1123 03:37:46.895460       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.895659       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.895879       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.896124       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.896395       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.896670       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.896956       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1123 03:37:46.897259       1 manager.go:138] initServers(): selector index 0 will register 6 devices
I1123 03:37:46.897269       1 factory.go:122] device added: [identifier: 0000:01:00.2, vendor: 1af4, device: 1041, driver: virtio-pci]
I1123 03:37:46.897272       1 factory.go:122] device added: [identifier: 0000:01:00.3, vendor: 1af4, device: 1041, driver: virtio-pci]
I1123 03:37:46.897275       1 factory.go:122] device added: [identifier: 0000:01:00.4, vendor: 1af4, device: 1041, driver: virtio-pci]
I1123 03:37:46.897277       1 factory.go:122] device added: [identifier: 0000:01:00.5, vendor: 1af4, device: 1041, driver: virtio-pci]
I1123 03:37:46.897280       1 factory.go:122] device added: [identifier: 0000:01:00.6, vendor: 1af4, device: 1041, driver: vfio-pci]
I1123 03:37:46.897284       1 factory.go:122] device added: [identifier: 0000:01:00.7, vendor: 1af4, device: 1041, driver: virtio-pci]
I1123 03:37:46.897303       1 manager.go:156] New resource server is created for sriov_dpu ResourcePool
I1123 03:37:46.897309       1 main.go:74] Starting all servers...
I1123 03:37:46.897475       1 server.go:254] starting sriov_dpu device plugin endpoint at: yusur.tech_sriov_dpu.sock
I1123 03:37:46.898073       1 server.go:282] sriov_dpu device plugin endpoint started serving
I1123 03:37:46.898109       1 main.go:79] All servers started.
I1123 03:37:46.898115       1 main.go:80] Listening for term signals
I1123 03:37:47.847008       1 server.go:116] Plugin: yusur.tech_sriov_dpu.sock gets registered successfully at Kubelet
I1123 03:37:47.847085       1 server.go:157] ListAndWatch(sriov_dpu) invoked
I1123 03:37:47.847107       1 server.go:170] ListAndWatch(sriov_dpu): send devices &ListAndWatchResponse{Devices:[]*Device{&Device{ID:0000:01:00.7,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.2,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.3,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.4,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.5,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.6,Health:Healthy,Topology:nil,},},}

Let's check the SR-IOV resources on the local node.

root@master:~/ovn/ovn-kubernetes/dist/images/autodeploy# kubectl describe node node1vdpa
Name:               node1vdpa
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    k8s.ovn.org/dpu-host=
                    k8s.ovn.org/vdpa-dpu-host=
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=node1vdpa
                    kubernetes.io/os=linux
                    node=node1
                    yusur-type=dpu
Annotations:        k8s.ovn.org/host-addresses: ["172.17.0.1","192.168.124.1","192.168.200.35","192.168.201.101"]
                    k8s.ovn.org/l3-gateway-config:
                      {"default":{"mode":"shared","interface-id":"brp0_node1vdpa","mac-address":"00:15:6d:ee:ce:cb","ip-addresses":["192.168.201.21/24"],"ip-add...
                    k8s.ovn.org/node-chassis-id: 58674eff-bed3-4cec-837d-afc88df23dc4
                    k8s.ovn.org/node-gateway-router-lrp-ifaddr: {"ipv4":"100.64.0.3/16"}
                    k8s.ovn.org/node-mgmt-port-mac-address: 0a:58:0a:f4:00:02
                    k8s.ovn.org/node-primary-ifaddr: {"ipv4":"192.168.201.21/24"}
                    k8s.ovn.org/node-subnets: {"default":"10.244.0.0/24"}
                    kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/containerd/containerd.sock
                    node.alpha.kubernetes.io/ttl: 0
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Tue, 21 Nov 2023 13:51:38 +0800
Taints:             <none>
Unschedulable:      false
Lease:
  HolderIdentity:  node1vdpa
  AcquireTime:     <unset>
  RenewTime:       Thu, 23 Nov 2023 11:39:37 +0800
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Thu, 23 Nov 2023 11:37:55 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Thu, 23 Nov 2023 11:37:55 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Thu, 23 Nov 2023 11:37:55 +0800   Wed, 22 Nov 2023 19:44:57 +0800   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Thu, 23 Nov 2023 11:37:55 +0800   Thu, 23 Nov 2023 10:51:27 +0800   KubeletReady                 kubelet is posting ready status. AppArmor enabled
Addresses:
  InternalIP:  192.168.201.21
  Hostname:    node1vdpa
Capacity:
  cpu:                   8
  ephemeral-storage:     24590672Ki
  hugepages-1Gi:         0
  hugepages-2Mi:         1200Mi
  memory:                8127040Ki
  pods:                  110
  yusur.tech/sriov_dpu:  6
Allocatable:
  cpu:                   8
  ephemeral-storage:     22662763278
  hugepages-1Gi:         0
  hugepages-2Mi:         1200Mi
  memory:                6795840Ki
  pods:                  110
  yusur.tech/sriov_dpu:  6
System Info:
  Machine ID:                 eabba479306945cfb24292224ef23d1a
  System UUID:                da7512b0-6152-49be-9bf6-de13d986c2c7
  Boot ID:                    a2a0a5dc-308e-4db6-a362-8c3d7a855a3a
  Kernel Version:             5.15.0-87-generic
  OS Image:                   Ubuntu 20.04.6 LTS
  Operating System:           linux
  Architecture:               amd64
  Container Runtime Version:  containerd://1.6.19
  Kubelet Version:            v1.24.2
  Kube-Proxy Version:         v1.24.2
PodCIDR:                      10.244.1.0/24
PodCIDRs:                     10.244.1.0/24
Non-terminated Pods:          (3 in total)
  Namespace                   Name                                    CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------                   ----                                    ------------  ----------  ---------------  -------------  ---
  kube-system                 kube-multus-ds-nntrh                    100m (1%)     100m (1%)   50Mi (0%)        50Mi (0%)      47m
  kube-system                 kube-sriov-device-plugin-amd64-8g52f    250m (3%)     1 (12%)     40Mi (0%)        200Mi (3%)     2m
  ovn-kubernetes              ovnkube-node-dpu-host-vdpa-cv7m2        100m (1%)     0 (0%)      300Mi (4%)       0 (0%)         48m
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource              Requests     Limits
  --------              --------     ------
  cpu                   450m (5%)    1100m (13%)
  memory                390Mi (5%)   250Mi (3%)
  ephemeral-storage     0 (0%)       0 (0%)
  hugepages-1Gi         0 (0%)       0 (0%)
  hugepages-2Mi         300Mi (25%)  300Mi (25%)
  yusur.tech/sriov_dpu  0            0
Events:
  Type    Reason     Age                From     Message
  ----    ------     ----               ----     -------
  Normal  NodeReady  48m (x4 over 45h)  kubelet  Node node1vdpa status is now: NodeReady

Fixed issue with virtio driver by adding a conditional check for the virtio path under the PCI address, preventing errors when the pf name cannot be found. This ensures successful acquisition of vf resources. Fixes k8snetworkplumbingwg#507 Signed-off-by: JasinlikeEatingOrange <[email protected]>

SchSeba · 2023-12-05T16:17:06Z

Hi,
please be aware that pfName is not for regular interfaces. you need to use the pciAddress if you want to add the devices to the sriov-device-plugin.

if not you will have an issue if you have a pod running using the device and you restart the device plugin he will not be able to find the device after the reboot.

adrianchiris · 2023-12-05T16:55:03Z

@JasinlikeEatingOrange can you change pfName selector to rootDevices selector and use PCI address of the physical device ?
this selector also support VF index range syntax

@SchSeba pciAddress selector will match on the exact device, i believe he needs rootDevices to get all VF devices of a given physical device.

SchSeba · 2023-12-05T17:09:06Z

but they are not VFs if I didn't miss anything from the logs you can see

I1123 02:59:05.269156       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1123 02:59:05.275501       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1123 02:59:05.275589       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1123 02:59:05.275653       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1123 02:59:05.275702       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1123 02:59:05.275752       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1123 02:59:05.275799       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1123 02:59:05.275864       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1123 02:59:05.275920       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1123 02:59:05.275984       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276007       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276029       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276042       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276052       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276061       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276072       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276082       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276093       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276210       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio network device
I1123 02:59:05.276239       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio network device

they are all Red Hat, Inc. Virtio network device these is not a PF they are just virtual devices.
I can also see Capacity: cpu: 8 so I am pretty sure it's a virtual server.

SchSeba · 2023-12-05T17:10:38Z

these are the only selectors supported for devices that are not sriov PFs

"pciAddresses" N Target device's pci address as string string list Default: null "pciAddresses": ["0000:03:02.0"]
"acpiIndexes" N Target device's acpi index as string string list Default: null "acpiIndexes": ["101"]

adrianchiris · 2023-12-06T07:59:31Z

but they are not VFs if I didn't miss anything from the logs you can see

from previous comment #507 (comment)

see section 2.

the pci device has a physfn link in sysfs and the physical device has virtfn files in sysfs, so was assuming its sriov.
maybe its not the case IDK...

SchSeba · 2023-12-06T10:33:17Z

I don't understand also.

@JasinlikeEatingOrange can you please explain what is this device as it doesn't look like a real PF

JasinlikeEatingOrange · 2023-12-07T09:03:58Z

I don't understand also.

@JasinlikeEatingOrange can you please explain what is this device as it doesn't look like a real PF

@SchSeba @adrianchiris
I apologize for not explaining this earlier. As a DPU manufacturer, Our actual DPU hardware devices are relatively expensive, with each DPU physical card costing approximately tens of thousands of dollars. Therefore, for development and testing purposes, we use several virtual machines to simulate these devices. When it comes to final online validation, we use physical cards. However, it's important to note that the code for simulating PF (Physical Function) and VF (Virtual Function) in the virtual environment is identical to that of the real hardware devices.what we are currently observing can be understood as a genuine physical network card with multiple virtual functions being created.
I answered your previous question. Does my response align with the inquiry from before? I'm concerned that I may have misunderstood it.

adrianchiris · 2023-12-07T14:44:53Z

does rootDevices selector OR pciAddresses work for your virtual env ?

what DPU are we talking about ? company model etc ?

SchSeba · 2023-12-07T16:57:33Z

@JasinlikeEatingOrange base on

However, it's important to note that the code for simulating PF (Physical Function) and VF (Virtual Function) in the virtual environment is identical to that of the real hardware devices.what we are currently observing can be understood as a genuine physical network card with multiple virtual functions being created.

Does this mean that the PF for your card is going to be a virtio device?

JasinlikeEatingOrange · 2023-12-08T12:26:52Z

rootDevices
it works but it will throw error W1208 12:24:37.819646 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"

I1208 12:24:37.632987       1 manager.go:51] Using Kubelet Plugin Registry Mode
I1208 12:24:37.647684       1 main.go:44] resource manager reading configs
I1208 12:24:37.647717       1 manager.go:79] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
                "rootDevices": ["0000:01:00.0#1-6"]
        }
      }
  ]
}
I1208 12:24:37.647806       1 factory.go:166] net device selector for resource sriov_dpu is &{DeviceSelectors:{Vendors:[] Devices:[] Drivers:[] PciAddresses:[]} PfNames:[] RootDevices:[0000:01:00.0#1-6] LinkTypes:[] DDPProfiles:[] IsRdma:false NeedVhostNet:false}
I1208 12:24:37.647831       1 manager.go:99] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice Selectors:0xc00000cd38 SelectorObj:0xc0001c3110}]
I1208 12:24:37.647874       1 manager.go:200] validating resource name "yusur.tech/sriov_dpu"
I1208 12:24:37.647879       1 main.go:60] Discovering host devices
I1208 12:24:37.810009       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1208 12:24:37.810358       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1208 12:24:37.810450       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1208 12:24:37.816184       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1208 12:24:37.816267       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1208 12:24:37.816325       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1208 12:24:37.816377       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1208 12:24:37.816440       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1208 12:24:37.816500       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1208 12:24:37.816576       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816594       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816615       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816625       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816636       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816647       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816657       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816667       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816679       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816689       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816717       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:24:37.816740       1 main.go:66] Initializing resource servers
I1208 12:24:37.816748       1 manager.go:105] number of config: 1
I1208 12:24:37.816751       1 manager.go:109]
I1208 12:24:37.816753       1 manager.go:110] Creating new ResourcePool: sriov_dpu
I1208 12:24:37.816756       1 manager.go:111] DeviceType: netDevice
I1208 12:24:37.816968       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.817177       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.817377       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.817597       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.817839       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.818106       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.818391       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:24:37.818496       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0b.0. error getting devlink device attributes for net device 0000:00:0b.0 no such device
I1208 12:24:37.818586       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0c.0. error getting devlink device attributes for net device 0000:00:0c.0 no such device
W1208 12:24:37.818595       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:00:0c.0/net: no such file or directory"
I1208 12:24:37.818833       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.818844       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.1/net: no such file or directory"
I1208 12:24:37.818989       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.818997       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.2/net: no such file or directory"
I1208 12:24:37.819162       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.819171       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.3/net: no such file or directory"
I1208 12:24:37.819360       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.819370       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.4/net: no such file or directory"
I1208 12:24:37.819637       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.819646       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"
I1208 12:24:37.819888       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.819897       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.6/net: no such file or directory"
I1208 12:24:37.820160       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:24:37.820169       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.7/net: no such file or directory"
I1208 12:24:37.820232       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:02:00.0. error getting devlink device attributes for net device 0000:02:00.0 no such device
W1208 12:24:37.820242       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:02:00.0/net: no such file or directory"
I1208 12:24:37.820306       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:10:00.0. error getting devlink device attributes for net device 0000:10:00.0 no such device
W1208 12:24:37.820314       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:10:00.0/net: no such file or directory"
I1208 12:24:37.820330       1 factory.go:106] device added: [pciAddr: 0000:01:00.2, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820333       1 factory.go:106] device added: [pciAddr: 0000:01:00.3, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820336       1 factory.go:106] device added: [pciAddr: 0000:01:00.4, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820338       1 factory.go:106] device added: [pciAddr: 0000:01:00.5, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820340       1 factory.go:106] device added: [pciAddr: 0000:01:00.6, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820343       1 factory.go:106] device added: [pciAddr: 0000:01:00.7, vendor: 1af4, device: 1041, driver: virtio-pci]
I1208 12:24:37.820352       1 manager.go:139] New resource server is created for sriov_dpu ResourcePool
I1208 12:24:37.820356       1 main.go:72] Starting all servers...
I1208 12:24:37.820537       1 server.go:199] starting sriov_dpu device plugin endpoint at: yusur.tech_sriov_dpu.sock
I1208 12:24:37.829701       1 server.go:226] sriov_dpu device plugin endpoint started serving
I1208 12:24:37.829722       1 main.go:77] All servers started.
I1208 12:24:37.829726       1 main.go:78] Listening for term signals
I1208 12:24:38.293372       1 server.go:110] Plugin: yusur.tech_sriov_dpu.sock gets registered successfully at Kubelet
I1208 12:24:38.293426       1 server.go:134] ListAndWatch(sriov_dpu) invoked
I1208 12:24:38.293431       1 server.go:142] ListAndWatch(sriov_dpu): send devices &ListAndWatchResponse{Devices:[]*Device{&Device{ID:0000:01:00.2,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.3,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.4,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.5,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.6,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.7,Health:Healthy,Topology:nil,},},}

JasinlikeEatingOrange · 2023-12-08T12:32:15Z

pciAddresses
Use pciAddresses to select vf resource.

root@master:~# kubectl logs -f -n kube-system      kube-sriov-device-plugin-amd64-8rg72
I1208 12:31:14.488334       1 manager.go:51] Using Kubelet Plugin Registry Mode
I1208 12:31:14.488497       1 main.go:44] resource manager reading configs
I1208 12:31:14.488519       1 manager.go:79] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
                "pciAddresses": ["0000:01:00.0#1-6"]
        }
      }
  ]
}
I1208 12:31:14.488632       1 factory.go:166] net device selector for resource sriov_dpu is &{DeviceSelectors:{Vendors:[] Devices:[] Drivers:[] PciAddresses:[0000:01:00.0#1-6]} PfNames:[] RootDevices:[] LinkTypes:[] DDPProfiles:[] IsRdma:false NeedVhostNet:false}
I1208 12:31:14.488645       1 manager.go:99] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice Selectors:0xc00011ad20 SelectorObj:0xc0001dd2b0}]
I1208 12:31:14.488678       1 manager.go:200] validating resource name "yusur.tech/sriov_dpu"
I1208 12:31:14.488682       1 main.go:60] Discovering host devices
I1208 12:31:14.631963       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1208 12:31:14.632309       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1208 12:31:14.632412       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1208 12:31:14.632484       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1208 12:31:14.632558       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1208 12:31:14.632621       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1208 12:31:14.632692       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1208 12:31:14.632764       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1208 12:31:14.632827       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1208 12:31:14.632906       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.689364       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693363       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693388       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693399       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693408       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693652       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693673       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693687       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693700       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693928       1 netDeviceProvider.go:84] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio network device
I1208 12:31:14.693966       1 main.go:66] Initializing resource servers
I1208 12:31:14.693974       1 manager.go:105] number of config: 1
I1208 12:31:14.693977       1 manager.go:109]
I1208 12:31:14.693980       1 manager.go:110] Creating new ResourcePool: sriov_dpu
I1208 12:31:14.693982       1 manager.go:111] DeviceType: netDevice
I1208 12:31:14.700548       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.700829       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.701085       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.701397       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.701727       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.702054       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.702368       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1208 12:31:14.705249       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0b.0. error getting devlink device attributes for net device 0000:00:0b.0 no such device
I1208 12:31:14.705367       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:00:0c.0. error getting devlink device attributes for net device 0000:00:0c.0 no such device
W1208 12:31:14.705382       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:00:0c.0/net: no such file or directory"
I1208 12:31:14.705721       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.705737       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.1/net: no such file or directory"
I1208 12:31:14.705913       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.705928       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.2/net: no such file or directory"
I1208 12:31:14.706125       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.706141       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.3/net: no such file or directory"
I1208 12:31:14.706353       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.706370       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.4/net: no such file or directory"
I1208 12:31:14.706613       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.706627       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"
I1208 12:31:14.706932       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.706946       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.6/net: no such file or directory"
I1208 12:31:14.707229       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
W1208 12:31:14.707242       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.7/net: no such file or directory"
I1208 12:31:14.707318       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:02:00.0. error getting devlink device attributes for net device 0000:02:00.0 no such device
W1208 12:31:14.707332       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:02:00.0/net: no such file or directory"
I1208 12:31:14.707410       1 utils.go:71] Devlink query for eswitch mode is not supported for device 0000:10:00.0. error getting devlink device attributes for net device 0000:10:00.0 no such device
W1208 12:31:14.707423       1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:10:00.0/net: no such file or directory"
I1208 12:31:14.707431       1 manager.go:125] no devices in device pool, skipping creating resource server for sriov_dpu
I1208 12:31:14.707435       1 main.go:72] Starting all servers...
I1208 12:31:14.707438       1 main.go:77] All servers started.
I1208 12:31:14.707443       1 main.go:78] Listening for term signals

JasinlikeEatingOrange · 2023-12-08T12:33:41Z

@JasinlikeEatingOrange base on

However, it's important to note that the code for simulating PF (Physical Function) and VF (Virtual Function) in the virtual environment is identical to that of the real hardware devices.what we are currently observing can be understood as a genuine physical network card with multiple virtual functions being created.

Does this mean that the PF for your card is going to be a virtio device?

Yes, Your understanding is correct.
During the verification phase, this card is simulated by a virtual machine. When it's actually sold to customers, the displayed effects are consistent. So, you can simply understand the PF as a real physical device. There's no need to worry whether it's a real physical device or not. In fact, a real physical device would also have this error.

adrianchiris · 2023-12-10T07:34:04Z

rootDevices
it works but it will throw error W1208 12:24:37.819646 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"

yes, its a warning [1] as expected. it would still expose related VFs as resources.

[1]

sriov-network-device-plugin/pkg/devices/gen_net.go

Line 52 in 30652a2

glog.Warningf("unable to get PF name %q", err.Error())

****

JasinlikeEatingOrange · 2023-12-10T10:13:51Z

rootDevices
it works but it will throw error W1208 12:24:37.819646 1 pciNetDevice.go:81] unable to get PF name "open /sys/bus/pci/devices/0000:01:00.5/net: no such file or directory"

yes, its a warning [1] as expected. it would still expose related VFs as resources.

[1]

sriov-network-device-plugin/pkg/devices/gen_net.go

Line 52 in 30652a2

glog.Warningf("unable to get PF name %q", err.Error())

yes if i use this modify image it will work, use pfname and rootDevices works. not throw warning.
should this modify merge to repo?

[1]

sriov-network-device-plugin/pkg/devices/gen_net.go

contributor can determine if it's necessary to modify it.

root@master:~# kubectl logs -f -n kube-system      kube-sriov-device-plugin-amd64-qgjzq
I1210 10:01:34.904510       1 manager.go:57] Using Kubelet Plugin Registry Mode
I1210 10:01:34.904700       1 main.go:46] resource manager reading configs
I1210 10:01:34.904730       1 manager.go:86] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
                "rootDevices": ["0000:01:00.0#1-6"]
        }
      }
  ]
}
I1210 10:01:34.904827       1 factory.go:209] *types.NetDeviceSelectors for resource sriov_dpu is [0xc0001b8e00]
I1210 10:01:34.904839       1 manager.go:106] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice ExcludeTopology:false Selectors:0xc000332870 AdditionalInfo:map[] SelectorObjs:[0xc0001b8e00]}]
I1210 10:01:34.904868       1 manager.go:217] validating resource name "yusur.tech/sriov_dpu"
I1210 10:01:34.904875       1 main.go:62] Discovering host devices
I1210 10:01:35.040963       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.0   02              unknown                 unknown
I1210 10:01:35.041013       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.1   02              unknown                 unknown
I1210 10:01:35.041017       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.2   02              unknown                 unknown
I1210 10:01:35.041020       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.3   02              unknown                 unknown
I1210 10:01:35.041022       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.4   02              unknown                 unknown
I1210 10:01:35.041025       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.5   02              unknown                 unknown
I1210 10:01:35.041028       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.6   02              unknown                 unknown
I1210 10:01:35.041030       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.7   02              unknown                 unknown
I1210 10:01:35.041032       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0b.0   02              unknown                 unknown
I1210 10:01:35.041035       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0c.0   02              Red Hat, Inc.           Virtio network device
I1210 10:01:35.041038       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041040       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.1   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041042       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.2   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041044       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.3   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041047       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.4   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041049       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.5   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041052       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.6   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041057       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.7   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041060       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:02:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041063       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:10:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.041068       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1210 10:01:35.041529       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1210 10:01:35.041653       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1210 10:01:35.041787       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1210 10:01:35.041891       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1210 10:01:35.041956       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1210 10:01:35.042016       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1210 10:01:35.042088       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1210 10:01:35.042144       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1210 10:01:35.042223       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1210 10:01:35.042247       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042271       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042290       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042300       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042310       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042322       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042334       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042346       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042356       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042379       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:01:35.042408       1 main.go:68] Initializing resource servers
I1210 10:01:35.042412       1 manager.go:117] number of config: 1
I1210 10:01:35.042416       1 manager.go:121] Creating new ResourcePool: sriov_dpu
I1210 10:01:35.042419       1 manager.go:122] DeviceType: netDevice
I1210 10:01:35.042498       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.042719       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.042938       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.043168       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.043421       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.043721       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.044029       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:01:35.044448       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.044640       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.044848       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.045078       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.045330       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.053897       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.054290       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:01:35.054732       1 manager.go:138] initServers(): selector index 0 will register 6 devices
I1210 10:01:35.054741       1 factory.go:122] device added: [identifier: 0000:01:00.2, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054745       1 factory.go:122] device added: [identifier: 0000:01:00.3, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054747       1 factory.go:122] device added: [identifier: 0000:01:00.4, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054750       1 factory.go:122] device added: [identifier: 0000:01:00.5, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054752       1 factory.go:122] device added: [identifier: 0000:01:00.6, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054757       1 factory.go:122] device added: [identifier: 0000:01:00.7, vendor: 1af4, device: 1041, driver: virtio-pci]
I1210 10:01:35.054796       1 manager.go:156] New resource server is created for sriov_dpu ResourcePool
I1210 10:01:35.054803       1 main.go:74] Starting all servers...
I1210 10:01:35.055023       1 server.go:254] starting sriov_dpu device plugin endpoint at: yusur.tech_sriov_dpu.sock
I1210 10:01:35.057946       1 server.go:282] sriov_dpu device plugin endpoint started serving
I1210 10:01:35.057975       1 main.go:79] All servers started.
I1210 10:01:35.057981       1 main.go:80] Listening for term signals
I1210 10:01:35.714697       1 server.go:116] Plugin: yusur.tech_sriov_dpu.sock gets registered successfully at Kubelet
I1210 10:01:35.714778       1 server.go:157] ListAndWatch(sriov_dpu) invoked
I1210 10:01:35.714798       1 server.go:170] ListAndWatch(sriov_dpu): send devices &ListAndWatchResponse{Devices:[]*Device{&Device{ID:0000:01:00.2,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.3,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.4,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.5,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.6,Health:Healthy,Topology:nil,},&Device{ID:0000:01:00.7,Health:Healthy,Topology:nil,},},}

But use pciAddresses can't find resources.

root@master:~# kubectl logs -f -n kube-system      kube-sriov-device-plugin-amd64-6w57p
I1210 10:10:19.221745       1 manager.go:57] Using Kubelet Plugin Registry Mode
I1210 10:10:19.237750       1 main.go:46] resource manager reading configs
I1210 10:10:19.237915       1 manager.go:86] raw ResourceList: {
  "resourceList": [{
        "resourceName": "sriov_dpu",
        "resourcePrefix": "yusur.tech",
        "selectors": {
                "pciAddresses": ["0000:01:00.0#1-6"]
        }
      }
  ]
}
I1210 10:10:19.238234       1 factory.go:209] *types.NetDeviceSelectors for resource sriov_dpu is [0xc0002a0e00]
I1210 10:10:19.238247       1 manager.go:106] unmarshalled ResourceList: [{ResourcePrefix:yusur.tech ResourceName:sriov_dpu DeviceType:netDevice ExcludeTopology:false Selectors:0xc00041c840 AdditionalInfo:map[] SelectorObjs:[0xc0002a0e00]}]
I1210 10:10:19.238384       1 manager.go:217] validating resource name "yusur.tech/sriov_dpu"
I1210 10:10:19.238392       1 main.go:62] Discovering host devices
I1210 10:10:19.333593       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.0 02              unknown                 unknown
I1210 10:10:19.334017       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.1 02              unknown                 unknown
I1210 10:10:19.335102       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.2 02              unknown                 unknown
I1210 10:10:19.335192       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.3 02              unknown                 unknown
I1210 10:10:19.335254       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.4 02              unknown                 unknown
I1210 10:10:19.335316       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.5 02              unknown                 unknown
I1210 10:10:19.335373       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.6 02              unknown                 unknown
I1210 10:10:19.335435       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0a.7 02              unknown                 unknown
I1210 10:10:19.335492       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0b.0 02              unknown                 unknown
I1210 10:10:19.335594       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:00:0c.0 02              Red Hat, Inc.           Virtio network device
I1210 10:10:19.335626       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335656       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.1 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335668       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.2 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335680       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.3 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335692       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.4 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335704       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.5 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335717       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.6 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335731       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:01:00.7 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335742       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:02:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335770       1 netDeviceProvider.go:67] netdevice AddTargetDevices(): device found: 0000:10:00.0 02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335796       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.0   02              unknown                 unknown
I1210 10:10:19.335811       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.1   02              unknown                 unknown
I1210 10:10:19.335818       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.2   02              unknown                 unknown
I1210 10:10:19.335822       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.3   02              unknown                 unknown
I1210 10:10:19.335827       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.4   02              unknown                 unknown
I1210 10:10:19.335832       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.5   02              unknown                 unknown
I1210 10:10:19.335837       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.6   02              unknown                 unknown
I1210 10:10:19.335841       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0a.7   02              unknown                 unknown
I1210 10:10:19.335844       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0b.0   02              unknown                 unknown
I1210 10:10:19.335847       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:00:0c.0   02              Red Hat, Inc.           Virtio network device
I1210 10:10:19.335850       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335855       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.1   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335858       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.2   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335861       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.3   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335865       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.4   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335869       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.5   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335873       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.6   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335876       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:01:00.7   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335879       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:02:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335886       1 auxNetDeviceProvider.go:84] auxnetdevice AddTargetDevices(): device found: 0000:10:00.0   02              Red Hat, Inc.           Virtio 1.0 network device
I1210 10:10:19.335892       1 main.go:68] Initializing resource servers
I1210 10:10:19.335896       1 manager.go:117] number of config: 1
I1210 10:10:19.335900       1 manager.go:121] Creating new ResourcePool: sriov_dpu
I1210 10:10:19.335903       1 manager.go:122] DeviceType: netDevice
I1210 10:10:19.336001       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.1. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.336256       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.2. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.336504       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.3. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.336782       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.4. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.337072       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.5. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.337408       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.6. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.337750       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:00:0a.7. error getting devlink device attributes for net device 0000:00:0a.0 no such device
I1210 10:10:19.338911       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.1. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.339158       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.2. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.339575       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.3. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.345010       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.4. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.349783       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.5. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.355430       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.6. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.355765       1 utils.go:81] Devlink query for eswitch mode is not supported for device 0000:01:00.7. error getting devlink device attributes for net device 0000:01:00.0 no such device
I1210 10:10:19.362164       1 manager.go:138] initServers(): selector index 0 will register 0 devices
I1210 10:10:19.362186       1 manager.go:142] no devices in device pool, skipping creating resource server for sriov_dpu
I1210 10:10:19.362191       1 main.go:74] Starting all servers...
I1210 10:10:19.362196       1 main.go:79] All servers started.
I1210 10:10:19.362199       1 main.go:80] Listening for term signals

SchSeba · 2023-12-12T20:51:45Z

ok so I think we are good to just use the rootDevices right?

SchSeba · 2024-08-19T14:13:34Z

closing this one

JasinlikeEatingOrange mentioned this issue Nov 8, 2023

Fix existing logic to prevent the unavailability of PFName. #508

Closed

JasinlikeEatingOrange mentioned this issue Nov 29, 2023

Fix can't get pfname for virtio pci path #514

Closed

SchSeba closed this as completed Aug 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use usespace vdpa can't find the pfName because the net path is wrong #507

use usespace vdpa can't find the pfName because the net path is wrong #507

JasinlikeEatingOrange commented Nov 8, 2023

adrianchiris commented Nov 9, 2023

JasinlikeEatingOrange commented Nov 16, 2023

adrianchiris commented Nov 16, 2023 •

edited

Loading

JasinlikeEatingOrange commented Nov 23, 2023

SchSeba commented Dec 5, 2023

adrianchiris commented Dec 5, 2023 •

edited

Loading

SchSeba commented Dec 5, 2023

SchSeba commented Dec 5, 2023 •

edited

Loading

adrianchiris commented Dec 6, 2023

SchSeba commented Dec 6, 2023

JasinlikeEatingOrange commented Dec 7, 2023

adrianchiris commented Dec 7, 2023

SchSeba commented Dec 7, 2023

JasinlikeEatingOrange commented Dec 8, 2023

JasinlikeEatingOrange commented Dec 8, 2023

JasinlikeEatingOrange commented Dec 8, 2023

adrianchiris commented Dec 10, 2023 •

edited

Loading

JasinlikeEatingOrange commented Dec 10, 2023 •

edited

Loading

SchSeba commented Dec 12, 2023

SchSeba commented Aug 19, 2024

use usespace vdpa can't find the pfName because the net path is wrong #507

use usespace vdpa can't find the pfName because the net path is wrong #507

Comments

JasinlikeEatingOrange commented Nov 8, 2023

What happened?

What did you expect to happen?

What are the minimal steps needed to reproduce the bug?

Anything else we need to know?

Component Versions

Config Files

Device pool config file location (Try '/etc/pcidp/config.json')

Multus config (Try '/etc/cni/multus/net.d')

CNI config (Try '/etc/cni/net.d/')

Kubernetes deployment type ( Bare Metal, Kubeadm etc.)

Kubeconfig file

SR-IOV Network Custom Resource Definition

Logs

SR-IOV Network Device Plugin Logs (use kubectl logs $PODNAME)

Multus logs (If enabled. Try '/var/log/multus.log' )

Kubelet logs (journalctl -u kubelet)

adrianchiris commented Nov 9, 2023

JasinlikeEatingOrange commented Nov 16, 2023

1. Situation

2. Solution

adrianchiris commented Nov 16, 2023 • edited Loading

JasinlikeEatingOrange commented Nov 23, 2023

1.Current original version:

2. Node information for node1vdpa

3. Solution

4. The effect after the modification.

SchSeba commented Dec 5, 2023

adrianchiris commented Dec 5, 2023 • edited Loading

SchSeba commented Dec 5, 2023

SchSeba commented Dec 5, 2023 • edited Loading

adrianchiris commented Dec 6, 2023

SchSeba commented Dec 6, 2023

JasinlikeEatingOrange commented Dec 7, 2023

adrianchiris commented Dec 7, 2023

SchSeba commented Dec 7, 2023

JasinlikeEatingOrange commented Dec 8, 2023

JasinlikeEatingOrange commented Dec 8, 2023

JasinlikeEatingOrange commented Dec 8, 2023

adrianchiris commented Dec 10, 2023 • edited Loading

JasinlikeEatingOrange commented Dec 10, 2023 • edited Loading

SchSeba commented Dec 12, 2023

SchSeba commented Aug 19, 2024

SR-IOV Network Device Plugin Logs (use `kubectl logs $PODNAME`)

adrianchiris commented Nov 16, 2023 •

edited

Loading

adrianchiris commented Dec 5, 2023 •

edited

Loading

SchSeba commented Dec 5, 2023 •

edited

Loading

adrianchiris commented Dec 10, 2023 •

edited

Loading

JasinlikeEatingOrange commented Dec 10, 2023 •

edited

Loading