dpif-netdev: Introduce pmd-rxq-affinity.

New 'other_config:pmd-rxq-affinity' field for Interface table to perform manual pinning of RX queues to desired cores. This functionality is required to achieve maximum performance because all kinds of ports have different cost of rx/tx operations and only user can know about expected workload on different ports. Example: # ./bin/ovs-vsctl set interface dpdk0 options:n_rxq=4 \ other_config:pmd-rxq-affinity="0:3,1:7,3:8" Queue #0 pinned to core 3; Queue #1 pinned to core 7; Queue #2 not pinned. Queue #3 pinned to core 8; It's decided to automatically isolate cores that have rxq explicitly assigned to them because it's useful to keep constant polling rate on some performance critical ports while adding/deleting other ports without explicit pinning of all ports. Signed-off-by: Ilya Maximets <[email protected]> Signed-off-by: Daniele Di Proietto <[email protected]>
russellb · Jul 27, 2016 · 3eb6785 · 3eb6785
1 parent a6a426d
commit 3eb6785
Show file tree

Hide file tree

Showing 5 changed files with 252 additions and 41 deletions.
diff --git a/INSTALL.DPDK.md b/INSTALL.DPDK.md
@@ -289,14 +289,57 @@ advanced install guide [INSTALL.DPDK-ADVANCED.md]
      # Check current stats
        ovs-appctl dpif-netdev/pmd-stats-show
 
+     # Clear previous stats
+       ovs-appctl dpif-netdev/pmd-stats-clear
+     ```
+
+  7. Port/rxq assigment to PMD threads
+
+     ```
      # Show port/rxq assignment
        ovs-appctl dpif-netdev/pmd-rxq-show
+     ```
 
-     # Clear previous stats
-       ovs-appctl dpif-netdev/pmd-stats-clear
+     To change default rxq assignment to pmd threads rxqs may be manually
+     pinned to desired cores using:
+
+     ```
+     ovs-vsctl set Interface <iface> \
+               other_config:pmd-rxq-affinity=<rxq-affinity-list>
      ```
+     where:
+
+     ```
+     <rxq-affinity-list> ::= NULL | <non-empty-list>
+     <non-empty-list> ::= <affinity-pair> |
+                          <affinity-pair> , <non-empty-list>
+     <affinity-pair> ::= <queue-id> : <core-id>
+     ```
+
+     Example:
+
+     ```
+     ovs-vsctl set interface dpdk0 options:n_rxq=4 \
+               other_config:pmd-rxq-affinity="0:3,1:7,3:8"
+
+     Queue #0 pinned to core 3;
+     Queue #1 pinned to core 7;
+     Queue #2 not pinned.
+     Queue #3 pinned to core 8;
+     ```
+
+     After that PMD threads on cores where RX queues was pinned will become
+     `isolated`. This means that this thread will poll only pinned RX queues.
+
+     WARNING: If there are no `non-isolated` PMD threads, `non-pinned` RX queues
+     will not be polled. Also, if provided `core_id` is not available (ex. this
+     `core_id` not in `pmd-cpu-mask`), RX queue will not be polled by any
+     PMD thread.
+
+     Isolation of PMD threads also can be checked using
+     `ovs-appctl dpif-netdev/pmd-rxq-show` command.
 
-  7. Stop vswitchd & Delete bridge
+  8. Stop vswitchd & Delete bridge
 
      ```
      ovs-appctl -t ovs-vswitchd exit

diff --git a/NEWS b/NEWS
@@ -45,6 +45,8 @@ Post-v2.5.0
        Old 'other_config:n-dpdk-rxqs' is no longer supported.
        Not supported by vHost interfaces. For them number of rx and tx queues
        is applied from connected virtio device.
+     * New 'other_config:pmd-rxq-affinity' field for PMD interfaces, that
+       allows to pin port's rx queues to desired cores.
      * New appctl command 'dpif-netdev/pmd-rxq-show' to check the port/rxq
        assignment.
      * Type of log messages from PMD threads changed from INFO to DBG.