From f76e19251017b4919505b6f85c2b3e3fc3e1c5f7 Mon Sep 17 00:00:00 2001 From: Don Newton Date: Thu, 7 Apr 2022 12:22:58 -0400 Subject: [PATCH 01/10] Packet_io.md update * include information on Generic Netlink userspace library * include information on Generic Netlink sniffer application Sign_of_by: don.newton@intel.com --- doc/pins/Packet_io.md | 58 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 56 insertions(+), 2 deletions(-) diff --git a/doc/pins/Packet_io.md b/doc/pins/Packet_io.md index cfd22220de..d2be71fca1 100644 --- a/doc/pins/Packet_io.md +++ b/doc/pins/Packet_io.md @@ -16,17 +16,18 @@ _Rev v0.1_ Rev | RevDate | Author(s) | Change Description ---- | ---------- | ----------- | ------------------ v0.1 | 06/23/2021 | Google, ONF | Initial Version +v0.2 | 03/31/2022 | Google, ONF | Extract Library ## Scope -This document covers the high level design aspects of Packet I/O in SONiC for P4Runtime application. +This document covers the high level design aspects of Packet I/O in SONiC for both the P4Runtime application and any subsequent application that would benefit from packet exchange using Generic Netlink Sockets. ## Overview -SONiC supports Packet I/O on Linux netdev interfaces but this does not meet some unique requirements of P4Runtime application. This document details the requirements and captures the design changes necessary to meet the new requirements. +SONiC supports Packet I/O on Linux netdev interfaces but this does not meet some unique requirements of P4Runtime application. This document details the requirements and captures the design changes necessary to meet the new requirements. This document also explains the extraction of a library from the P4Runtime application in order to make the functionality available to other applications as required. In addition to the new library this document will explain a new application which serves as a model for using the library and allows for TCPDump like sniffing capability and packet generation using Generic Netlink Sockets. **Requirements** @@ -176,5 +177,58 @@ Vendor work is needed to enable the creation of the “submit_to_ingress” port ![drawing](images/p4rt_flow.png) +## New Library Usage + +The library functionality is defined in genl-packet/receive_genetlink.h. First a callback function must be defined with the following signature: + +``` +using ReceiveCallbackFunction = std::function; +``` +Then the StartReceive function is called which returns a thread which calls the callback function on packet receipt.  +``` +std::thread StartReceive( + packet_metadata::ReceiveCallbackFunction callback_function, + nl_recvmsg_msg_cb_t process_callback_function); +``` +A version of process_callback_function is implemented in genl-packet/receive_genenlink.cc and will be used if process_callback_function is NULL. The purpose of process_callback_function is to extract the netlink attributes: source port, destination port and payload and pass them to the ReceiveCallbackFunction. This can be implemented by the library user if the logic in the library is insufficient. + +## Library Phases +- **Current:** + - The current kernel module, documented in PacketIO.md, remains unchanged and is supplied by the asic vendor or delegates. + - There is a single multicast group / queue that all producers and consumers can use.  All packets are sent to all consumers.  Filtering must be done post-consumption. + - Packet meta-data is fixed. + +- **Dynamic generic netlink:** + - The kernel module will have an API that allows user space applications to manage multicast groups and queues at runtime. + - The library will allow applications to specify which multicast group / queue to listen to and allow for basic filtering. + - Packet meta-data continues to be fixed. +- **Dynamic meta-data:** + - Kernel module is likely unchanged. + - Multicast group / queue functionality is likely unchanged. + - Packet meta-data can be specified by the using application.   + +## Sniffer Application: +The sniffer provides the means of a tcpdump-like tool to listen to the genetlink device. The sniffer can be used for listening to traffic, as well as recording the traffic into a file or displaying to standard out. The resulting pcapng file can then be viewed using Wireshark. The sender can be used to send an example packet or packets from a pcap/pcapng file through genetlink. The sender also registers a new genetlink family and group called genl_packet and packets respectively. Both sniffer and sender use the pcapplusplus library which is an actively maintained open source library. + +Both the sender and the sniffer can be compiled via bazel or sonic-buildimage. Either way, once compiled or the necessary binary installed the following commands can be used to use the two applications ([sniffer] indicates the sniffer application and [sender] indicates the sender application): +``` +- [sniffer] : launches the sniffer and records all packets into a file named out.pcapng. +- [sniffer] -a : will either append the packets to out.pcapng or to a custom filename if given. +- [sniffer] -o=- : This will print the hex representation of the received packets to standard out. +- [sniffer] -o=hello.pcapng : By providing a filename you can write the genetlink packets into a given file. In this example it will be hello.pcapng. +- [sniffer] -verbose : will print out verbose information about the packets received including metadata and packet contents. +``` + +The packet metadata carried with process_callback_function gets put into a comment in the pcapng. If the sniffer is to be run outside of P4Runtime the user might want to construct their own custom receive thread using customCallbackReceive found in the header file for the sniffer, since the carried metadata might be different. + +``` +- sudo [sender] : will send a sample packet using genetlink. +- sudo [sender] -inputfile=hello.pcapng : will read the packets from a given file and send them via genetlink. +- sudo [sender] -packet=AABBCCDD : will send the given packet in hex representation via genetlink. +``` + + From f9c084ed21505b285e07257954bc0461c159b24f Mon Sep 17 00:00:00 2001 From: Brian O'Connor Date: Wed, 4 May 2022 17:21:41 -0700 Subject: [PATCH 02/10] Clean up --- doc/pins/Packet_io.md | 76 +++++++++++++++---------------------------- 1 file changed, 26 insertions(+), 50 deletions(-) diff --git a/doc/pins/Packet_io.md b/doc/pins/Packet_io.md index d2be71fca1..9c6581c326 100644 --- a/doc/pins/Packet_io.md +++ b/doc/pins/Packet_io.md @@ -1,23 +1,17 @@ -# **PacketIO** +# Packet I/O -_Rev v0.1_ - - -## -**Table of Content** +_Rev v0.2_ +## Table of Contents [TOC] - - ## Revision Rev | RevDate | Author(s) | Change Description ---- | ---------- | ----------- | ------------------ v0.1 | 06/23/2021 | Google, ONF | Initial Version -v0.2 | 03/31/2022 | Google, ONF | Extract Library - +v0.2 | 03/31/2022 | Google, ONF | Extract common packet receiver library ## Scope @@ -30,11 +24,10 @@ This document covers the high level design aspects of Packet I/O in SONiC for bo SONiC supports Packet I/O on Linux netdev interfaces but this does not meet some unique requirements of P4Runtime application. This document details the requirements and captures the design changes necessary to meet the new requirements. This document also explains the extraction of a library from the P4Runtime application in order to make the functionality available to other applications as required. In addition to the new library this document will explain a new application which serves as a model for using the library and allows for TCPDump like sniffing capability and packet generation using Generic Netlink Sockets. -**Requirements** - +### Requirements -**Receive** +#### Receive On the receive side, the controller needs to receive the CPU punted packets with the basic metadata information about the packet. Receive packets arrive on any of the configured L2, L3, LAG or VLAN interfaces created by the controller. Below are the specific requirements for the receive packets, @@ -45,15 +38,12 @@ On the receive side, the controller needs to receive the CPU punted packets with 2. On the received packets, the controller needs the input port and target egress port (path the packet would have forwarded by the switch pipeline) as part of the receive metadata information. Input port is a required standard field but certain applications on the controller like traceroute will also need the target egress port. -**Transmit** - +#### Transmit There are 2 requirements on the transmit side, - 1. Controller needs to be able to send packets out on any of the configured ports, this is generally called the directed transmit. - 2. The controller also needs a capability to leverage the ASIC’s forwarding pipeline instead of self determining which port the packet should be sent out. So, a new transmit mode called the ‘submit_to_ingress’ (aka ‘send_to_ingress’) is needed to transmit packets on the ingress pipeline to let the ASIC’s forwarding pipeline select the egress port. @@ -68,30 +58,24 @@ Receive solution leverages on the existing SAI attributes (`SAI_HOSTIF_TYPE_GENE ![drawing](images/genetlink_rx.png) - Once the user defined traps and genetlink host interfaces are created, incoming punted packets matching the specific flow (with user defined trap) will carry a special tag in their packet header that identifies them as belonging to the special user defined trap. The kernel driver that processes the incoming DMA receive packets will distinguish them based on this tag in the packet header and redirect them to either the genetlink interface or the regular netdev interfaces. CoppOrch will process the [new trap group per cpu queue](https://github.com/pins/sonic-buildimage/blob/pins/202012_20210206/files/image_config/copp/copp_cfg.j2#L67-L106) and create the required host interfaces, user-defined traps, etc. It will use a specific user-defined trap for each ACL entry that P4Runtime installs based on the specific CPU QoS queue the punted packets should take for that ACL. - - ![drawing](images/cpu_qos_trap.png) - ## Vendor changes for Receive path Receive path changes come into picture when the packet is received by the host CPU (kernel DMA or via pcie path) and the packet has to be pre-processed before being sent out to the application. The pre-processing involves parsing the metadata information (vendor specific - steps 1 & 2) and sending the packet (along with the attributes) out via the generic netlink interface in a common manner (step 3). The 3 parts in the solution are: -**1. Determine packet path** +### 1. Determine packet path The vendor code needs to distinguish the packets that need to be sent via `netdev` vs `generic netlink` interface. This is left to vendor choice of implementation but typically involves maintaining a map of an unique packet identifier to dispatch method in the receive code. Incoming packet’s metadata field value is looked up in the table and the packet is dispatched via the corresponding method. The vendor specific work involved here are, - - * Adding the additional block of code in the receive path to handle the new dispatch method for P4Runtime. For example, the below if statement captures a sample code to call the generic netlink packet send function for packets matching the genl_packet key. ``` @@ -114,14 +98,12 @@ knet_filter_cb(uint8_t * pkt, int size, int dev_no, void *meta, ``` - - -**2. Extract and convert packet metadata** +### 2. Extract and convert packet metadata Pull out the packet metadata information from the header and convert it from a vendor specific format (like unit, port number) to a generic format (like ifindex) that is understandable by the application. We leave it to the vendors on how this conversion is done but they should be able to leverage a large part of this code from their Sflow implementation. -**3. Send packet on generic netlink interface** +### 3. Send packet on generic netlink interface The final step is to pack all the necessary packet attributes like ingress, egress ifindex number etc in a generic netlink format and send it on the generic netlink multicast socket (to be consumed by the application). @@ -130,7 +112,6 @@ We have a submodule that can accomplish this task in a vendor-independent way . The change required for P4Runtime integration is, - * Get all the packet metadata information in the generic format (from step 2) and invoke the generic netlink send function. The send function arguments will be able to accommodate all the different packet attributes supported and fit a variable argument list so that vendors can call it as per the packet type and their support for that attribute. @@ -156,7 +137,6 @@ The netdev port attributes are specified as below in the [copp_cfg.j2](https://g }, ``` - [Copporch.cpp](https://github.com/Azure/sonic-swss/blob/fb06c32b2e25e6057514e9455e997ff7edcb7340/orchagent/copporch.cpp) will parse the above fields and add the netdev port attributes for the CPU port and invoke the SAI hostif create API to create the netdev port for the new submit_to_ingress interface. P4Runtime will create a socket for the “submit_to_ingress” netdev port and write into this socket for packets marked to send to “submit_to_ingress”. Packets egressing out of the socket end up as incoming packets on the CPU port of the switch. The packets now follow the regular switch pipeline to get forwarded out. @@ -166,51 +146,51 @@ P4Runtime will create a socket for the “submit_to_ingress” netdev port and w Vendor work is needed to enable the creation of the “submit_to_ingress” port and allow packets in the CPU port. The additional vendor work needed are, - - * Currently, SAI supports the creation of netdev type hostif only for physical port, vlan or LAG interface. Extend the SAI hostif create API to take care of creating a netdev port associated with the CPU port of the switch. * Vendor specific properties to enable packet forwarding in the ASIC when packets are ingressing via the CPU port instead of a front panel port. ## Event flows - ![drawing](images/p4rt_flow.png) ## New Library Usage -The library functionality is defined in genl-packet/receive_genetlink.h. First a callback function must be defined with the following signature: +The library functionality is defined in `genl-packet/receive_genetlink.h`. First, the application must define a callback function with the following signature: ``` using ReceiveCallbackFunction = std::function; ``` -Then the StartReceive function is called which returns a thread which calls the callback function on packet receipt.  + +The function will be called by the library every time a packet is received and will pass the packet payload and metadata that is specified in the signature. + +To start receiving packets, the application must call the `StartReceive` function and provide its `ReceiveCallbackFunction`. The function returns a thread that is used by the library to listen for packets. + ``` std::thread StartReceive( - packet_metadata::ReceiveCallbackFunction callback_function, - nl_recvmsg_msg_cb_t process_callback_function); + packet_metadata::ReceiveCallbackFunction callback_function); ``` -A version of process_callback_function is implemented in genl-packet/receive_genenlink.cc and will be used if process_callback_function is NULL. The purpose of process_callback_function is to extract the netlink attributes: source port, destination port and payload and pass them to the ReceiveCallbackFunction. This can be implemented by the library user if the logic in the library is insufficient. ## Library Phases -- **Current:** - - The current kernel module, documented in PacketIO.md, remains unchanged and is supplied by the asic vendor or delegates. +### 1. Current + - The current kernel module, documented in this HLD, remains unchanged and is supplied by the ASIC vendor. - There is a single multicast group / queue that all producers and consumers can use.  All packets are sent to all consumers.  Filtering must be done post-consumption. - Packet meta-data is fixed. -- **Dynamic generic netlink:** +### 2. Dynamic generic netlink - The kernel module will have an API that allows user space applications to manage multicast groups and queues at runtime. - The library will allow applications to specify which multicast group / queue to listen to and allow for basic filtering. - - Packet meta-data continues to be fixed. -- **Dynamic meta-data:** - - Kernel module is likely unchanged. + - Packet metadata continues to be fixed. + +### 3. Dynamic metadata + - The kernel module will have an API that allows user space applications to customize the metadata that is provided to the application. This will require some design and will likely lean on the `PacketIn` header that is provided by the P4Info. - Multicast group / queue functionality is likely unchanged. - - Packet meta-data can be specified by the using application.   + - Packet metadata can be specified by the using application in the P4 program.   ## Sniffer Application: -The sniffer provides the means of a tcpdump-like tool to listen to the genetlink device. The sniffer can be used for listening to traffic, as well as recording the traffic into a file or displaying to standard out. The resulting pcapng file can then be viewed using Wireshark. The sender can be used to send an example packet or packets from a pcap/pcapng file through genetlink. The sender also registers a new genetlink family and group called genl_packet and packets respectively. Both sniffer and sender use the pcapplusplus library which is an actively maintained open source library. +The `sniffer` application provides the means of a `tcpdump`-like tool to listen to the genetlink device. The sniffer can be used for listening to traffic, as well as recording the traffic into a file or displaying to standard out. The resulting pcapng file can then be viewed using Wireshark. The `sender` application can be used to send an example packet or packets from a pcap/pcapng file through genetlink. The sender also registers a new genetlink family and group called genl_packet and packets respectively. Both sniffer and sender use the pcapplusplus library which is an actively maintained open source library. Both the sender and the sniffer can be compiled via bazel or sonic-buildimage. Either way, once compiled or the necessary binary installed the following commands can be used to use the two applications ([sniffer] indicates the sniffer application and [sender] indicates the sender application): ``` @@ -228,7 +208,3 @@ The packet metadata carried with process_callback_function gets put into a comme - sudo [sender] -inputfile=hello.pcapng : will read the packets from a given file and send them via genetlink. - sudo [sender] -packet=AABBCCDD : will send the given packet in hex representation via genetlink. ``` - - - - From c6f2e17f92176d4afad6f604cbd7e5d759740e4b Mon Sep 17 00:00:00 2001 From: Don Newton Date: Wed, 22 Jun 2022 12:51:12 -0400 Subject: [PATCH 03/10] added block to describe the sender application --- doc/pins/Packet_io.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/pins/Packet_io.md b/doc/pins/Packet_io.md index 9c6581c326..0cded3cc61 100644 --- a/doc/pins/Packet_io.md +++ b/doc/pins/Packet_io.md @@ -203,6 +203,10 @@ Both the sender and the sniffer can be compiled via bazel or sonic-buildimage. E The packet metadata carried with process_callback_function gets put into a comment in the pcapng. If the sniffer is to be run outside of P4Runtime the user might want to construct their own custom receive thread using customCallbackReceive found in the header file for the sniffer, since the carried metadata might be different. +##Sender application +The `sender` application is currently used for testing purposes. It can create a packet, add the appropriate metadata and then send it to the generic netlink device where the `listening` application can pick it up and act on it. It is also useful for recreating error conditions where generic netlink functionality is part of the causality chain. +New applications that are designed to send to the generic netlink devices directly instead of using a kernel module can use this code as a template. + ``` - sudo [sender] : will send a sample packet using genetlink. - sudo [sender] -inputfile=hello.pcapng : will read the packets from a given file and send them via genetlink. From 8c5cd4cf50ca6d13db9a4e6c59ad36261e9a530c Mon Sep 17 00:00:00 2001 From: Don Newton Date: Wed, 22 Jun 2022 12:52:45 -0400 Subject: [PATCH 04/10] fixed typo --- doc/pins/Packet_io.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/pins/Packet_io.md b/doc/pins/Packet_io.md index 0cded3cc61..a2080125f1 100644 --- a/doc/pins/Packet_io.md +++ b/doc/pins/Packet_io.md @@ -203,7 +203,7 @@ Both the sender and the sniffer can be compiled via bazel or sonic-buildimage. E The packet metadata carried with process_callback_function gets put into a comment in the pcapng. If the sniffer is to be run outside of P4Runtime the user might want to construct their own custom receive thread using customCallbackReceive found in the header file for the sniffer, since the carried metadata might be different. -##Sender application +## Sender application: The `sender` application is currently used for testing purposes. It can create a packet, add the appropriate metadata and then send it to the generic netlink device where the `listening` application can pick it up and act on it. It is also useful for recreating error conditions where generic netlink functionality is part of the causality chain. New applications that are designed to send to the generic netlink devices directly instead of using a kernel module can use this code as a template. From 00a4f817a9fd3912f7afda59f4ea18c15a1f3029 Mon Sep 17 00:00:00 2001 From: Don Newton Date: Wed, 22 Jun 2022 13:04:52 -0400 Subject: [PATCH 05/10] adjusting the sender/sniffer narrative --- doc/pins/Packet_io.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/doc/pins/Packet_io.md b/doc/pins/Packet_io.md index a2080125f1..d655f601b2 100644 --- a/doc/pins/Packet_io.md +++ b/doc/pins/Packet_io.md @@ -190,9 +190,16 @@ std::thread StartReceive( - Packet metadata can be specified by the using application in the P4 program.   ## Sniffer Application: -The `sniffer` application provides the means of a `tcpdump`-like tool to listen to the genetlink device. The sniffer can be used for listening to traffic, as well as recording the traffic into a file or displaying to standard out. The resulting pcapng file can then be viewed using Wireshark. The `sender` application can be used to send an example packet or packets from a pcap/pcapng file through genetlink. The sender also registers a new genetlink family and group called genl_packet and packets respectively. Both sniffer and sender use the pcapplusplus library which is an actively maintained open source library. +The `sniffer` application provides the means of a `tcpdump`-like tool to listen to the genetlink device. The sniffer can be used for listening to traffic, as well as recording the traffic into a file or displaying to standard out. The resulting pcapng file can then be viewed using Wireshark. The `sender` application can be used to send an example packet or packets from a pcap/pcapng file through genetlink. The sender also registers a new genetlink family and group called genl_packet and packets respectively. + +## Sender application: +The `sender` application is currently used for testing purposes. It can create a packet, add the appropriate metadata and then send it to the generic netlink device where the `listening` application can pick it up and act on it. It is also useful for recreating error conditions where generic netlink functionality is part of the causality chain. +New applications that are designed to send to the generic netlink devices directly instead of using a kernel module can use this code as a template. Both the sender and the sniffer can be compiled via bazel or sonic-buildimage. Either way, once compiled or the necessary binary installed the following commands can be used to use the two applications ([sniffer] indicates the sniffer application and [sender] indicates the sender application): + +Both sniffer and sender use the pcapplusplus library which is an actively maintained open source library. + ``` - [sniffer] : launches the sniffer and records all packets into a file named out.pcapng. - [sniffer] -a : will either append the packets to out.pcapng or to a custom filename if given. @@ -203,9 +210,6 @@ Both the sender and the sniffer can be compiled via bazel or sonic-buildimage. E The packet metadata carried with process_callback_function gets put into a comment in the pcapng. If the sniffer is to be run outside of P4Runtime the user might want to construct their own custom receive thread using customCallbackReceive found in the header file for the sniffer, since the carried metadata might be different. -## Sender application: -The `sender` application is currently used for testing purposes. It can create a packet, add the appropriate metadata and then send it to the generic netlink device where the `listening` application can pick it up and act on it. It is also useful for recreating error conditions where generic netlink functionality is part of the causality chain. -New applications that are designed to send to the generic netlink devices directly instead of using a kernel module can use this code as a template. ``` - sudo [sender] : will send a sample packet using genetlink. From 352fe6e403f6ce7eadc48bf8adf4c80726b6ef21 Mon Sep 17 00:00:00 2001 From: Don Newton Date: Wed, 22 Jun 2022 13:08:10 -0400 Subject: [PATCH 06/10] Further tweaks --- doc/pins/Packet_io.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/doc/pins/Packet_io.md b/doc/pins/Packet_io.md index d655f601b2..5832a2cd48 100644 --- a/doc/pins/Packet_io.md +++ b/doc/pins/Packet_io.md @@ -192,14 +192,6 @@ std::thread StartReceive( ## Sniffer Application: The `sniffer` application provides the means of a `tcpdump`-like tool to listen to the genetlink device. The sniffer can be used for listening to traffic, as well as recording the traffic into a file or displaying to standard out. The resulting pcapng file can then be viewed using Wireshark. The `sender` application can be used to send an example packet or packets from a pcap/pcapng file through genetlink. The sender also registers a new genetlink family and group called genl_packet and packets respectively. -## Sender application: -The `sender` application is currently used for testing purposes. It can create a packet, add the appropriate metadata and then send it to the generic netlink device where the `listening` application can pick it up and act on it. It is also useful for recreating error conditions where generic netlink functionality is part of the causality chain. -New applications that are designed to send to the generic netlink devices directly instead of using a kernel module can use this code as a template. - -Both the sender and the sniffer can be compiled via bazel or sonic-buildimage. Either way, once compiled or the necessary binary installed the following commands can be used to use the two applications ([sniffer] indicates the sniffer application and [sender] indicates the sender application): - -Both sniffer and sender use the pcapplusplus library which is an actively maintained open source library. - ``` - [sniffer] : launches the sniffer and records all packets into a file named out.pcapng. - [sniffer] -a : will either append the packets to out.pcapng or to a custom filename if given. @@ -208,11 +200,21 @@ Both sniffer and sender use the pcapplusplus library which is an actively mainta - [sniffer] -verbose : will print out verbose information about the packets received including metadata and packet contents. ``` -The packet metadata carried with process_callback_function gets put into a comment in the pcapng. If the sniffer is to be run outside of P4Runtime the user might want to construct their own custom receive thread using customCallbackReceive found in the header file for the sniffer, since the carried metadata might be different. - +## Sender application: +The `sender` application is currently used for testing purposes. It can create a packet, add the appropriate metadata and then send it to the generic netlink device where the `listening` application can pick it up and act on it. It is also useful for recreating error conditions where generic netlink functionality is part of the causality chain. +New applications that are designed to send to the generic netlink devices directly instead of using a kernel module can use this code as a template. ``` - sudo [sender] : will send a sample packet using genetlink. - sudo [sender] -inputfile=hello.pcapng : will read the packets from a given file and send them via genetlink. - sudo [sender] -packet=AABBCCDD : will send the given packet in hex representation via genetlink. ``` + +Both the sender and the sniffer can be compiled via bazel or sonic-buildimage. Either way, once compiled or the necessary binary installed the following commands can be used to use the two applications ([sniffer] indicates the sniffer application and [sender] indicates the sender application): + +Both sniffer and sender use the pcapplusplus library which is an actively maintained open source library. + + +The packet metadata carried with process_callback_function gets put into a comment in the pcapng. If the sniffer is to be run outside of P4Runtime the user might want to construct their own custom receive thread using customCallbackReceive found in the header file for the sniffer, since the carried metadata might be different. + + From d84acffa39f9481a4657bd08d08fb56812ad937b Mon Sep 17 00:00:00 2001 From: Don Newton Date: Wed, 22 Jun 2022 13:14:42 -0400 Subject: [PATCH 07/10] reordering --- doc/pins/Packet_io.md | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/doc/pins/Packet_io.md b/doc/pins/Packet_io.md index 5832a2cd48..f6ab206f1a 100644 --- a/doc/pins/Packet_io.md +++ b/doc/pins/Packet_io.md @@ -189,32 +189,28 @@ std::thread StartReceive( - Multicast group / queue functionality is likely unchanged. - Packet metadata can be specified by the using application in the P4 program.   -## Sniffer Application: +## Sniffer and Sender Applications: The `sniffer` application provides the means of a `tcpdump`-like tool to listen to the genetlink device. The sniffer can be used for listening to traffic, as well as recording the traffic into a file or displaying to standard out. The resulting pcapng file can then be viewed using Wireshark. The `sender` application can be used to send an example packet or packets from a pcap/pcapng file through genetlink. The sender also registers a new genetlink family and group called genl_packet and packets respectively. +The `sender` application is currently used for testing purposes. It can create a packet, add the appropriate metadata and then send it to the generic netlink device where the `listening` application can pick it up and act on it. It is also useful for recreating error conditions where generic netlink functionality is part of the causality chain. +New applications that are designed to send to the generic netlink devices directly instead of using a kernel module can use this code as a template. + +Both the sender and the sniffer can be compiled via bazel or sonic-buildimage. Either way, once compiled or the necessary binary installed the following commands can be used to use the two applications ([sniffer] indicates the sniffer application and [sender] indicates the sender application): + +Both sniffer and sender use the pcapplusplus library which is an actively maintained open source library. + +The packet metadata carried with process_callback_function gets put into a comment in the pcapng. If the sniffer is to be run outside of P4Runtime the user might want to construct their own custom receive thread using customCallbackReceive found in the header file for the sniffer, since the carried metadata might be different. + + ``` - [sniffer] : launches the sniffer and records all packets into a file named out.pcapng. - [sniffer] -a : will either append the packets to out.pcapng or to a custom filename if given. - [sniffer] -o=- : This will print the hex representation of the received packets to standard out. - [sniffer] -o=hello.pcapng : By providing a filename you can write the genetlink packets into a given file. In this example it will be hello.pcapng. - [sniffer] -verbose : will print out verbose information about the packets received including metadata and packet contents. -``` -## Sender application: -The `sender` application is currently used for testing purposes. It can create a packet, add the appropriate metadata and then send it to the generic netlink device where the `listening` application can pick it up and act on it. It is also useful for recreating error conditions where generic netlink functionality is part of the causality chain. -New applications that are designed to send to the generic netlink devices directly instead of using a kernel module can use this code as a template. - -``` - sudo [sender] : will send a sample packet using genetlink. - sudo [sender] -inputfile=hello.pcapng : will read the packets from a given file and send them via genetlink. - sudo [sender] -packet=AABBCCDD : will send the given packet in hex representation via genetlink. ``` -Both the sender and the sniffer can be compiled via bazel or sonic-buildimage. Either way, once compiled or the necessary binary installed the following commands can be used to use the two applications ([sniffer] indicates the sniffer application and [sender] indicates the sender application): - -Both sniffer and sender use the pcapplusplus library which is an actively maintained open source library. - - -The packet metadata carried with process_callback_function gets put into a comment in the pcapng. If the sniffer is to be run outside of P4Runtime the user might want to construct their own custom receive thread using customCallbackReceive found in the header file for the sniffer, since the carried metadata might be different. - - From 01c7d12bfdd9c10cddfe0b417285a64e81cd9483 Mon Sep 17 00:00:00 2001 From: Don Newton Date: Wed, 6 Jul 2022 11:43:56 -0400 Subject: [PATCH 08/10] Rename sender app to generator to avoid confusion --- doc/pins/Packet_io.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/doc/pins/Packet_io.md b/doc/pins/Packet_io.md index f6ab206f1a..54513d9e2c 100644 --- a/doc/pins/Packet_io.md +++ b/doc/pins/Packet_io.md @@ -189,15 +189,15 @@ std::thread StartReceive( - Multicast group / queue functionality is likely unchanged. - Packet metadata can be specified by the using application in the P4 program.   -## Sniffer and Sender Applications: -The `sniffer` application provides the means of a `tcpdump`-like tool to listen to the genetlink device. The sniffer can be used for listening to traffic, as well as recording the traffic into a file or displaying to standard out. The resulting pcapng file can then be viewed using Wireshark. The `sender` application can be used to send an example packet or packets from a pcap/pcapng file through genetlink. The sender also registers a new genetlink family and group called genl_packet and packets respectively. +## Sniffer and Packet Generator Applications: +The `sniffer` application provides the means of a `tcpdump`-like tool to listen to the genetlink device. The sniffer can be used for listening to traffic, as well as recording the traffic into a file or displaying to standard out. The resulting pcapng file can then be viewed using Wireshark. The `generator` application can be used to send an example packet or packets from a pcap/pcapng file through the genetlink interface. The generator also registers a new genetlink family and group called genl_packet and packets respectively. -The `sender` application is currently used for testing purposes. It can create a packet, add the appropriate metadata and then send it to the generic netlink device where the `listening` application can pick it up and act on it. It is also useful for recreating error conditions where generic netlink functionality is part of the causality chain. +The `generator` application is currently used for testing purposes. It can create a packet, add the appropriate metadata and then send it to the generic netlink device where the `listening` application can pick it up and act on it. It is also useful for recreating error conditions where generic netlink functionality is part of the causality chain. It should be noted that the generator does not send packets out any other type of interface whether physical or kernel based. New applications that are designed to send to the generic netlink devices directly instead of using a kernel module can use this code as a template. -Both the sender and the sniffer can be compiled via bazel or sonic-buildimage. Either way, once compiled or the necessary binary installed the following commands can be used to use the two applications ([sniffer] indicates the sniffer application and [sender] indicates the sender application): +Both the generator and the sniffer can be compiled via bazel or sonic-buildimage. Either way, once compiled or the necessary binary installed the following commands can be used to use the two applications ([sniffer] indicates the sniffer application and [generator] indicates the packet generator application): -Both sniffer and sender use the pcapplusplus library which is an actively maintained open source library. +Both sniffer and generator use the pcapplusplus library which is an actively maintained open source library. The packet metadata carried with process_callback_function gets put into a comment in the pcapng. If the sniffer is to be run outside of P4Runtime the user might want to construct their own custom receive thread using customCallbackReceive found in the header file for the sniffer, since the carried metadata might be different. @@ -209,8 +209,8 @@ The packet metadata carried with process_callback_function gets put into a comme - [sniffer] -o=hello.pcapng : By providing a filename you can write the genetlink packets into a given file. In this example it will be hello.pcapng. - [sniffer] -verbose : will print out verbose information about the packets received including metadata and packet contents. -- sudo [sender] : will send a sample packet using genetlink. -- sudo [sender] -inputfile=hello.pcapng : will read the packets from a given file and send them via genetlink. -- sudo [sender] -packet=AABBCCDD : will send the given packet in hex representation via genetlink. +- sudo [generator] : will send a sample packet using genetlink. +- sudo [generator] -inputfile=hello.pcapng : will read the packets from a given file and send them via genetlink. +- sudo [generator] -packet=AABBCCDD : will send the given packet in hex representation via genetlink. ``` From d9d70d01ad001c5528abd63e66483009e5604d66 Mon Sep 17 00:00:00 2001 From: Don Newton Date: Fri, 5 Aug 2022 18:13:51 +0000 Subject: [PATCH 09/10] updating pins.hld to point to Packet_io.hld --- doc/pins/pins_hld.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/pins/pins_hld.md b/doc/pins/pins_hld.md index a7588f8f7e..33dd588313 100644 --- a/doc/pins/pins_hld.md +++ b/doc/pins/pins_hld.md @@ -281,7 +281,7 @@ Here is the full list of supplementary HLD docs: [p4rt-db-hld]: in_progress.md [p4orch-hld]: p4orch_hld.md [appl-state-hld]: in_progress.md -[packet-hld]: in_progress.md +[packet-hld]: Packet_io.md [buildimage-repo]: https://github.com/Azure/sonic-buildimage From 8034f83c3a979e304dda532ef41cfa335e66c36f Mon Sep 17 00:00:00 2001 From: Don Newton Date: Tue, 9 Aug 2022 15:47:38 -0400 Subject: [PATCH 10/10] dummy --- doc/pins/Packet_io.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/pins/Packet_io.md b/doc/pins/Packet_io.md index 54513d9e2c..0cb7c501ef 100644 --- a/doc/pins/Packet_io.md +++ b/doc/pins/Packet_io.md @@ -212,5 +212,6 @@ The packet metadata carried with process_callback_function gets put into a comme - sudo [generator] : will send a sample packet using genetlink. - sudo [generator] -inputfile=hello.pcapng : will read the packets from a given file and send them via genetlink. - sudo [generator] -packet=AABBCCDD : will send the given packet in hex representation via genetlink. + ```