Add a chapter about logging custom structs with PerfEventArray

docs/book/start/getting-data-to-user-space.md

@@ -0,0 +1,169 @@
+# Getting data to user-space
+
+In previous chapters we were logging packets with aya-log. However, what if we
+need to send additional data about the packet or any other type of information
+for the user-space program to utilize? In this chapter, we will explore how to
+leverage perf buffers and define data structures in the *-common* crate to
+transfer data from eBPF program to user-space applications. By doing so,
+user-space programs can access and utilize the transferred data effectively.
+
+!!! example "Source code"
+
+    Full code for the example in this chapter is available
+    [here](https://github.com/aya-rs/book/tree/main/examples/xdp-perfbuf-custom-data)
+
+## Sharing data
+
+In this chapter, we will be sending data from the kernel space to the user space
+by writing it into a struct and outputting it to user space. We can achieve this
+by using an eBPF map. There are different types of maps available, but in this
+case, we will use `PerfEventArray`.
+
+`PerfEventArray` is a collection of per-CPU circular buffers that enable the
+kernel to emit events (defined as custom structs) to user space. Each CPU has
+its own buffer, and the eBPF program emits an event to the buffer of the CPU
+it's currently running on. The events are out of order, meaning that they arrive
+in the user-space in a different order than they were created and sent from the
+eBPF program.
+
+To gather events from all CPUs, a user space program needs to spawn a thread for
+each CPU to poll for the events and then iterate over them.
+
+The data structure we'll be using needs to hold an IPv4 address and a port.
+
+```rust linenums="1" title="xdp-perfbuf-custom-data-common/src/lib.rs"
+--8<-- "examples/xdp-perfbuf-custom-data/xdp-perfbuf-custom-data-common/src/lib.rs"
+```
+
+1. We implement the `aya::Pod` trait for our struct since it is Plain Old Data
+as can be safely converted to a byte slice and back.
+
+!!! tip "Alignment, padding and verifier errors"
+
+    At program load time, the eBPF verifier checks that all the memory used is
+    properly initialized. This can be a problem if - to ensure alignment - the
+    compiler inserts padding bytes between fields in your types.
+
+    **Example:**
+
+    ```rust
+    #[repr(C)]
+    struct SourceInfo {
+        source_port: u16,
+        source_ip: u32,
+    }
+
+    let source_port = ...;
+    let source_ip = ...;
+    let si = SourceInfo { source_port, source_ip };
+    ```
+
+    In the example above, the compiler will insert two extra bytes between the
+    struct fields `source_port` and `source_ip` to make sure that `source_ip` is
+    correctly aligned to a 4-byte address (assuming `mem::align_of::<u32>() ==
+    4`).  Since padding bytes are typically not initialized by the compiler,
+    this will result in the infamous `invalid indirect read from stack` verifier
+    error.
+
+    To avoid the error, you can either manually ensure that all the fields in
+    your types are correctly aligned (e.g. by explicitly adding padding or by
+    making field types larger to enforce alignment) or use `#[repr(packed)]`.
+    Since the latter comes with its own foot-guns and can perform less
+    efficiently, explicitly adding padding or tweaking alignment is recommended.
+
+    **Solution ensuring alignment using larger types:**
+
+    ```rust
+    #[repr(C)]
+    struct SourceInfo {
+        source_port: u32,
+        source_ip: u32,
+    }
+
+    let source_port = ...;
+    let source_ip = ...;
+    let si = SourceInfo { source_port, source_ip };
+    ```
+
+    **Solution with explicit padding:**
+
+    ```rust
+    #[repr(C)]
+    struct SourceInfo {
+        source_port: u16,
+        padding: u16,
+        source_ip: u32,
+    }
+
+    let source_port = ...;
+    let source_ip = ...;
+    let si = SourceInfo { source_port, padding: 0, source_ip };
+    ```
+
+## Extracting packet data from the context and into the map
+
+The eBPF program code in this section is similar to the one in the previous
+chapters. It extracts the source IP address and port information from packet
+headers.
+
+The difference is that after obtaining the data from the headers, we create a
+`PacketLog` struct and output it to our `PerfEventArray` instead of logging data
+directly.
+
+The resulting code looks like this:
+
+```rust linenums="1" title="xdp-perfbuf-custom-data-ebpf/src/main.rs"
+--8<-- "examples/xdp-perfbuf-custom-data/xdp-perfbuf-custom-data-ebpf/src/main.rs"
+```
+
+1. Create our map.
+2. Output the event to the map.
+
+## Reading data
+
+To read from the perf event array in user space, we need to choose one of the
+following types:
+
+* `AsyncPerfEventArray` which is designed for use with
+  [async Rust](https://rust-lang.github.io/async-book/01_getting_started/01_chapter.html).
+* `PerfEventArray`, intended for synchronous Rust.
+
+By default, [our project template](https://github.com/aya-rs/aya-template) is
+written in async Rust and uses the [Tokio runtime](https://tokio.rs/). Therefore,
+we will use `AsyncPerfEventArray` in this chapter.
+
+To read from the `AsyncPerfEventArray`, we must call
+`AsyncPerfEventArray::open()` for each online CPU and poll the file descriptor
+for events.
+
+Additionally, we need to add a dependency on `bytes` to `xdp-log/Cargo.toml`.
+This library simplifies handling the chunks of bytes yielded by the
+`AsyncPerfEventArray`.
+
+Here's the code:
+
+```rust linenums="1" title="xdp-perfbuf-custom-data/src/main.rs"
+--8<-- "examples/xdp-perfbuf-custom-data/xdp-perfbuf-custom-data/src/main.rs"
+```
+
+1. Define our map.
+2. Call `open()` for each online CPU.
+3. Spawn a `tokio::task`.
+4. Create buffers.
+5. Read events in to buffers.
+6. Use `read_unaligned` to read the event data into a `PacketLog`.
+7. Log the packet data.
+
+## Running the program
+
+As before, you can overwrite the interface by by providing the interface name as
+a parameter, for example, `RUST_LOG=info cargo xtask run -- --iface wlp2s0`.
+
+```console
+$ RUST_LOG=info cargo xtask run
+[2023-01-25T08:57:41Z INFO  xdp_perfbuf_custom_data] SRC IP: 60.235.240.157, SRC_PORT: 443
+[2023-01-25T08:57:41Z INFO  xdp_perfbuf_custom_data] SRC IP: 98.21.76.76, SRC_PORT: 443
+[2023-01-25T08:57:41Z INFO  xdp_perfbuf_custom_data] SRC IP: 95.194.217.172, SRC_PORT: 443
+[2023-01-25T08:57:41Z INFO  xdp_perfbuf_custom_data] SRC IP: 95.194.217.172, SRC_PORT: 443
+[2023-01-25T08:57:41Z INFO  xdp_perfbuf_custom_data] SRC IP: 95.10.251.142, SRC_PORT: 443
+```

examples/xdp-perfbuf-custom-data/.cargo/config.toml

@@ -0,0 +1,2 @@
+[alias]
+xtask = "run --package xtask --"

examples/xdp-perfbuf-custom-data/.gitignore

@@ -0,0 +1,13 @@
+### https://raw.github.com/github/gitignore/master/Rust.gitignore
+
+# Generated by Cargo
+# will have compiled files and executables
+debug/
+target/
+
+# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
+# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
+Cargo.lock
+
+# These are backup files generated by rustfmt
+**/*.rs.bk

examples/xdp-perfbuf-custom-data/.vim/coc-settings.json

@@ -0,0 +1,3 @@
+{
+  "rust-analyzer.linkedProjects": ["Cargo.toml", "xdp-perfbuf-custom-data-ebpf/Cargo.toml"]
+}

examples/xdp-perfbuf-custom-data/.vscode/settings.json

@@ -0,0 +1,3 @@
+{
+  "rust-analyzer.linkedProjects": ["Cargo.toml", "xdp-perfbuf-custom-data-ebpf/Cargo.toml"]
+}

examples/xdp-perfbuf-custom-data/Cargo.toml

@@ -0,0 +1,2 @@
+[workspace]
+members = ["xdp-perfbuf-custom-data", "xdp-perfbuf-custom-data-common", "xtask"]

examples/xdp-perfbuf-custom-data/README.md

@@ -0,0 +1,28 @@
+# xdp-perfbuf-custom-data
+
+## Prerequisites
+
+1. Install a rust stable toolchain: `rustup install stable`
+1. Install a rust nightly toolchain: `rustup install nightly`
+1. Install bpf-linker: `cargo install bpf-linker`
+
+## Build eBPF
+
+```bash
+cargo xtask build-ebpf
+```
+
+To perform a release build you can use the `--release` flag.
+You may also change the target architecture with the `--target` flag
+
+## Build Userspace
+
+```bash
+cargo build
+```
+
+## Run
+
+```bash
+RUST_LOG=info cargo xtask run
+```

examples/xdp-perfbuf-custom-data/xdp-perfbuf-custom-data-common/Cargo.toml

@@ -0,0 +1,14 @@
+[package]
+name = "xdp-perfbuf-custom-data-common"
+version = "0.1.0"
+edition = "2021"
+
+[features]
+default = []
+user = [ "aya" ]
+
+[dependencies]
+aya = { version = ">=0.11", optional=true }
+
+[lib]
+path = "src/lib.rs"

examples/xdp-perfbuf-custom-data/xdp-perfbuf-custom-data-common/src/lib.rs

@@ -0,0 +1,11 @@
+#![no_std]
+
+#[repr(C)]
+#[derive(Clone, Copy)]
+pub struct PacketLog {
+    pub ipv4_address: u32,
+    pub port: u32,
+}
+
+#[cfg(feature = "user")]
+unsafe impl aya::Pod for PacketLog {} // (1)

examples/xdp-perfbuf-custom-data/xdp-perfbuf-custom-data-ebpf/.cargo/config.toml

@@ -0,0 +1,6 @@
+[build]
+target-dir = "../target"
+target = "bpfel-unknown-none"
+
+[unstable]
+build-std = ["core"]

examples/xdp-perfbuf-custom-data/xdp-perfbuf-custom-data-ebpf/.vim/coc-settings.json

@@ -0,0 +1,4 @@
+{
+    "rust-analyzer.cargo.target": "bpfel-unknown-none",
+    "rust-analyzer.checkOnSave.allTargets": false
+}

examples/xdp-perfbuf-custom-data/xdp-perfbuf-custom-data-ebpf/.vscode/settings.json

@@ -0,0 +1,4 @@
+{
+    "rust-analyzer.cargo.target": "bpfel-unknown-none",
+    "rust-analyzer.checkOnSave.allTargets": false
+}

examples/xdp-perfbuf-custom-data/xdp-perfbuf-custom-data-ebpf/Cargo.toml

@@ -0,0 +1,33 @@
+[package]
+name = "xdp-perfbuf-custom-data-ebpf"
+version = "0.1.0"
+edition = "2021"
+
+[dependencies]
+aya-bpf = { git = "https://github.com/aya-rs/aya", branch = "main" }
+aya-log-ebpf = { git = "https://github.com/aya-rs/aya", branch = "main" }
+xdp-perfbuf-custom-data-common = { path = "../xdp-perfbuf-custom-data-common" }
+network-types = "0.0.4"
+
+[[bin]]
+name = "xdp-perfbuf-custom-data"
+path = "src/main.rs"
+
+[profile.dev]
+opt-level = 3
+debug = false
+debug-assertions = false
+overflow-checks = false
+lto = true
+panic = "abort"
+incremental = false
+codegen-units = 1
+rpath = false
+
+[profile.release]
+lto = true
+panic = "abort"
+codegen-units = 1
+
+[workspace]
+members = []

examples/xdp-perfbuf-custom-data/xdp-perfbuf-custom-data-ebpf/rust-toolchain.toml

@@ -0,0 +1,5 @@
+[toolchain]
+channel="nightly"
+# The source code of rustc, provided by the rust-src component, is needed for
+# building eBPF programs.
+components = [ "rustc", "rust-std", "cargo", "rust-docs", "rustfmt", "clippy", "rust-src" ]

examples/xdp-perfbuf-custom-data/xdp-perfbuf-custom-data-ebpf/src/main.rs

@@ -0,0 +1,82 @@
+#![no_std]
+#![no_main]
+
+use core::mem;
+
+use aya_bpf::{
+    bindings::xdp_action,
+    macros::{map, xdp},
+    maps::PerfEventArray,
+    programs::XdpContext,
+};
+use network_types::{
+    eth::{EthHdr, EtherType},
+    ip::{IpProto, Ipv4Hdr},
+    tcp::TcpHdr,
+    udp::UdpHdr,
+};
+
+use xdp_perfbuf_custom_data_common::PacketLog;
+
+#[map(name = "EVENTS")] // (1)
+static EVENTS: PerfEventArray<PacketLog> =
+    PerfEventArray::<PacketLog>::with_max_entries(1024, 0);
+
+#[xdp(name = "xdp_perfbuf_custom_data")]
+pub fn xdp_perfbuf_custom_data(ctx: XdpContext) -> u32 {
+    match try_xdp_perfbuf_custom_data(ctx) {
+        Ok(ret) => ret,
+        Err(_) => xdp_action::XDP_ABORTED,
+    }
+}
+
+#[inline(always)]
+fn ptr_at<T>(ctx: &XdpContext, offset: usize) -> Result<*const T, ()> {
+    let start = ctx.data();
+    let end = ctx.data_end();
+    let len = mem::size_of::<T>();
+
+    if start + offset + len > end {
+        return Err(());
+    }
+
+    Ok((start + offset) as *const T)
+}
+
+fn try_xdp_perfbuf_custom_data(ctx: XdpContext) -> Result<u32, ()> {
+    let ethhdr: *const EthHdr = ptr_at(&ctx, 0)?;
+    match unsafe { (*ethhdr).ether_type } {
+        EtherType::Ipv4 => {}
+        _ => return Ok(xdp_action::XDP_PASS),
+    }
+
+    let ipv4hdr: *const Ipv4Hdr = ptr_at(&ctx, EthHdr::LEN)?;
+    let source_addr = unsafe { (*ipv4hdr).src_addr };
+
+    let source_port = match unsafe { (*ipv4hdr).proto } {
+        IpProto::Tcp => {
+            let tcphdr: *const TcpHdr =
+                ptr_at(&ctx, EthHdr::LEN + Ipv4Hdr::LEN)?;
+            u16::from_be(unsafe { (*tcphdr).source })
+        }
+        IpProto::Udp => {
+            let udphdr: *const UdpHdr =
+                ptr_at(&ctx, EthHdr::LEN + Ipv4Hdr::LEN)?;
+            u16::from_be(unsafe { (*udphdr).source })
+        }
+        _ => return Err(()),
+    };
+
+    let log_entry = PacketLog {
+        ipv4_address: source_addr,
+        port: source_port as u32,
+    };
+    EVENTS.output(&ctx, &log_entry, 0); // (2)
+
+    Ok(xdp_action::XDP_PASS)
+}
+
+#[panic_handler]
+fn panic(_info: &core::panic::PanicInfo) -> ! {
+    unsafe { core::hint::unreachable_unchecked() }
+}