Improve documentation

Dockerfile

@@ -22,9 +22,6 @@ RUN apt-get update && \
     apt-get purge -y curl `# Cleanup...` && \
     rm -rf '/var/lib/apt/lists/*'
 
-COPY c /c
-COPY csharp /csharp
-COPY ada /ada
-COPY rust /rust
+COPY . /.
 
 CMD ["bash"]

ReadMe.md

@@ -1,22 +1,22 @@
--- WEIRD: This MUST be of size 64, otherwise the card locks up quickly (even the heatup in the benchmarks doesn't finish)
+# TinyNF
 
-noinline run / inlined agent/queues run
+This repository contains the "TinyNF" driver codebase.
 
-# TinyNF
+It was originally associated with the paper ["A Simpler and Faster NIC Driver Model for Network Functions"](https://www.usenix.org/conference/osdi20/presentation/pirelli) (OSDI 2020).
+You can still find that version of the code, and the experimental scripts to reproduce that paper, in the `osdi20` branch.
 
-This repository contains code originally associated with the paper ["A Simpler and Faster NIC Driver Model for Network Functions"](https://www.usenix.org/conference/osdi20/presentation/pirelli) (OSDI 2020).
-It has been extended with more programming languages, a second driver model, and different scripts for experiments.
+It was extended for the paper ["Safe low-level code without overhead is practical"](https://conf.researchr.org/details/icse-2023/icse-2023-technical-track/18/Safe-low-level-code-without-overhead-is-practical) (ICSE 2023).
+The repo contains more programming languages, a second driver model, and scripts to reproduce the new paper's experiments.
 
 
 ## Code
 
 The code of the drivers is in the `ada`, `c`, `csharp`, and `rust` folders.
 We refer to "agents" in the code for the restricted TinyNF model and "queues" for the flexible DPDK model.
 
-All languages use a `Makefile.benchmarking` file to compile, which itself delegates to the language's native compiler / build system.
+All languages provide a `Makefile` to `build` and `format` the code, wrapping other build systems when needed.
 The following parameters are available:
 
-- `TN_CC` (`c` only): The compiler, tested with GCC and Clang
 - `TN_MODE`: The kind of driver:
   - `restricted` (default): The "restricted" model, which is the original TinyNF one
   - `const` (`ada`, `c`, and `rust` only): The restricted model with a constant number of devices, instead of detecting them at run-time
@@ -27,10 +27,29 @@ The following parameters are available:
 Note that despite needing extensions, the Rust driver does not support `TN_MODE=safe` because, due to Rust's ownership model,
 unsafe code _must_ be used in the hot loop for volatile reads and writes, whereas C# allows these reads and writes in safe code.
 
+All languages force inlining of the driver methods into a forced-not-inlined "run" method to ensure a fair comparison and to make extracting the hot loop code easy.
+
+All languages use a 64-bit integer to represent packet lengths, even though the card only supports 16 bits, because not doing so often causes the card to lock up in non-C implementations.
+(Yes, this sounds extremely weird, but that's what empirical evidence says...)
+
+
+## Dependencies
+
+To compile each language, you will need `make`, as well as a compiler:
+- `ada`: `gnat`, though any other compiler might work
+- `c`: `gcc` or `clang`, though any other C11 compiler should work
+- `csharp`: `dotnet`, version 7 or above
+- `rust`: `rustc`, a version that supports Rust 2021
+
+If you don't want to install those on your machine, we provide a `Dockerfile`, just run `docker build -t tinynf . ; docker run -it tinynf` (you might need `sudo` for Docker).
+This file is also useful if you want to know how to install the dependencies on any Ubuntu machine.
+(The Dockerfile does not include `clang-format`, which is used for `make format` in `c`, you'll have to install that manually if you want to auto-format the C code)
+
 
 ## Experiments
 
-The benchmarking scripts for NFs, which are independent of the rest, are in `benchmarking/`.
+The benchmarking scripts for network functions, which are independent of the rest, are in `benchmarking/`.
 
 The experiments presented in the paper, including replication instructions, are in `experiments/`.
-We've also provided the actual data collected on our hardware in `experiments/results_example`; you can rename the folder to `results` and run the scripts to plot it as per the instructions.
+We've also provided the actual data collected on our hardware in `experiments/results_example`;
+you can rename the folder to `results` and run the scripts to plot it as per the instructions.

ada/ReadMe.md

@@ -1 +1,6 @@
-This is the Ada version. Note that the queues are split in more files because the RX and TX functions need arrays of a generic argument, that requires a generic package, and GNAT requires 1 package per file.
+This is the Ada version.
+
+Note that the queues are split in more files because the RX and TX functions need arrays of a generic argument, that requires a generic package, and GNAT requires 1 package per file.
+
+Overall, this is a best-effort implementation in terms of code cleanliness, it was not written by Ada experts, far from it.
+But it works!

c/ixgbe/ReadMe.md

@@ -1,5 +1,5 @@
 All references in the code are to the Intel 82599 Data Sheet unless otherwise noted.
-It used to be publicly available but the link is dead, though you may have luck googling it.
+It used to be publicly available but the link is dead, though you may have luck looking it up.
 
 ### Interpretations
 

csharp/ReadMe.md

@@ -1,2 +1,3 @@
 This is the C# version, including the C# extensions in `TinyNF.Unsafe`.
+
 The code is split into three projects so that `TinyNF` can be compiled without passing the `/unsafe` switch to the compiler, ensuring it contains no unsafe code.

experiments/ReadMe.md

@@ -1,21 +1,27 @@
 # Experiments
 
-This folder contains experiments from the paper.
+The ICSE 2023 paper's central claim is that it is possible to write safe code with no run-time overhead compared to C.
+This is implemented as two driver models in C, Ada, C#, and Rust.
 
-**Table 2**:
-`cd code-metrics ; ./tabulate-metrics.sh`
-(this may slightly differ from the paper depending on your exact compiler versions)
+_By definition, you are unlikely to get the exact same results as the paper since it depends on your exact compiler versions and your CPU.
+However, the relative claims should hold._
 
-**Figures**:
-Check out the prerequisites below, `cd perf`, then `./bench.sh`, which will take a few hours.
-Then `./plot.sh` assuming you have Python with Matplotlib; run `. setup-virtualenv-graphing.sh` on Ubuntu to create a virtualenv with it if needed.
 
+## Theory (minutes)
 
-## Prerequisites
+You will need `cloc` in addition to the compilers mentioned in the top-level readme; or use the provided Dockerfile.
 
-Most of the experiments are about performance, measured with benchmarks.
+Run `cd code-metrics ; ./tabulate-metrics.sh` which will generate an `assembly/` directory containing the assembly code for all hot loops in all drivers.
+This can be manually checked for a lack of compiler-inserted bounds checks.
+That script also outputs **Table 2** from the paper, and takes less than a minute.
 
-To run these benchmarks, you need two machines running Linux:
+## Practice (hours)
+
+For the **figures**, with the exception of Figure 2 which is a simplified form of `rust/src/lifed.rs`, you need a proper testbed.
+Check out the prerequisites below, `cd perf`, then `./bench.sh`, which will take a few hours.
+Then `./plot.sh` assuming you have Python with Matplotlib; run `. setup-virtualenv-graphing.sh` on Ubuntu to create a virtualenv with it if needed.
+
+To run the benchmarks, you need two machines running Linux:
 - A "device under test" machine with two Intel 82599ES NICs on the same NUMA node, from which you will run the experiment scripts
 - A "tester" machine connected to the other one by two 10G Ethernet cables
 
@@ -33,12 +39,8 @@ Assuming a 2-CPU machine whose second CPU has cores 8 to 15, we recommend the fo
 - `idle=poll cpuidle.off=1`: Force the CPU to spin instead of using waits for idling
 - `intel_pstate=disable`: Allow Linux to set the CPU frequency via `cpupower` instead of letting the Intel driver choose
 
-You will also need the following software, in addition to the compilers for each language:
-- the NativeAOT dependencies: https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/
-
+You will also need the following software on the device under test machine, in addition to the dependencies mentioned in the top-level readme:
 - The library `libtbb2`, available under that name in most package repositories
-- The build tool `make`, available under that name in most package repositories
-- The shell utility `cloc`, available under that name in most package repositories
 - The shell utility `cpupower`, available under names such as `linux-tools-generic` (Ubuntu) in package repositories
 
 Due to how long some of these scripts take, if you are running them via SSH, you may want to use an utility such as `byobu`, `tmux`, or `screen`,

rust/ReadMe.md

@@ -1,3 +1 @@
 This is the Rust version. The extensions are in `src/lifed.rs`.
-
-Useful note: to auto-format, run `shopt -s globstar; rustfmt src/**/*.rs`