Driver: Implement audio scheduler (#179)

This PR implements a custom scheduler for audio threads, which reduces thread use and (often) memory consumption.

To save threads and memory (e.g., packet buffer allocations), Songbird parks Mixer tasks which do not have any live Tracks.
These are now all co-located on a single async 'Idle' task.
This task is responsible for managing UDP keepalive messages for each task, maintaining event state, and executing any Mixer task messages.
Whenever any message arrives which adds a `Track`, the mixer task is moved to a live thread.
The Idle task inspects task counts and execution time on each thread, choosing the first live thread with room, and creating a new one if needed.

Each live thread is responsible for running as many live mixers as it can in a single tick every 20ms: this currently defaults to 16 mixers per thread, but is user-configurable.
A live thread also stores RTP packet blocks to be written into by each sub-task.
Each live thread has a conservative limit of 18ms that it will aim to stay under: if all work takes longer than this, it will offload the task with the highest mixing cost once per tick onto another (possibly new) live worker thread.

ARCHITECTURE.md

@@ -35,7 +35,7 @@ Audio processing remains synchronous for the following reasons:
 Songbird subdivides voice connection handling into several long- and short-lived tasks.
 
 * **Core**: Handles and directs commands received from the driver. Responsible for connection/reconnection, and creates network tasks.
-* **Mixer**: Combines audio sources together, Opus encodes the result, and encrypts the built packets every 20ms. Responsible for handling track commands/state, and transmitting completed voice packets and keepalive messages. ***Synchronous***.
+* **Mixer**: Combines audio sources together, Opus encodes the result, and encrypts the built packets every 20ms. Responsible for handling track commands/state, and transmitting completed voice packets and keepalive messages. ***Synchronous when live***.
 * **Thread Pool**: A dynamically sized thread-pool for I/O tasks. Creates lazy tracks using `Compose` if sync creation is needed, otherwise spawns a tokio task. Seek operations always go to the thread pool. ***Synchronous***.
 * **Disposer**: Used by mixer thread to dispose of data with potentially long/blocking `Drop` implementations (i.e., audio sources). ***Synchronous***.
 * **Events**: Stores and runs event handlers, tracks event timing, and handles 
@@ -46,6 +46,33 @@ Songbird subdivides voice connection handling into several long- and short-lived
 
 ![](images/driver.png)
 
+## Scheduler
+To save threads and memory (e.g., packet buffer allocations), Songbird parks Mixer tasks which do not have any live Tracks.
+These are all co-located on a single async task.
+This task is responsible for managing UDP keepalive messages for each task, maintaining event state, and executing any Mixer task messages.
+Whenever any message arrives which adds a `Track`, the mixer task is moved to a live thread.
+The Idle task inspects task counts and execution time on each thread, choosing the first live thread with room, creating a new one if needed.
+
+Each live thread is responsible for running as many live mixers as it can in a single tick every 20ms: this currently defaults to 16 mixers per thread, but is user-configurable.
+A live thread also stores RTP packet blocks to be written into by each sub-task.
+Audio threads have a budget of 20ms to complete all message handling, mixing, encoding, and encryption.
+*These threads are synchronous, as explained above: the bulk costs (i.e., encoding) are compute-bound work and would block the Tokio executor.*
+Mixer logic is handled in this order to minimise deadline variance:
+```
+handle idle->live messages
+handle all driver->mixer messages
+cleanup idle/dead mixers
+mix + encode + encrypt all mixers into packet buffer
+check for excess packet blocks
+sleep 'til next 20ms boundary
+
+send all packets, adjust RTP fields
+handle per-track messages
+```
+Each live thread has a conservative limit of 18ms that it will aim to stay under: if all work takes longer than this, it will offload the task with the highest mixing cost once per 20ms tick.
+
+![](images/scheduler.png)
+
 ```
 src/driver/*
 ```

Cargo.toml

@@ -23,6 +23,7 @@ derivative = "2"
 discortp = { default-features = false, features = ["discord", "pnet", "rtp"], optional = true, version = "0.5" }
 flume = { optional = true, version = "0.10" }
 futures = "0.3"
+nohash-hasher = { optional = true, version = "0.2.0" }
 once_cell = { optional = true, version = "1" }
 parking_lot = { optional = true, version = "0.12" }
 pin-project = "1"
@@ -62,11 +63,11 @@ version = "0.11"
 optional = true
 
 [dev-dependencies]
+byteorder = "1"
 criterion = "0.4"
 ntest = "0.9"
 symphonia = { version = "0.5.2", features = ["aac", "isomp4", "mp3"] }
-utils = { path = "utils" }
-tokio = { version = "1", features = ["rt", "rt-multi-thread"] }
+tokio = { version = "1", features = ["rt", "rt-multi-thread", "test-util"] }
 
 [features]
 # Core features
@@ -93,6 +94,7 @@ driver = [
     "dep:discortp",
     "dep:reqwest",
     "dep:flume",
+    "dep:nohash-hasher",
     "dep:once_cell",
     "dep:parking_lot",
     "dep:rand",
@@ -141,7 +143,10 @@ receive = ["dep:bytes", "discortp?/demux", "discortp?/rtcp"]
 
 # Used for docgen/testing/benchmarking.
 full-doc = ["default", "twilight", "builtin-queue", "receive"]
-internals = []
+internals = ["dep:byteorder"]
+
+[lib]
+bench = false
 
 [[bench]]
 name = "base-mixing"

benches/base-mixing.rs

@@ -6,6 +6,7 @@ use songbird::{
         MixMode,
     },
     input::{codecs::*, Input, LiveInput, Parsed},
+    test_utils as utils,
 };
 use std::io::Cursor;
 use symphonia_core::audio::{AudioBuffer, Layout, SampleBuffer, Signal, SignalSpec};