feat: Faster UDP/IO on Apple platforms #1993

larseggert · 2024-09-20T10:25:25Z

This uses Apple's private sendmsg_x and recvmsg_x system calls for multi-packet UDP I/O.

Ralith · 2024-09-20T19:40:32Z

Is there interest in seeing TX support via sendmsg_x?

We found there wasn't much performance benefit, and was considerable difficulty taking advantage of, sendmmsg-style batching. IIRC the _x functions on macOS have more to offer than that, though. Will this unblock segmentation offload or other incidental optimizations?

quinn-udp/build.rs

larseggert · 2024-09-23T06:02:47Z

Bench on main:

test large_data_10_streams  ... bench:  27,558,791 ns/iter (+/- 13,459,810) = 380 MB/s
test large_data_1_stream    ... bench:  24,324,266 ns/iter (+/- 19,219,937) = 43 MB/s
test small_data_100_streams ... bench:  19,437,900 ns/iter (+/- 20,065,941)
test small_data_1_stream    ... bench:  11,465,128 ns/iter (+/- 8,699,934)

Bench with this PR:

test large_data_10_streams  ... bench:  28,829,216 ns/iter (+/- 15,924,956) = 363 MB/s
test large_data_1_stream    ... bench:  14,354,999 ns/iter (+/- 20,039,122) = 73 MB/s
test small_data_100_streams ... bench:  14,061,741 ns/iter (+/- 17,311,517)
test small_data_1_stream    ... bench:  19,194,441 ns/iter (+/- 5,012,070)

Surprised that large_data_10_streams and small_data_1_stream are slower...

Ralith · 2024-09-23T06:34:07Z

Those tests tend to be extremely noisy, as the huge variance suggests. A targeted quinn-udp benchmark might be more useful.

larseggert · 2024-09-23T06:42:37Z

We've also found on neqo that multi-packet RX without multi-packet TX has limited benefits, since the RX batch size will be very small.

larseggert · 2024-09-23T07:29:18Z

I added sendmsg_x support, mostly to see what the performance difference would be. But it seems that none of the benches or tests call send with a Transmit struct where segment_size is not None?

mxinden · 2024-09-23T09:36:20Z

A targeted quinn-udp benchmark might be more useful.

How about using the throughput.rs benchmark @larseggert?

https://github.com/quinn-rs/quinn/blob/main/quinn-udp/benches/throughput.rs

larseggert · 2024-09-23T12:32:18Z

With @mxinden's benchmark. Baseline:

gso_true/throughput     time:   [58.076 ms 58.230 ms 58.387 ms]
                        thrpt:  [171.27 MiB/s 171.73 MiB/s 172.19 MiB/s]

Only sendmsg_x:

gso_true/throughput     time:   [15.143 ms 15.189 ms 15.236 ms]
                        thrpt:  [656.35 MiB/s 658.36 MiB/s 660.37 MiB/s]
                 change:
                        time:   [-74.028% -73.915% -73.808%] (p = 0.00 < 0.05)
                        thrpt:  [+281.80% +283.36% +285.04%]
                        Performance has improved.

Both sendmsg_x and recvmsg_x:

gso_true/throughput     time:   [12.632 ms 12.682 ms 12.731 ms]
                        thrpt:  [785.46 MiB/s 788.53 MiB/s 791.61 MiB/s]
                 change:
                        time:   [-78.321% -78.221% -78.112%] (p = 0.00 < 0.05)
                        thrpt:  [+356.88% +359.16% +361.27%]
                        Performance has improved.

Both sendmsg_x and recvmsg_x with BATCH_SIZE of 64:

gso_true/throughput     time:   [11.640 ms 11.682 ms 11.725 ms]
                        thrpt:  [852.85 MiB/s 856.00 MiB/s 859.07 MiB/s]
                 change:
                        time:   [-80.030% -79.938% -79.844%] (p = 0.00 < 0.05)
                        thrpt:  [+396.13% +398.45% +400.75%]
                        Performance has improved.

djc

LGTM.

Please squash all of the changes into a single commit?

quinn-udp/benches/throughput.rs

quinn-udp/src/unix.rs

mxinden

Impressive results. Great to see the MacOS _x syscalls work for QUIC UDP IO.

quinn-udp/src/unix.rs

Ralith

Can we enable real GSO/GRO using these interfaces?

quinn-udp/benches/throughput.rs

larseggert · 2024-09-24T04:51:53Z

No. They are the equivalent of the mmsg Linux calls. AFAIK Apple doesn't have GSO/GRO via the socket interface.

quinn-udp/src/cmsg/unix.rs

quinn-udp/src/unix.rs

quinn-udp/benches/throughput.rs

larseggert · 2024-10-01T16:24:34Z

Are you waiting on anything from me on this?

mxinden

quinn-udp/benches/throughput.rs will need more changes to still support non-apple platform. @larseggert I believe we will need to either run it multi-threaded, or use some kind of executor, e.g. tokio. I can prepare a commit in the next couple of days. Sorry for missing this in earlier reviews.

Changes itself look good to me.

quinn-udp/benches/throughput.rs

quinn-udp/src/unix.rs

Done.

djc · 2024-10-09T14:32:56Z

@Ralith can you do another round on this one?

larseggert · 2024-10-10T05:16:33Z

Once @mxinden's fix to the bench is in, I will rebase and squash this PR.

AndrewDryga · 2024-10-10T17:43:56Z

Hey guys 👋, is there any chance Apple won't approve apps that are using those private syscalls in the App Store? They are notorious for doing so and even de-listing apps for using anything "undocumented". See 2.5.1 here: https://developer.apple.com/app-store/review/guidelines/

See one of such cases here: https://9to5mac.com/2019/11/04/electron-app-rejections/

How they will find out? Apple employs automated tools to scan apps for the usage of private APIs. If sendmsg_x and recvmsg_x are detected, the app is at risk of being flagged.

larseggert · 2024-10-10T17:49:20Z

The use of the private syscalls is now behind a non-default feature.

AndrewDryga · 2024-10-10T17:51:22Z

@larseggert should we add a big fat warning saying that if you enable this flag you will violate Apple ToS so it's only should be enabled if app is not distributed via App Store (or notarized for EU)?

This uses Apple's private sendmsg_x and recvmsg_x system calls for multi-packet UDP I/O.

larseggert · 2024-10-24T12:16:23Z

Shouldn't the features CI workflow have included fast-apple-datapath? I don't see it in the logs. (Or I am missing something.)

mxinden · 2024-10-24T12:33:52Z

Should be fine I believe:

running `cargo check --no-default-features --features direct-log,fast-apple-datapath,log,tracing` on quinn-udp (1754/1772)
running `cargo check --no-default-features --features default` on quinn-udp (1755/1772)
running `cargo check --no-default-features --features direct-log` on quinn-udp (1756/1772)
running `cargo check --no-default-features --features default,direct-log` on quinn-udp (1757/1772)
running `cargo check --no-default-features --features fast-apple-datapath` on quinn-udp (1758/1772)
running `cargo check --no-default-features --features default,fast-apple-datapath` on quinn-udp (1759/1772)
running `cargo check --no-default-features --features direct-log,fast-apple-datapath` on quinn-udp (1760/1772)
running `cargo check --no-default-features --features default,direct-log,fast-apple-datapath` on quinn-udp (1761/1772)
running `cargo check --no-default-features --features log` on quinn-udp (1762/1772)
running `cargo check --no-default-features --features direct-log,log` on quinn-udp (1763/1772)
running `cargo check --no-default-features --features fast-apple-datapath,log` on quinn-udp (1764/1772)
running `cargo check --no-default-features --features direct-log,fast-apple-datapath,log` on quinn-udp (1765/1772)
running `cargo check --no-default-features --features tracing` on quinn-udp (1766/1772)
running `cargo check --no-default-features --features direct-log,tracing` on quinn-udp (1767/1772)
running `cargo check --no-default-features --features fast-apple-datapath,tracing` on quinn-udp (1768/1772)
running `cargo check --no-default-features --features direct-log,fast-apple-datapath,tracing` on quinn-udp (1769/1772)
running `cargo check --no-default-features --features log,tracing` on quinn-udp (1770/1772)
running `cargo check --no-default-features --features direct-log,log,tracing` on quinn-udp (1771/1772)
running `cargo check --no-default-features --features fast-apple-datapath,log,tracing` on quinn-udp (1772/1772)

Currently we use `quinn-udp` `v0.5.4`. `quinn-udp` `v0.5.5` fixes [`recvmmsg` calls on Android x86](quinn-rs/quinn#1966). `quinn-udp` `v0.5.6` adds [experimental multi-message support on Apple platforms](quinn-rs/quinn#1993) and [fixes an unnecessary `windows-sys` version restriction](quinn-rs/quinn#2021). While not strictly necessary, given that our current version specification (i.e. `version = "0.5.4"`) already allows users to use Neqo with `quinn-udp` `v0.5.6`, this commit updates to `quinn-udp` `v0.5.6` anyways, thus making sure CI tests with latest version. In case mozilla#2208 lands, future compatible version updates would touch the `Cargo.lock` file, not `Cargo.toml`.

Currently we use `quinn-udp` `v0.5.4`. `quinn-udp` `v0.5.5` fixes [`recvmmsg` calls on Android x86](quinn-rs/quinn#1966). `quinn-udp` `v0.5.6` adds [experimental multi-message support on Apple platforms](quinn-rs/quinn#1993) and [fixes an unnecessary `windows-sys` version restriction](quinn-rs/quinn#2021). While not strictly necessary, given that our current version specification (i.e. `version = "0.5.4"`) already allows users to use Neqo with `quinn-udp` `v0.5.6`, this commit updates to `quinn-udp` `v0.5.6` anyways, thus making sure CI tests with latest version. In case #2208 lands, future compatible version updates would touch the `Cargo.lock` file, not `Cargo.toml`.

Currently we use `quinn-udp` `v0.5.4`. `quinn-udp` `v0.5.5` fixes [`recvmmsg` calls on Android x86](quinn-rs/quinn#1966). `quinn-udp` `v0.5.6` adds [experimental multi-message support on Apple platforms](quinn-rs/quinn#1993) and [fixes an unnecessary `windows-sys` version restriction](quinn-rs/quinn#2021). While not strictly necessary, given that our current version specification (i.e. `version = "0.5.4"`) already allows users to use Neqo with `quinn-udp` `v0.5.6`, this commit updates to `quinn-udp` `v0.5.6` anyways, thus making sure CI tests with latest version. In case #2208 lands, future compatible version updates would touch the `Cargo.lock` file, not `Cargo.toml`. Co-authored-by: Lars Eggert <[email protected]>

mxinden · 2024-11-27T19:26:14Z

quinn-udp/src/unix.rs

+        io::ErrorKind::Interrupted => {
+            // Retry the transmission
+        }


Given that this is not executed in a loop, it doesn't actually retry, but instead drop the packet to be sent.

We do retry on Linux:

quinn/quinn-udp/src/unix.rs

Lines 311 to 314 in 2edf192

io::ErrorKind::Interrupted => {

// Retry the transmission

continue;

}

But not on the BSDs or the slow Apple:

quinn/quinn-udp/src/unix.rs

Lines 433 to 435 in 2edf192

io::ErrorKind::Interrupted => {

// Retry the transmission

}

I don't recall whether we discussed this. @larseggert was this by design?

I think this was an oversight. It would be good to have identical behavior.

Agreed. @larseggert I will follow-up with a pull request.

With quinn-rs#2017, the concrete `send` implementations per platform are supposed to propagate `io::Error`s. Those errors are then eiter logged and dropped in `UdpSocketState::send` or further propagated in `UdpSocketState::try_send`. The `fast_apple` `send` implementation added in quinn-rs#1993 does not follow this pattern. This commit adjusts the `fast_apple` implementation accordingly.

With #2017, the concrete `send` implementations per platform are supposed to propagate `io::Error`s. Those errors are then eiter logged and dropped in `UdpSocketState::send` or further propagated in `UdpSocketState::try_send`. The `fast_apple` `send` implementation added in #1993 does not follow this pattern. This commit adjusts the `fast_apple` implementation accordingly.

Ralith reviewed Sep 20, 2024

View reviewed changes

quinn-udp/build.rs Outdated Show resolved Hide resolved

larseggert marked this pull request as ready for review September 23, 2024 09:10

djc approved these changes Sep 23, 2024

View reviewed changes

quinn-udp/benches/throughput.rs Outdated Show resolved Hide resolved

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

mxinden reviewed Sep 23, 2024

View reviewed changes

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

Ralith previously requested changes Sep 23, 2024

View reviewed changes

quinn-udp/benches/throughput.rs Outdated Show resolved Hide resolved

djc reviewed Sep 24, 2024

View reviewed changes

quinn-udp/src/cmsg/unix.rs Outdated Show resolved Hide resolved

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

quinn-udp/src/unix.rs Show resolved Hide resolved

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

mxinden reviewed Sep 24, 2024

View reviewed changes

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

Ralith reviewed Sep 25, 2024

View reviewed changes

quinn-udp/benches/throughput.rs Outdated Show resolved Hide resolved

quinn-udp/benches/throughput.rs Outdated Show resolved Hide resolved

larseggert requested a review from Ralith October 1, 2024 16:24

larseggert mentioned this pull request Oct 3, 2024

perf: don't allocate in UDP send & recv path mozilla/neqo#2093

Closed

mxinden reviewed Oct 3, 2024

View reviewed changes

mxinden mentioned this pull request Oct 8, 2024

GSO on macOS #1554

Open

mxinden reviewed Oct 8, 2024

View reviewed changes

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

quinn-udp/src/unix.rs Outdated Show resolved Hide resolved

mxinden mentioned this pull request Oct 8, 2024

bench(udp): support all GSO and recvmmsg permutations across platforms larseggert/quinn#1

Closed

mxinden mentioned this pull request Oct 10, 2024

bench(udp): run GSO, GRO and recvmmsg permutations #2010

Merged

feat: Faster UDP/IO on Apple platforms

d9e8564

This uses Apple's private sendmsg_x and recvmsg_x system calls for multi-packet UDP I/O.

larseggert force-pushed the feat-apple-datapath branch from de8f280 to d9e8564 Compare October 24, 2024 11:52

djc added this pull request to the merge queue Oct 25, 2024

Merged via the queue into quinn-rs:main with commit adc4a06 Oct 25, 2024
14 checks passed

larseggert deleted the feat-apple-datapath branch October 25, 2024 07:29

mxinden mentioned this pull request Oct 28, 2024

chore(quinn-udp): increase crate patch version to v0.5.6 #2022

Merged

mxinden mentioned this pull request Oct 30, 2024

chore: update to quinn-udp v0.5.6 mozilla/neqo#2209

Merged

thomaseizinger mentioned this pull request Nov 1, 2024

feat(connlib): utilise GSO for UDP sockets firezone/firezone#7210

Merged

psumbera added a commit to psumbera/quinn that referenced this pull request Nov 5, 2024

Unbreak Solaris build after quinn-rs#1993

9ef3057

psumbera added a commit to psumbera/quinn that referenced this pull request Nov 5, 2024

Unbreak Solaris build after quinn-rs#1993

79453bc

djc pushed a commit that referenced this pull request Nov 5, 2024

Unbreak Solaris build after #1993

722835b

djc mentioned this pull request Nov 11, 2024

Unable to build quinn-udp (0.5.7) for NetBSD: error[E0428]: the name "recv" is defined multiple times #2033

Closed

mxinden mentioned this pull request Nov 16, 2024

perf: use recvmmsg in addition to GRO mozilla/neqo#2137

Closed

mxinden reviewed Nov 27, 2024

View reviewed changes

mxinden mentioned this pull request Nov 29, 2024

fix(udp): propagate error on apple_fast #2073

Merged

mxinden mentioned this pull request Dec 10, 2024

perf(udp): enable multi-message IO on apple mozilla/neqo#2279

Closed

mxinden mentioned this pull request Dec 23, 2024

perf(udp): multi-message receive on apple mozilla/neqo#2302

Merged

mxinden mentioned this pull request Jan 9, 2025

ci: run quinn-udp tests with fast-apple-datapath #2130

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Faster UDP/IO on Apple platforms #1993

feat: Faster UDP/IO on Apple platforms #1993

larseggert commented Sep 20, 2024 •

edited

Loading

Ralith commented Sep 20, 2024

larseggert commented Sep 23, 2024

Ralith commented Sep 23, 2024

larseggert commented Sep 23, 2024

larseggert commented Sep 23, 2024

mxinden commented Sep 23, 2024

larseggert commented Sep 23, 2024 •

edited

Loading

djc left a comment

mxinden left a comment

Ralith left a comment

larseggert commented Sep 24, 2024

larseggert commented Oct 1, 2024

mxinden left a comment

djc commented Oct 9, 2024

larseggert commented Oct 10, 2024 •

edited

Loading

AndrewDryga commented Oct 10, 2024 •

edited

Loading

larseggert commented Oct 10, 2024

AndrewDryga commented Oct 10, 2024 •

edited

Loading

larseggert commented Oct 24, 2024

mxinden commented Oct 24, 2024

mxinden Nov 27, 2024

larseggert Nov 28, 2024

mxinden Nov 28, 2024

	io::ErrorKind::Interrupted => {
	// Retry the transmission
	continue;
	}

feat: Faster UDP/IO on Apple platforms #1993

feat: Faster UDP/IO on Apple platforms #1993

Conversation

larseggert commented Sep 20, 2024 • edited Loading

Ralith commented Sep 20, 2024

larseggert commented Sep 23, 2024

Ralith commented Sep 23, 2024

larseggert commented Sep 23, 2024

larseggert commented Sep 23, 2024

mxinden commented Sep 23, 2024

larseggert commented Sep 23, 2024 • edited Loading

djc left a comment

Choose a reason for hiding this comment

mxinden left a comment

Choose a reason for hiding this comment

Ralith left a comment

Choose a reason for hiding this comment

larseggert commented Sep 24, 2024

larseggert commented Oct 1, 2024

mxinden left a comment

Choose a reason for hiding this comment

djc commented Oct 9, 2024

larseggert commented Oct 10, 2024 • edited Loading

AndrewDryga commented Oct 10, 2024 • edited Loading

larseggert commented Oct 10, 2024

AndrewDryga commented Oct 10, 2024 • edited Loading

larseggert commented Oct 24, 2024

mxinden commented Oct 24, 2024

mxinden Nov 27, 2024

Choose a reason for hiding this comment

larseggert Nov 28, 2024

Choose a reason for hiding this comment

mxinden Nov 28, 2024

Choose a reason for hiding this comment

larseggert commented Sep 20, 2024 •

edited

Loading

larseggert commented Sep 23, 2024 •

edited

Loading

larseggert commented Oct 10, 2024 •

edited

Loading

AndrewDryga commented Oct 10, 2024 •

edited

Loading

AndrewDryga commented Oct 10, 2024 •

edited

Loading