Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: prepare v0.11.0 #2239

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Conversation

mxinden
Copy link
Collaborator

@mxinden mxinden commented Nov 19, 2024

New release to get #2237 into mozilla-central.

Minor bump (v0.10.0 -> v0.11.0) due to #2216.

Copy link

codecov bot commented Nov 19, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.37%. Comparing base (c6b518c) to head (ee96421).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2239   +/-   ##
=======================================
  Coverage   95.37%   95.37%           
=======================================
  Files         112      112           
  Lines       36569    36569           
=======================================
  Hits        34879    34879           
  Misses       1690     1690           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

Copy link

Failed Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

@mxinden
Copy link
Collaborator Author

mxinden commented Nov 19, 2024

mozilla-central depends on url v2.5.1:

https://searchfox.org/mozilla-central/rev/6050bf4eca89956c9d91bfd89fa59294ae32a689/Cargo.lock#6715-6725

The latest version of url is v2.5.3, which we automatically updated to, given that we don't check in a Cargo.lock (yet #2208).

url v0.2.5.3introduced thestd` feature which broke our CI. #2219 enabled it.

Given that mozilla-central still uses url v2.5.1 without the std feature, we can't use Neqo v0.11.0 as it is today in mozilla-central.

Moving to draft for now, until I figure out a way forward.

@mxinden mxinden marked this pull request as draft November 19, 2024 08:56
Copy link

Benchmark results

Performance differences relative to c6b518c.

coalesce_acked_from_zero 1+1 entries: 💚 Performance has improved.
       time:   [86.150 ns 86.421 ns 86.698 ns]
       change: [-5.6942% -5.3228% -4.9586%] (p = 0.00 < 0.05)

Found 9 outliers among 100 measurements (9.00%)
3 (3.00%) high mild
6 (6.00%) high severe

coalesce_acked_from_zero 3+1 entries: 💚 Performance has improved.
       time:   [97.211 ns 97.435 ns 97.690 ns]
       change: [-5.3141% -4.3594% -3.6973%] (p = 0.00 < 0.05)

Found 10 outliers among 100 measurements (10.00%)
2 (2.00%) high mild
8 (8.00%) high severe

coalesce_acked_from_zero 10+1 entries: 💚 Performance has improved.
       time:   [97.002 ns 97.326 ns 97.734 ns]
       change: [-4.0799% -3.6789% -3.2980%] (p = 0.00 < 0.05)

Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) high mild
5 (5.00%) high severe

coalesce_acked_from_zero 1000+1 entries: 💚 Performance has improved.
       time:   [78.464 ns 78.646 ns 78.859 ns]
       change: [-5.7377% -4.5812% -3.2875%] (p = 0.00 < 0.05)

Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe

RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [111.42 ms 111.56 ms 111.79 ms]
       change: [-0.5023% -0.3626% -0.1511%] (p = 0.00 < 0.05)

Found 10 outliers among 100 measurements (10.00%)
8 (8.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe

transfer/pacing-false/varying-seeds: No change in performance detected.
       time:   [26.031 ms 27.189 ms 28.337 ms]
       change: [-8.3682% -2.8845% +2.9754%] (p = 0.34 > 0.05)
transfer/pacing-true/varying-seeds: Change within noise threshold.
       time:   [33.134 ms 34.830 ms 36.554 ms]
       change: [-13.596% -8.0261% -1.4016%] (p = 0.02 < 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

transfer/pacing-false/same-seed: No change in performance detected.
       time:   [25.003 ms 25.725 ms 26.463 ms]
       change: [-6.9113% -3.0369% +1.2852%] (p = 0.15 > 0.05)
transfer/pacing-true/same-seed: No change in performance detected.
       time:   [41.354 ms 43.411 ms 45.501 ms]
       change: [-9.6995% -3.9091% +2.3295%] (p = 0.21 > 0.05)
1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: No change in performance detected.
       time:   [902.42 ms 910.67 ms 919.04 ms]
       thrpt:  [108.81 MiB/s 109.81 MiB/s 110.81 MiB/s]
change:
       time:   [-0.8746% +0.5176% +1.8675%] (p = 0.46 > 0.05)
       thrpt:  [-1.8333% -0.5150% +0.8823%]
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected.
       time:   [314.43 ms 317.36 ms 320.24 ms]
       thrpt:  [31.226 Kelem/s 31.510 Kelem/s 31.803 Kelem/s]
change:
       time:   [-2.3404% -1.0136% +0.3366%] (p = 0.15 > 0.05)
       thrpt:  [-0.3355% +1.0240% +2.3965%]
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: Change within noise threshold.
       time:   [33.901 ms 34.160 ms 34.437 ms]
       thrpt:  [29.039  elem/s 29.274  elem/s 29.498  elem/s]
change:
       time:   [+0.1278% +1.2092% +2.2109%] (p = 0.03 < 0.05)
       thrpt:  [-2.1631% -1.1948% -0.1277%]

Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low mild
7 (7.00%) high mild
2 (2.00%) high severe

1-conn/1-100mb-resp/mtu-1504 (aka. Upload)/client: Change within noise threshold.
       time:   [1.6410 s 1.6581 s 1.6755 s]
       thrpt:  [59.684 MiB/s 60.310 MiB/s 60.937 MiB/s]
change:
       time:   [+0.0812% +1.7350% +3.4571%] (p = 0.05 < 0.05)
       thrpt:  [-3.3415% -1.7054% -0.0812%]

Client/server transfer results

Transfer of 33554432 bytes over loopback.

Client Server CC Pacing MTU Mean [ms] Min [ms] Max [ms]
gquiche gquiche 1504 576.7 ± 72.1 506.1 720.4
neqo gquiche reno on 1504 786.6 ± 87.1 740.2 1028.1
neqo gquiche reno 1504 771.3 ± 21.4 745.1 805.2
neqo gquiche cubic on 1504 790.9 ± 45.0 760.7 898.5
neqo gquiche cubic 1504 783.3 ± 41.2 752.9 882.9
msquic msquic 1504 254.2 ± 125.3 103.6 480.7
neqo msquic reno on 1504 218.5 ± 13.9 198.2 237.6
neqo msquic reno 1504 317.0 ± 110.1 212.0 497.0
neqo msquic cubic on 1504 217.1 ± 15.4 199.2 247.7
neqo msquic cubic 1504 246.1 ± 56.9 212.6 401.1
gquiche neqo reno on 1504 758.6 ± 114.7 608.1 950.7
gquiche neqo reno 1504 761.4 ± 164.0 587.2 1120.8
gquiche neqo cubic on 1504 740.9 ± 84.8 624.3 875.5
gquiche neqo cubic 1504 793.0 ± 83.6 631.9 907.0
msquic neqo reno on 1504 717.2 ± 10.2 700.3 729.5
msquic neqo reno 1504 773.5 ± 105.2 692.0 975.4
msquic neqo cubic on 1504 727.8 ± 18.9 702.5 767.2
msquic neqo cubic 1504 721.1 ± 12.0 705.0 741.2
neqo neqo reno on 1504 682.9 ± 52.9 648.6 828.3
neqo neqo reno 1504 682.0 ± 48.1 656.2 816.4
neqo neqo cubic on 1504 661.2 ± 36.3 633.8 757.9
neqo neqo cubic 1504 637.9 ± 57.9 611.2 801.5

⬇️ Download logs

@larseggert
Copy link
Collaborator

Sounds like we need to pin url to 2.5.1?

@mxinden mxinden mentioned this pull request Nov 19, 2024
@mxinden
Copy link
Collaborator Author

mxinden commented Nov 26, 2024

Waiting for https://phabricator.services.mozilla.com/D229493 to land, updating mozilla-central to url v2.5.4. I will proceed here afterwards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants