Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Pin nightly #2335

Merged
merged 3 commits into from
Jan 9, 2025
Merged

ci: Pin nightly #2335

merged 3 commits into from
Jan 9, 2025

Conversation

larseggert
Copy link
Collaborator

Copy link
Collaborator

@mxinden mxinden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI is failing though.

@larseggert larseggert enabled auto-merge January 9, 2025 15:34
Copy link

github-actions bot commented Jan 9, 2025

Failed Interop Tests

QUIC Interop Runner, client vs. server, differences relative to ef9e4dc.

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

@larseggert larseggert disabled auto-merge January 9, 2025 15:58
@larseggert larseggert merged commit db0a332 into mozilla:main Jan 9, 2025
61 of 62 checks passed
Copy link

github-actions bot commented Jan 9, 2025

Benchmark results

Performance differences relative to ef9e4dc.

decode 4096 bytes, mask ff: No change in performance detected.
       time:   [11.181 µs 11.214 µs 11.253 µs]
       change: [-0.4124% -0.0652% +0.2973%] (p = 0.73 > 0.05)

Found 14 outliers among 100 measurements (14.00%)
4 (4.00%) low severe
2 (2.00%) low mild
8 (8.00%) high severe

decode 1048576 bytes, mask ff: No change in performance detected.
       time:   [2.8876 ms 2.8972 ms 2.9082 ms]
       change: [-0.4753% +0.0511% +0.5632%] (p = 0.85 > 0.05)

Found 10 outliers among 100 measurements (10.00%)
10 (10.00%) high severe

decode 4096 bytes, mask 7f: No change in performance detected.
       time:   [19.535 µs 19.578 µs 19.629 µs]
       change: [-0.2555% +0.1160% +0.5499%] (p = 0.59 > 0.05)

Found 17 outliers among 100 measurements (17.00%)
2 (2.00%) low severe
2 (2.00%) low mild
1 (1.00%) high mild
12 (12.00%) high severe

decode 1048576 bytes, mask 7f: No change in performance detected.
       time:   [5.0713 ms 5.0847 ms 5.0997 ms]
       change: [-0.4410% -0.0417% +0.3670%] (p = 0.83 > 0.05)

Found 16 outliers among 100 measurements (16.00%)
16 (16.00%) high severe

decode 4096 bytes, mask 3f: No change in performance detected.
       time:   [5.5359 µs 5.5655 µs 5.6002 µs]
       change: [-0.8246% -0.1849% +0.3660%] (p = 0.57 > 0.05)

Found 12 outliers among 100 measurements (12.00%)
5 (5.00%) low mild
7 (7.00%) high severe

decode 1048576 bytes, mask 3f: No change in performance detected.
       time:   [1.4154 ms 1.4222 ms 1.4292 ms]
       change: [-0.4060% +0.1913% +0.8735%] (p = 0.57 > 0.05)

Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) high mild
7 (7.00%) high severe

coalesce_acked_from_zero 1+1 entries: No change in performance detected.
       time:   [98.846 ns 99.160 ns 99.481 ns]
       change: [-0.2428% +0.2529% +0.7848%] (p = 0.33 > 0.05)

Found 14 outliers among 100 measurements (14.00%)
10 (10.00%) high mild
4 (4.00%) high severe

coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [116.62 ns 116.94 ns 117.30 ns]
       change: [-0.3030% +0.0773% +0.4877%] (p = 0.72 > 0.05)

Found 15 outliers among 100 measurements (15.00%)
4 (4.00%) low mild
3 (3.00%) high mild
8 (8.00%) high severe

coalesce_acked_from_zero 10+1 entries: No change in performance detected.
       time:   [116.36 ns 116.74 ns 117.23 ns]
       change: [-0.5172% +0.3326% +1.2595%] (p = 0.50 > 0.05)

Found 13 outliers among 100 measurements (13.00%)
5 (5.00%) low mild
1 (1.00%) high mild
7 (7.00%) high severe

coalesce_acked_from_zero 1000+1 entries: No change in performance detected.
       time:   [97.137 ns 97.278 ns 97.429 ns]
       change: [-0.9701% -0.1223% +0.7525%] (p = 0.79 > 0.05)

Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe

RxStreamOrderer::inbound_frame(): No change in performance detected.
       time:   [111.26 ms 111.31 ms 111.36 ms]
       change: [-0.1435% -0.0692% -0.0021%] (p = 0.06 > 0.05)

Found 19 outliers among 100 measurements (19.00%)
8 (8.00%) low mild
11 (11.00%) high mild

SentPackets::take_ranges: No change in performance detected.
       time:   [5.4137 µs 5.5205 µs 5.6404 µs]
       change: [-16.352% -5.0630% +3.0796%] (p = 0.55 > 0.05)

Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe

transfer/pacing-false/varying-seeds: No change in performance detected.
       time:   [42.883 ms 42.964 ms 43.046 ms]
       change: [-0.4753% -0.2120% +0.0402%] (p = 0.12 > 0.05)

Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high mild

transfer/pacing-true/varying-seeds: No change in performance detected.
       time:   [43.147 ms 43.218 ms 43.289 ms]
       change: [-0.3981% -0.1601% +0.1000%] (p = 0.20 > 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

transfer/pacing-false/same-seed: Change within noise threshold.
       time:   [42.928 ms 42.988 ms 43.049 ms]
       change: [+0.2140% +0.4186% +0.6263%] (p = 0.00 < 0.05)
transfer/pacing-true/same-seed: Change within noise threshold.
       time:   [43.305 ms 43.365 ms 43.424 ms]
       change: [-0.4091% -0.2116% -0.0144%] (p = 0.04 < 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild

1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: No change in performance detected.
       time:   [870.87 ms 880.68 ms 890.68 ms]
       thrpt:  [112.27 MiB/s 113.55 MiB/s 114.83 MiB/s]
change:
       time:   [-2.3979% -0.8399% +0.7895%] (p = 0.30 > 0.05)
       thrpt:  [-0.7833% +0.8470% +2.4568%]
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected.
       time:   [300.28 ms 302.46 ms 304.69 ms]
       thrpt:  [32.820 Kelem/s 33.062 Kelem/s 33.302 Kelem/s]
change:
       time:   [-1.0647% -0.0528% +0.9050%] (p = 0.92 > 0.05)
       thrpt:  [-0.8969% +0.0528% +1.0762%]

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: Change within noise threshold.
       time:   [34.191 ms 34.350 ms 34.528 ms]
       thrpt:  [28.962  elem/s 29.112  elem/s 29.248  elem/s]
change:
       time:   [-1.9124% -1.0565% -0.2243%] (p = 0.02 < 0.05)
       thrpt:  [+0.2248% +1.0678% +1.9497%]

Found 10 outliers among 100 measurements (10.00%)
5 (5.00%) low mild
1 (1.00%) high mild
4 (4.00%) high severe

1-conn/1-100mb-resp/mtu-1504 (aka. Upload)/client: No change in performance detected.
       time:   [1.6830 s 1.7010 s 1.7193 s]
       thrpt:  [58.164 MiB/s 58.790 MiB/s 59.418 MiB/s]
change:
       time:   [-1.0266% +0.3753% +1.8060%] (p = 0.60 > 0.05)
       thrpt:  [-1.7740% -0.3739% +1.0373%]

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

Client/server transfer results

Transfer of 33554432 bytes over loopback.

Client Server CC Pacing MTU Mean [ms] Min [ms] Max [ms]
gquiche gquiche 1504 537.7 ± 51.4 501.0 641.6
neqo gquiche reno on 1504 760.7 ± 18.7 739.8 799.3
neqo gquiche reno 1504 768.4 ± 26.2 747.8 839.1
neqo gquiche cubic on 1504 807.7 ± 100.0 751.7 1031.1
neqo gquiche cubic 1504 836.5 ± 102.6 768.1 1048.5
msquic msquic 1504 189.1 ± 121.5 101.2 583.2
neqo msquic reno on 1504 249.7 ± 73.4 210.8 452.3
neqo msquic reno 1504 277.9 ± 92.7 204.1 457.6
neqo msquic cubic on 1504 242.6 ± 64.8 207.7 441.6
neqo msquic cubic 1504 241.3 ± 60.6 213.7 440.6
gquiche neqo reno on 1504 734.2 ± 76.8 645.6 860.5
gquiche neqo reno 1504 681.6 ± 85.8 553.3 808.3
gquiche neqo cubic on 1504 781.8 ± 130.5 615.8 1051.2
gquiche neqo cubic 1504 691.1 ± 106.7 557.4 919.3
msquic neqo reno on 1504 490.3 ± 26.3 461.3 532.1
msquic neqo reno 1504 502.2 ± 75.8 467.4 717.0
msquic neqo cubic on 1504 490.8 ± 49.8 467.3 631.6
msquic neqo cubic 1504 517.3 ± 88.8 460.8 725.4
neqo neqo reno on 1504 525.9 ± 47.2 442.0 595.0
neqo neqo reno 1504 598.5 ± 87.9 496.6 770.7
neqo neqo cubic on 1504 576.9 ± 91.2 498.2 829.7
neqo neqo cubic 1504 550.1 ± 83.7 474.9 730.2

⬇️ Download logs

Copy link

codecov bot commented Jan 9, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.34%. Comparing base (ef9e4dc) to head (daf9d53).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2335      +/-   ##
==========================================
- Coverage   93.37%   93.34%   -0.04%     
==========================================
  Files         114      114              
  Lines       36908    36908              
  Branches    36908    36908              
==========================================
- Hits        34464    34451      -13     
- Misses       1662     1675      +13     
  Partials      782      782              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@larseggert larseggert deleted the ci-pin-nightly branch January 10, 2025 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants