chore: improve codec test coverage
#3248
Merged
+37
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add tests to eliminate missed mutants in `neqo-common/src/codec.rs`: - `builder_inas_decoder`: add `Decoder` inequality test - `partial_eq_respects_skip`: add `Encoder` inequality test - `buffer_vs_encoder_len`: add `!is_empty()` test for `Encoder` - `buffer_write_zeroes`: add `!is_empty()` test for `Buffer` - `truncate`: test `Encoder::truncate` - `with_capacity`: test `Encoder::with_capacity` - `buffer_truncate`: test `Buffer::truncate` for all buffer types
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3248 +/- ##
==========================================
- Coverage 93.50% 93.45% -0.05%
==========================================
Files 125 125
Lines 36637 36665 +28
Branches 36637 36665 +28
==========================================
+ Hits 34256 34265 +9
- Misses 1529 1548 +19
Partials 852 852
|
larseggert
requested review from
KershawChang,
martinthomson and
mxinden
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive test coverage for the codec module in neqo-common to eliminate missed mutants during mutation testing. The changes focus on testing edge cases and alternative code paths for Encoder, Decoder, and Buffer types, including inequality tests, empty/non-empty state transitions, and truncation operations.
Key Changes:
- Added inequality assertions to existing equality tests for
DecoderandEncoder - Added new test functions
truncate()andwith_capacity()forEncoder - Enhanced existing tests with
!is_empty()assertions after write operations - Added comprehensive
buffer_truncate()test covering all buffer type implementations
Contributor
Failed Interop TestsQUIC Interop Runner, client vs. server, differences relative to
All resultsSucceeded Interop TestsQUIC Interop Runner, client vs. server neqo-pr as client
neqo-pr as server
Unsupported Interop TestsQUIC Interop Runner, client vs. server neqo-pr as client
neqo-pr as server
|
Contributor
Client/server transfer resultsPerformance differences relative to ab7ae57. Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.
Table above only shows statistically significant changes. See all results below. All resultsTransfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.
Download data for |
Contributor
|
Bencher Report| Branch | chore-coverage-neqo-common/src/codec.rs |
| Testbed | On-prem |
🚨 1 Alert
| Iteration | Benchmark | Measure Units | View | Benchmark Result (Result Δ%) | Upper Boundary (Limit %) |
|---|---|---|---|---|---|
| 9 | neqo-s2n-cubic | Latency milliseconds (ms) | 📈 plot 🚷 threshold 🚨 alert (🔔) | 224.80 ms(+1.85%)Baseline: 220.72 ms | 224.25 ms (100.25%) |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| google-neqo-cubic | 📈 view plot 🚷 view threshold | 272.29 ms(-1.96%)Baseline: 277.73 ms | 287.14 ms (94.83%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| msquic-neqo-cubic | 📈 view plot 🚷 view threshold | 205.12 ms(-2.84%)Baseline: 211.11 ms | 243.24 ms (84.33%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo-google-cubic | 📈 view plot 🚷 view threshold | 768.51 ms(+0.43%)Baseline: 765.19 ms | 790.03 ms (97.28%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo-msquic-cubic | 📈 view plot 🚷 view threshold | 161.22 ms(+2.12%)Baseline: 157.88 ms | 161.38 ms (99.90%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo-neqo-cubic-nopacing | 📈 view plot 🚷 view threshold | 96.24 ms(-0.14%)Baseline: 96.38 ms | 98.28 ms (97.93%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo-neqo-cubic | 📈 view plot 🚷 view threshold | 97.71 ms(+0.24%)Baseline: 97.47 ms | 99.70 ms (98.01%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo-neqo-reno-nopacing | 📈 view plot 🚷 view threshold | 95.78 ms(-0.83%)Baseline: 96.58 ms | 99.22 ms (96.53%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo-neqo-reno | 📈 view plot 🚷 view threshold | 96.72 ms(-0.80%)Baseline: 97.49 ms | 99.66 ms (97.05%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo-quiche-cubic | 📈 view plot 🚷 view threshold | 194.57 ms(+1.10%)Baseline: 192.44 ms | 195.37 ms (99.59%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo-s2n-cubic | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 224.80 ms(+1.85%)Baseline: 220.72 ms | 224.25 ms (100.25%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| quiche-neqo-cubic | 📈 view plot 🚷 view threshold | 154.59 ms(+0.60%)Baseline: 153.67 ms | 156.45 ms (98.82%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| s2n-neqo-cubic | 📈 view plot 🚷 view threshold | 173.73 ms(-0.07%)Baseline: 173.84 ms | 177.16 ms (98.06%) |
Contributor
Benchmark resultsSignificant performance differences relative to 83e264c. 1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client: 💔 Performance has regressed by +1.5496%. time: [213.14 ms 213.55 ms 214.10 ms]
thrpt: [467.08 MiB/s 468.27 MiB/s 469.17 MiB/s]
change:
time: [+1.2953% +1.5496% +1.8457] (p = 0.00 < 0.05)
thrpt: [-1.8122% -1.5260% -1.2787]
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe1-streams/each-1000-bytes/wallclock-time: 💚 Performance has improved by -1.4733%. time: [580.17 µs 580.86 µs 581.53 µs]
change: [-1.9695% -1.4733% -1.0172] (p = 0.00 < 0.05)
Performance has improved.All results1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: Change within noise threshold. time: [207.34 ms 207.71 ms 208.15 ms]
thrpt: [480.42 MiB/s 481.44 MiB/s 482.29 MiB/s]
change:
time: [-0.8964% -0.6409% -0.3741] (p = 0.00 < 0.05)
thrpt: [+0.3755% +0.6450% +0.9045]
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high severe1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: Change within noise threshold. time: [280.07 ms 281.94 ms 283.89 ms]
thrpt: [35.225 Kelem/s 35.468 Kelem/s 35.706 Kelem/s]
change:
time: [-2.5873% -1.6826% -0.7940] (p = 0.00 < 0.05)
thrpt: [+0.8004% +1.7114% +2.6560]
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: No change in performance detected. time: [38.424 ms 38.544 ms 38.679 ms]
thrpt: [25.854 B/s 25.944 B/s 26.025 B/s]
change:
time: [-0.6963% -0.2124% +0.2497] (p = 0.39 > 0.05)
thrpt: [-0.2491% +0.2128% +0.7012]
No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high severe1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client: 💔 Performance has regressed by +1.5496%. time: [213.14 ms 213.55 ms 214.10 ms]
thrpt: [467.08 MiB/s 468.27 MiB/s 469.17 MiB/s]
change:
time: [+1.2953% +1.5496% +1.8457] (p = 0.00 < 0.05)
thrpt: [-1.8122% -1.5260% -1.2787]
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severedecode 4096 bytes, mask ff: No change in performance detected. time: [4.5199 µs 4.5284 µs 4.5378 µs]
change: [-0.2584% +0.0775% +0.4421] (p = 0.68 > 0.05)
No change in performance detected.
Found 25 outliers among 100 measurements (25.00%)
1 (1.00%) low severe
1 (1.00%) low mild
7 (7.00%) high mild
16 (16.00%) high severedecode 1048576 bytes, mask ff: No change in performance detected. time: [1.1606 ms 1.1625 ms 1.1645 ms]
change: [-0.8762% -0.0298% +0.8545] (p = 0.95 > 0.05)
No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
1 (1.00%) high mild
10 (10.00%) high severedecode 4096 bytes, mask 7f: No change in performance detected. time: [5.7880 µs 5.7969 µs 5.8066 µs]
change: [-0.4952% -0.0386% +0.3639] (p = 0.86 > 0.05)
No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severedecode 1048576 bytes, mask 7f: Change within noise threshold. time: [1.4753 ms 1.4780 ms 1.4810 ms]
change: [-0.8403% -0.5826% -0.3127] (p = 0.00 < 0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severedecode 4096 bytes, mask 3f: No change in performance detected. time: [5.5390 µs 5.5473 µs 5.5555 µs]
change: [-0.7752% -0.2821% +0.0858] (p = 0.22 > 0.05)
No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severedecode 1048576 bytes, mask 3f: No change in performance detected. time: [1.4155 ms 1.4178 ms 1.4202 ms]
change: [-0.1907% +0.0177% +0.2475] (p = 0.88 > 0.05)
No change in performance detected.1-streams/each-1000-bytes/wallclock-time: 💚 Performance has improved by -1.4733%. time: [580.17 µs 580.86 µs 581.53 µs]
change: [-1.9695% -1.4733% -1.0172] (p = 0.00 < 0.05)
Performance has improved.1-streams/each-1000-bytes/simulated-time: No change in performance detected. time: [119.06 ms 119.29 ms 119.52 ms]
thrpt: [8.1708 KiB/s 8.1866 KiB/s 8.2022 KiB/s]
change:
time: [-0.1673% +0.1048% +0.3680] (p = 0.43 > 0.05)
thrpt: [-0.3667% -0.1047% +0.1676]
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high mild1000-streams/each-1-bytes/wallclock-time: No change in performance detected. time: [12.314 ms 12.351 ms 12.389 ms]
change: [-0.1328% +0.2810% +0.6971] (p = 0.20 > 0.05)
No change in performance detected.1000-streams/each-1-bytes/simulated-time: No change in performance detected. time: [2.3294 s 2.3328 s 2.3362 s]
thrpt: [428.05 B/s 428.67 B/s 429.29 B/s]
change:
time: [-0.0672% +0.1306% +0.3336] (p = 0.20 > 0.05)
thrpt: [-0.3325% -0.1304% +0.0673]
No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild1000-streams/each-1000-bytes/wallclock-time: Change within noise threshold. time: [50.444 ms 50.596 ms 50.780 ms]
change: [+0.3777% +0.7468% +1.1823] (p = 0.00 < 0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe1000-streams/each-1000-bytes/simulated-time: No change in performance detected. time: [16.153 s 16.418 s 16.683 s]
thrpt: [58.538 KiB/s 59.483 KiB/s 60.456 KiB/s]
change:
time: [-2.8063% -0.6626% +1.5135] (p = 0.55 > 0.05)
thrpt: [-1.4909% +0.6671% +2.8873]
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mildcoalesce_acked_from_zero 1+1 entries: No change in performance detected. time: [89.195 ns 89.532 ns 89.890 ns]
change: [-0.7235% -0.2435% +0.2226] (p = 0.32 > 0.05)
No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severecoalesce_acked_from_zero 3+1 entries: No change in performance detected. time: [105.83 ns 106.22 ns 106.66 ns]
change: [-2.3779% -0.8760% +0.2865] (p = 0.23 > 0.05)
No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
12 (12.00%) high severecoalesce_acked_from_zero 10+1 entries: No change in performance detected. time: [105.29 ns 105.75 ns 106.30 ns]
change: [-1.1006% -0.4371% +0.1902] (p = 0.20 > 0.05)
No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
5 (5.00%) low severe
1 (1.00%) low mild
2 (2.00%) high mild
5 (5.00%) high severecoalesce_acked_from_zero 1000+1 entries: No change in performance detected. time: [90.104 ns 90.278 ns 90.477 ns]
change: [-1.1140% -0.5479% +0.0419] (p = 0.05 > 0.05)
No change in performance detected.
Found 19 outliers among 100 measurements (19.00%)
7 (7.00%) high mild
12 (12.00%) high severeRxStreamOrderer::inbound_frame(): Change within noise threshold. time: [109.50 ms 109.66 ms 109.85 ms]
change: [+0.3629% +0.5233% +0.6887] (p = 0.00 < 0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severesent::Packets::take_ranges: No change in performance detected. time: [4.4333 µs 4.5188 µs 4.5901 µs]
change: [-4.6580% -1.7437% +1.4714] (p = 0.30 > 0.05)
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mildtransfer/pacing-false/varying-seeds/wallclock-time/run: Change within noise threshold. time: [23.857 ms 23.889 ms 23.939 ms]
change: [+0.8444% +1.1007% +1.3635] (p = 0.00 < 0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severetransfer/pacing-false/varying-seeds/simulated-time/run: No change in performance detected. time: [23.941 s 23.941 s 23.941 s]
thrpt: [171.09 KiB/s 171.09 KiB/s 171.09 KiB/s]
change:
time: [+0.0000% +0.0000% +0.0000] (p = NaN > 0.05)
thrpt: [+0.0000% +0.0000% +0.0000]
No change in performance detected.transfer/pacing-true/varying-seeds/wallclock-time/run: Change within noise threshold. time: [24.096 ms 24.113 ms 24.131 ms]
change: [+0.7208% +0.8843% +1.0229] (p = 0.00 < 0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
3 (3.00%) high mildtransfer/pacing-true/varying-seeds/simulated-time/run: No change in performance detected. time: [23.676 s 23.676 s 23.676 s]
thrpt: [173.01 KiB/s 173.01 KiB/s 173.01 KiB/s]
change:
time: [+0.0000% +0.0000% +0.0000] (p = NaN > 0.05)
thrpt: [+0.0000% +0.0000% +0.0000]
No change in performance detected.transfer/pacing-false/same-seed/wallclock-time/run: Change within noise threshold. time: [23.619 ms 23.635 ms 23.651 ms]
change: [-1.2550% -1.1444% -1.0340] (p = 0.00 < 0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mildtransfer/pacing-false/same-seed/simulated-time/run: No change in performance detected. time: [23.941 s 23.941 s 23.941 s]
thrpt: [171.09 KiB/s 171.09 KiB/s 171.09 KiB/s]
change:
time: [+0.0000% +0.0000% +0.0000] (p = NaN > 0.05)
thrpt: [+0.0000% +0.0000% +0.0000]
No change in performance detected.transfer/pacing-true/same-seed/wallclock-time/run: Change within noise threshold. time: [24.065 ms 24.083 ms 24.101 ms]
change: [-0.6243% -0.5161% -0.4120] (p = 0.00 < 0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mildtransfer/pacing-true/same-seed/simulated-time/run: No change in performance detected. time: [23.676 s 23.676 s 23.676 s]
thrpt: [173.01 KiB/s 173.01 KiB/s 173.01 KiB/s]
change:
time: [+0.0000% +0.0000% +0.0000] (p = NaN > 0.05)
thrpt: [+0.0000% +0.0000% +0.0000]
No change in performance detected.Download data for |
martinthomson
approved these changes
Dec 14, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add tests to eliminate missed mutants in
neqo-common/src/codec.rs:builder_inas_decoder: addDecoderinequality testpartial_eq_respects_skip: addEncoderinequality testbuffer_vs_encoder_len: add!is_empty()test forEncoderbuffer_write_zeroes: add!is_empty()test forBuffertruncate: testEncoder::truncatewith_capacity: testEncoder::with_capacitybuffer_truncate: testBuffer::truncatefor all buffer types