Skip to content

Commit dc0752c

Browse files
thomhurstthomhurst
authored
chore: update benchmark results (#3934)
1 parent 2abcffc commit dc0752c

18 files changed

+355
-351
lines changed

‎docs/docs/benchmarks/AsyncTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 2
77
# AsyncTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-28** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-29** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 576.3 ms | 576.2 ms | 5.87 ms |
20-
| NUnit | 4.4.0 | 689.7 ms | 688.0 ms | 7.29 ms |
21-
| MSTest | 4.0.2 | 644.4 ms | 643.1 ms | 9.95 ms |
22-
| xUnit3 | 3.2.0 | 734.3 ms | 729.7 ms | 10.57 ms |
23-
| **TUnit (AOT)** | 1.2.11 | 124.6 ms | 124.6 ms | 0.44 ms |
19+
| **TUnit** | 1.2.11 | 573.3 ms | 573.7 ms | 4.43 ms |
20+
| NUnit | 4.4.0 | 658.8 ms | 660.6 ms | 4.99 ms |
21+
| MSTest | 4.0.2 | 631.5 ms | 631.2 ms | 8.74 ms |
22+
| xUnit3 | 3.2.1 | 716.2 ms | 715.4 ms | 6.28 ms |
23+
| **TUnit (AOT)** | 1.2.11 | 124.1 ms | 124.2 ms | 0.20 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
5858
xychart-beta
5959
title "AsyncTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 882
62-
bar [576.3, 689.7, 644.4, 734.3, 124.6]
61+
y-axis "Time (ms)" 0 --> 860
62+
bar [573.3, 658.8, 631.5, 716.2, 124.1]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-28T00:29:30.392Z*
75+
*Last generated: 2025-11-29T00:27:17.985Z*

‎docs/docs/benchmarks/BuildTime.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 8
77
# Build Performance Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-28** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-29** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -18,10 +18,10 @@ Compilation time comparison across frameworks:
1818

1919
| Framework | Version | Mean | Median | StdDev |
2020
|-----------|---------|------|--------|--------|
21-
| **TUnit** | 1.2.11 | 2.206 s | 2.201 s | 0.0426 s |
22-
| Build_NUnit | 4.4.0 | 1.761 s | 1.762 s | 0.0320 s |
23-
| Build_MSTest | 4.0.2 | 1.828 s | 1.830 s | 0.0351 s |
24-
| Build_xUnit3 | 3.2.0 | 1.709 s | 1.702 s | 0.0388 s |
21+
| **TUnit** | 1.2.11 | 2.050 s | 2.047 s | 0.0253 s |
22+
| Build_NUnit | 4.4.0 | 1.644 s | 1.646 s | 0.0243 s |
23+
| Build_MSTest | 4.0.2 | 1.714 s | 1.709 s | 0.0303 s |
24+
| Build_xUnit3 | 3.2.1 | 1.639 s | 1.639 s | 0.0300 s |
2525

2626
## 📈 Visual Comparison
2727

@@ -60,7 +60,7 @@ xychart-beta
6060
title "Build Time Comparison"
6161
x-axis ["Build_TUnit", "Build_NUnit", "Build_MSTest", "Build_xUnit3"]
6262
y-axis "Time (s)" 0 --> 3
63-
bar [2.206, 1.761, 1.828, 1.709]
63+
bar [2.05, 1.644, 1.714, 1.639]
6464
```
6565

6666
---
@@ -69,4 +69,4 @@ xychart-beta
6969
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7070
:::
7171

72-
*Last generated: 2025-11-28T00:29:30.394Z*
72+
*Last generated: 2025-11-29T00:27:17.987Z*

‎docs/docs/benchmarks/DataDrivenTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 3
77
# DataDrivenTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-28** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-29** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 482.09 ms | 480.78 ms | 4.482 ms |
20-
| NUnit | 4.4.0 | 675.54 ms | 671.70 ms | 18.149 ms |
21-
| MSTest | 4.0.2 | 662.49 ms | 662.42 ms | 5.452 ms |
22-
| xUnit3 | 3.2.0 | 675.22 ms | 672.45 ms | 9.573 ms |
23-
| **TUnit (AOT)** | 1.2.11 | 24.02 ms | 24.07 ms | 0.159 ms |
19+
| **TUnit** | 1.2.11 | 501.86 ms | 501.49 ms | 4.693 ms |
20+
| NUnit | 4.4.0 | 535.87 ms | 534.70 ms | 12.790 ms |
21+
| MSTest | 4.0.2 | 502.71 ms | 503.69 ms | 12.277 ms |
22+
| xUnit3 | 3.2.1 | 579.28 ms | 578.02 ms | 8.138 ms |
23+
| **TUnit (AOT)** | 1.2.11 | 24.82 ms | 24.80 ms | 0.166 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
5858
xychart-beta
5959
title "DataDrivenTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 811
62-
bar [482.09, 675.54, 662.49, 675.22, 24.02]
61+
y-axis "Time (ms)" 0 --> 696
62+
bar [501.86, 535.87, 502.71, 579.28, 24.82]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-28T00:29:30.392Z*
75+
*Last generated: 2025-11-29T00:27:17.985Z*

‎docs/docs/benchmarks/MassiveParallelTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 4
77
# MassiveParallelTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-28** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-29** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 599.1 ms | 597.3 ms | 5.47 ms |
20-
| NUnit | 4.4.0 | 1,205.9 ms | 1,207.4 ms | 4.44 ms |
21-
| MSTest | 4.0.2 | 2,989.7 ms | 2,990.1 ms | 9.41 ms |
22-
| xUnit3 | 3.2.0 | 3,076.8 ms | 3,076.2 ms | 9.36 ms |
23-
| **TUnit (AOT)** | 1.2.11 | 131.0 ms | 131.0 ms | 0.47 ms |
19+
| **TUnit** | 1.2.11 | 597.5 ms | 597.8 ms | 3.59 ms |
20+
| NUnit | 4.4.0 | 1,183.2 ms | 1,182.1 ms | 4.90 ms |
21+
| MSTest | 4.0.2 | 2,966.5 ms | 2,968.0 ms | 5.16 ms |
22+
| xUnit3 | 3.2.1 | 3,066.1 ms | 3,065.1 ms | 15.64 ms |
23+
| **TUnit (AOT)** | 1.2.11 | 132.2 ms | 132.3 ms | 0.51 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
5858
xychart-beta
5959
title "MassiveParallelTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 3693
62-
bar [599.1, 1205.9, 2989.7, 3076.8, 131]
61+
y-axis "Time (ms)" 0 --> 3680
62+
bar [597.5, 1183.2, 2966.5, 3066.1, 132.2]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-28T00:29:30.393Z*
75+
*Last generated: 2025-11-29T00:27:17.986Z*

‎docs/docs/benchmarks/MatrixTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 5
77
# MatrixTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-28** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-29** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 563.40 ms | 563.35 ms | 4.161 ms |
20-
| NUnit | 4.4.0 | 1,537.43 ms | 1,536.67 ms | 8.606 ms |
21-
| MSTest | 4.0.2 | 1,490.73 ms | 1,491.63 ms | 5.278 ms |
22-
| xUnit3 | 3.2.0 | 1,582.33 ms | 1,582.19 ms | 8.848 ms |
23-
| **TUnit (AOT)** | 1.2.11 | 79.14 ms | 78.95 ms | 0.395 ms |
19+
| **TUnit** | 1.2.11 | 574.81 ms | 575.40 ms | 5.417 ms |
20+
| NUnit | 4.4.0 | 1,582.59 ms | 1,581.77 ms | 6.462 ms |
21+
| MSTest | 4.0.2 | 1,540.42 ms | 1,540.75 ms | 5.446 ms |
22+
| xUnit3 | 3.2.1 | 1,630.94 ms | 1,627.40 ms | 11.609 ms |
23+
| **TUnit (AOT)** | 1.2.11 | 79.00 ms | 79.08 ms | 0.235 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
5858
xychart-beta
5959
title "MatrixTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1899
62-
bar [563.4, 1537.43, 1490.73, 1582.33, 79.14]
61+
y-axis "Time (ms)" 0 --> 1958
62+
bar [574.81, 1582.59, 1540.42, 1630.94, 79]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-28T00:29:30.393Z*
75+
*Last generated: 2025-11-29T00:27:17.986Z*

‎docs/docs/benchmarks/ScaleTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 6
77
# ScaleTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-28** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-29** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 532.88 ms | 531.46 ms | 3.588 ms |
20-
| NUnit | 4.4.0 | 582.78 ms | 583.36 ms | 6.070 ms |
21-
| MSTest | 4.0.2 | 500.42 ms | 498.63 ms | 6.733 ms |
22-
| xUnit3 | 3.2.0 | 593.09 ms | 591.89 ms | 6.914 ms |
23-
| **TUnit (AOT)** | 1.2.11 | 45.31 ms | 45.54 ms | 3.177 ms |
19+
| **TUnit** | 1.2.11 | 535.07 ms | 534.68 ms | 4.870 ms |
20+
| NUnit | 4.4.0 | 585.44 ms | 584.11 ms | 12.531 ms |
21+
| MSTest | 4.0.2 | 509.11 ms | 505.09 ms | 11.032 ms |
22+
| xUnit3 | 3.2.1 | 594.97 ms | 593.95 ms | 6.401 ms |
23+
| **TUnit (AOT)** | 1.2.11 | 46.00 ms | 46.41 ms | 3.968 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
5858
xychart-beta
5959
title "ScaleTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 712
62-
bar [532.88, 582.78, 500.42, 593.09, 45.31]
61+
y-axis "Time (ms)" 0 --> 714
62+
bar [535.07, 585.44, 509.11, 594.97, 46]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-28T00:29:30.393Z*
75+
*Last generated: 2025-11-29T00:27:17.986Z*

‎docs/docs/benchmarks/SetupTeardownTests.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 7
77
# SetupTeardownTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-28** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-29** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,10 +16,10 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 571.6 ms | 571.4 ms | 3.64 ms |
20-
| NUnit | 4.4.0 | 1,159.0 ms | 1,158.6 ms | 8.21 ms |
21-
| MSTest | 4.0.2 | 1,138.2 ms | 1,134.9 ms | 12.44 ms |
22-
| xUnit3 | 3.2.0 | 1,210.7 ms | 1,208.0 ms | 11.71 ms |
19+
| **TUnit** | 1.2.11 | 558.0 ms | 557.6 ms | 3.82 ms |
20+
| NUnit | 4.4.0 | 1,148.5 ms | 1,146.8 ms | 8.24 ms |
21+
| MSTest | 4.0.2 | 1,120.1 ms | 1,121.5 ms | 6.42 ms |
22+
| xUnit3 | 3.2.1 | 1,198.8 ms | 1,197.8 ms | 9.10 ms |
2323
| **TUnit (AOT)** | 1.2.11 | NA | NA | NA |
2424

2525
## 📈 Visual Comparison
@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-28** from the latest CI
5858
xychart-beta
5959
title "SetupTeardownTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1453
62-
bar [571.6, 1159, 1138.2, 1210.7, 0]
61+
y-axis "Time (ms)" 0 --> 1439
62+
bar [558, 1148.5, 1120.1, 1198.8, 0]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-28T00:29:30.394Z*
75+
*Last generated: 2025-11-29T00:27:17.987Z*

‎docs/docs/benchmarks/index.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 1
77
# Performance Benchmarks
88

99
:::info Last Updated
10-
These benchmarks were automatically generated on **2025-11-28** from the latest CI run.
10+
These benchmarks were automatically generated on **2025-11-29** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -38,7 +38,7 @@ These benchmarks compare TUnit against the most popular .NET testing frameworks:
3838
| Framework | Version Tested |
3939
|-----------|----------------|
4040
| **TUnit** | 1.2.11 |
41-
| **xUnit v3** | 3.2.0 |
41+
| **xUnit v3** | 3.2.1 |
4242
| **NUnit** | 4.4.0 |
4343
| **MSTest** | 4.0.2 |
4444

@@ -80,4 +80,4 @@ These benchmarks run automatically daily via [GitHub Actions](https://github.com
8080
Each benchmark runs multiple iterations with statistical analysis to ensure accuracy. Results may vary based on hardware and test characteristics.
8181
:::
8282

83-
*Last generated: 2025-11-28T00:29:30.395Z*
83+
*Last generated: 2025-11-29T00:27:17.987Z*

‎docs/static/benchmarks/AsyncTests.json‎

Lines changed: 22 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"timestamp": "2025-11-28T00:29:30.392Z",
2+
"timestamp": "2025-11-29T00:27:17.985Z",
33
"category": "AsyncTests",
44
"environment": {
55
"benchmarkDotNetVersion": "BenchmarkDotNet v0.15.7, Linux Ubuntu 24.04.3 LTS (Noble Numbat)",
@@ -10,42 +10,42 @@
1010
{
1111
"Method": "TUnit",
1212
"Version": "1.2.11",
13-
"Mean": "576.3 ms",
14-
"Error": "6.27 ms",
15-
"StdDev": "5.87 ms",
16-
"Median": "576.2 ms"
13+
"Mean": "573.3 ms",
14+
"Error": "4.73 ms",
15+
"StdDev": "4.43 ms",
16+
"Median": "573.7 ms"
1717
},
1818
{
1919
"Method": "NUnit",
2020
"Version": "4.4.0",
21-
"Mean": "689.7 ms",
22-
"Error": "7.79 ms",
23-
"StdDev": "7.29 ms",
24-
"Median": "688.0 ms"
21+
"Mean": "658.8 ms",
22+
"Error": "5.97 ms",
23+
"StdDev": "4.99 ms",
24+
"Median": "660.6 ms"
2525
},
2626
{
2727
"Method": "MSTest",
2828
"Version": "4.0.2",
29-
"Mean": "644.4 ms",
30-
"Error": "10.64 ms",
31-
"StdDev": "9.95 ms",
32-
"Median": "643.1 ms"
29+
"Mean": "631.5 ms",
30+
"Error": "9.34 ms",
31+
"StdDev": "8.74 ms",
32+
"Median": "631.2 ms"
3333
},
3434
{
3535
"Method": "xUnit3",
36-
"Version": "3.2.0",
37-
"Mean": "734.3 ms",
38-
"Error": "11.92 ms",
39-
"StdDev": "10.57 ms",
40-
"Median": "729.7 ms"
36+
"Version": "3.2.1",
37+
"Mean": "716.2 ms",
38+
"Error": "7.08 ms",
39+
"StdDev": "6.28 ms",
40+
"Median": "715.4 ms"
4141
},
4242
{
4343
"Method": "TUnit_AOT",
4444
"Version": "1.2.11",
45-
"Mean": "124.6 ms",
46-
"Error": "0.53 ms",
47-
"StdDev": "0.44 ms",
48-
"Median": "124.6 ms"
45+
"Mean": "124.1 ms",
46+
"Error": "0.23 ms",
47+
"StdDev": "0.20 ms",
48+
"Median": "124.2 ms"
4949
}
5050
]
5151
}

0 commit comments

Comments
 (0)