Skip to content

Commit 48c115f

Browse files
thomhurstthomhurst
authored
chore: update benchmark results (#3890)
1 parent 4f7972d commit 48c115f

18 files changed

+338
-334
lines changed

‎docs/docs/benchmarks/AsyncTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 2
77
# AsyncTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-20** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-23** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 556.3 ms | 556.3 ms | 4.46 ms |
20-
| NUnit | 4.4.0 | 656.0 ms | 652.5 ms | 9.79 ms |
21-
| MSTest | 4.0.2 | 626.0 ms | 625.3 ms | 8.22 ms |
22-
| xUnit3 | 3.2.0 | 701.3 ms | 698.3 ms | 7.16 ms |
23-
| **TUnit (AOT)** | 1.2.11 | 124.3 ms | 124.3 ms | 0.31 ms |
19+
| **TUnit** | 1.2.11 | 565.8 ms | 565.5 ms | 2.61 ms |
20+
| NUnit | 4.4.0 | 663.9 ms | 663.3 ms | 8.44 ms |
21+
| MSTest | 4.0.2 | 627.6 ms | 628.8 ms | 4.97 ms |
22+
| xUnit3 | 3.2.0 | 715.6 ms | 714.6 ms | 8.31 ms |
23+
| **TUnit (AOT)** | 1.2.11 | 124.2 ms | 124.3 ms | 0.35 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
5858
xychart-beta
5959
title "AsyncTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 842
62-
bar [556.3, 656, 626, 701.3, 124.3]
61+
y-axis "Time (ms)" 0 --> 859
62+
bar [565.8, 663.9, 627.6, 715.6, 124.2]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-20T00:28:21.079Z*
75+
*Last generated: 2025-11-23T00:33:08.378Z*

‎docs/docs/benchmarks/BuildTime.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 8
77
# Build Performance Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-20** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-23** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -18,10 +18,10 @@ Compilation time comparison across frameworks:
1818

1919
| Framework | Version | Mean | Median | StdDev |
2020
|-----------|---------|------|--------|--------|
21-
| **TUnit** | 1.2.11 | 2.096 s | 2.087 s | 0.0247 s |
22-
| Build_NUnit | 4.4.0 | 1.707 s | 1.713 s | 0.0257 s |
23-
| Build_MSTest | 4.0.2 | 1.792 s | 1.795 s | 0.0227 s |
24-
| Build_xUnit3 | 3.2.0 | 1.660 s | 1.664 s | 0.0171 s |
21+
| **TUnit** | 1.2.11 | 2.057 s | 2.052 s | 0.0363 s |
22+
| Build_NUnit | 4.4.0 | 1.639 s | 1.641 s | 0.0244 s |
23+
| Build_MSTest | 4.0.2 | 1.720 s | 1.722 s | 0.0156 s |
24+
| Build_xUnit3 | 3.2.0 | 1.620 s | 1.620 s | 0.0195 s |
2525

2626
## 📈 Visual Comparison
2727

@@ -60,7 +60,7 @@ xychart-beta
6060
title "Build Time Comparison"
6161
x-axis ["Build_TUnit", "Build_NUnit", "Build_MSTest", "Build_xUnit3"]
6262
y-axis "Time (s)" 0 --> 3
63-
bar [2.096, 1.707, 1.792, 1.66]
63+
bar [2.057, 1.639, 1.72, 1.62]
6464
```
6565

6666
---
@@ -69,4 +69,4 @@ xychart-beta
6969
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7070
:::
7171

72-
*Last generated: 2025-11-20T00:28:21.081Z*
72+
*Last generated: 2025-11-23T00:33:08.380Z*

‎docs/docs/benchmarks/DataDrivenTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 3
77
# DataDrivenTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-20** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-23** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 504.85 ms | 504.93 ms | 4.652 ms |
20-
| NUnit | 4.4.0 | 602.92 ms | 602.92 ms | 8.093 ms |
21-
| MSTest | 4.0.2 | 617.40 ms | 616.45 ms | 5.510 ms |
22-
| xUnit3 | 3.2.0 | 614.39 ms | 611.15 ms | 11.735 ms |
23-
| **TUnit (AOT)** | 1.2.11 | 25.60 ms | 25.69 ms | 0.624 ms |
19+
| **TUnit** | 1.2.11 | 493.89 ms | 493.75 ms | 3.597 ms |
20+
| NUnit | 4.4.0 | 526.36 ms | 525.02 ms | 7.358 ms |
21+
| MSTest | 4.0.2 | 493.66 ms | 494.45 ms | 7.024 ms |
22+
| xUnit3 | 3.2.0 | 572.74 ms | 575.02 ms | 7.551 ms |
23+
| **TUnit (AOT)** | 1.2.11 | 23.74 ms | 23.76 ms | 0.112 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
5858
xychart-beta
5959
title "DataDrivenTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 741
62-
bar [504.85, 602.92, 617.4, 614.39, 25.6]
61+
y-axis "Time (ms)" 0 --> 688
62+
bar [493.89, 526.36, 493.66, 572.74, 23.74]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-20T00:28:21.080Z*
75+
*Last generated: 2025-11-23T00:33:08.378Z*

‎docs/docs/benchmarks/MassiveParallelTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 4
77
# MassiveParallelTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-20** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-23** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 601.9 ms | 601.6 ms | 7.25 ms |
20-
| NUnit | 4.4.0 | 1,206.3 ms | 1,203.0 ms | 13.87 ms |
21-
| MSTest | 4.0.2 | 2,983.3 ms | 2,984.1 ms | 5.48 ms |
22-
| xUnit3 | 3.2.0 | 3,069.8 ms | 3,070.7 ms | 4.67 ms |
23-
| **TUnit (AOT)** | 1.2.11 | 131.1 ms | 131.1 ms | 0.45 ms |
19+
| **TUnit** | 1.2.11 | 603.5 ms | 603.0 ms | 3.89 ms |
20+
| NUnit | 4.4.0 | 1,213.0 ms | 1,213.1 ms | 7.41 ms |
21+
| MSTest | 4.0.2 | 3,000.0 ms | 2,998.4 ms | 9.38 ms |
22+
| xUnit3 | 3.2.0 | 3,089.8 ms | 3,087.9 ms | 8.18 ms |
23+
| **TUnit (AOT)** | 1.2.11 | 131.4 ms | 131.3 ms | 0.36 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
5858
xychart-beta
5959
title "MassiveParallelTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 3684
62-
bar [601.9, 1206.3, 2983.3, 3069.8, 131.1]
61+
y-axis "Time (ms)" 0 --> 3708
62+
bar [603.5, 1213, 3000, 3089.8, 131.4]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-20T00:28:21.080Z*
75+
*Last generated: 2025-11-23T00:33:08.379Z*

‎docs/docs/benchmarks/MatrixTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 5
77
# MatrixTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-20** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-23** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 561.09 ms | 560.90 ms | 2.690 ms |
20-
| NUnit | 4.4.0 | 1,568.63 ms | 1,568.52 ms | 6.683 ms |
21-
| MSTest | 4.0.2 | 1,527.48 ms | 1,528.24 ms | 5.902 ms |
22-
| xUnit3 | 3.2.0 | 1,607.99 ms | 1,607.91 ms | 4.911 ms |
23-
| **TUnit (AOT)** | 1.2.11 | 78.98 ms | 79.04 ms | 0.171 ms |
19+
| **TUnit** | 1.2.11 | 559.21 ms | 559.52 ms | 4.212 ms |
20+
| NUnit | 4.4.0 | 1,586.70 ms | 1,588.09 ms | 15.075 ms |
21+
| MSTest | 4.0.2 | 1,551.25 ms | 1,551.96 ms | 13.470 ms |
22+
| xUnit3 | 3.2.0 | 1,628.81 ms | 1,616.75 ms | 19.456 ms |
23+
| **TUnit (AOT)** | 1.2.11 | 78.72 ms | 78.56 ms | 0.429 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
5858
xychart-beta
5959
title "MatrixTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1930
62-
bar [561.09, 1568.63, 1527.48, 1607.99, 78.98]
61+
y-axis "Time (ms)" 0 --> 1955
62+
bar [559.21, 1586.7, 1551.25, 1628.81, 78.72]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-20T00:28:21.080Z*
75+
*Last generated: 2025-11-23T00:33:08.379Z*

‎docs/docs/benchmarks/ScaleTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 6
77
# ScaleTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-20** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-23** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 522.86 ms | 521.98 ms | 4.879 ms |
20-
| NUnit | 4.4.0 | 614.14 ms | 613.81 ms | 8.401 ms |
21-
| MSTest | 4.0.2 | 620.73 ms | 619.89 ms | 10.717 ms |
22-
| xUnit3 | 3.2.0 | 613.65 ms | 611.98 ms | 7.848 ms |
23-
| **TUnit (AOT)** | 1.2.11 | 46.30 ms | 46.04 ms | 3.593 ms |
19+
| **TUnit** | 1.2.11 | 519.71 ms | 517.98 ms | 5.112 ms |
20+
| NUnit | 4.4.0 | 565.90 ms | 566.29 ms | 7.194 ms |
21+
| MSTest | 4.0.2 | 497.34 ms | 495.08 ms | 11.872 ms |
22+
| xUnit3 | 3.2.0 | 576.49 ms | 578.68 ms | 9.104 ms |
23+
| **TUnit (AOT)** | 1.2.11 | 46.08 ms | 46.38 ms | 3.790 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
5858
xychart-beta
5959
title "ScaleTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 745
62-
bar [522.86, 614.14, 620.73, 613.65, 46.3]
61+
y-axis "Time (ms)" 0 --> 692
62+
bar [519.71, 565.9, 497.34, 576.49, 46.08]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-20T00:28:21.081Z*
75+
*Last generated: 2025-11-23T00:33:08.379Z*

‎docs/docs/benchmarks/SetupTeardownTests.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 7
77
# SetupTeardownTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-20** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-23** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,10 +16,10 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.11 | 586.0 ms | 586.0 ms | 7.14 ms |
20-
| NUnit | 4.4.0 | 1,160.8 ms | 1,161.1 ms | 8.37 ms |
21-
| MSTest | 4.0.2 | 1,133.4 ms | 1,132.9 ms | 11.79 ms |
22-
| xUnit3 | 3.2.0 | 1,220.2 ms | 1,221.4 ms | 4.97 ms |
19+
| **TUnit** | 1.2.11 | 584.4 ms | 584.6 ms | 6.90 ms |
20+
| NUnit | 4.4.0 | 1,172.5 ms | 1,170.6 ms | 7.24 ms |
21+
| MSTest | 4.0.2 | 1,145.4 ms | 1,144.0 ms | 13.26 ms |
22+
| xUnit3 | 3.2.0 | 1,245.2 ms | 1,247.0 ms | 17.89 ms |
2323
| **TUnit (AOT)** | 1.2.11 | NA | NA | NA |
2424

2525
## 📈 Visual Comparison
@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-20** from the latest CI
5858
xychart-beta
5959
title "SetupTeardownTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1465
62-
bar [586, 1160.8, 1133.4, 1220.2, 0]
61+
y-axis "Time (ms)" 0 --> 1495
62+
bar [584.4, 1172.5, 1145.4, 1245.2, 0]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-20T00:28:21.081Z*
75+
*Last generated: 2025-11-23T00:33:08.380Z*

‎docs/docs/benchmarks/index.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 1
77
# Performance Benchmarks
88

99
:::info Last Updated
10-
These benchmarks were automatically generated on **2025-11-20** from the latest CI run.
10+
These benchmarks were automatically generated on **2025-11-23** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -80,4 +80,4 @@ These benchmarks run automatically daily via [GitHub Actions](https://github.com
8080
Each benchmark runs multiple iterations with statistical analysis to ensure accuracy. Results may vary based on hardware and test characteristics.
8181
:::
8282

83-
*Last generated: 2025-11-20T00:28:21.082Z*
83+
*Last generated: 2025-11-23T00:33:08.380Z*

‎docs/static/benchmarks/AsyncTests.json‎

Lines changed: 20 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"timestamp": "2025-11-20T00:28:21.079Z",
2+
"timestamp": "2025-11-23T00:33:08.378Z",
33
"category": "AsyncTests",
44
"environment": {
55
"benchmarkDotNetVersion": "BenchmarkDotNet v0.15.7, Linux Ubuntu 24.04.3 LTS (Noble Numbat)",
@@ -10,41 +10,41 @@
1010
{
1111
"Method": "TUnit",
1212
"Version": "1.2.11",
13-
"Mean": "556.3 ms",
14-
"Error": "5.03 ms",
15-
"StdDev": "4.46 ms",
16-
"Median": "556.3 ms"
13+
"Mean": "565.8 ms",
14+
"Error": "2.95 ms",
15+
"StdDev": "2.61 ms",
16+
"Median": "565.5 ms"
1717
},
1818
{
1919
"Method": "NUnit",
2020
"Version": "4.4.0",
21-
"Mean": "656.0 ms",
22-
"Error": "11.05 ms",
23-
"StdDev": "9.79 ms",
24-
"Median": "652.5 ms"
21+
"Mean": "663.9 ms",
22+
"Error": "9.52 ms",
23+
"StdDev": "8.44 ms",
24+
"Median": "663.3 ms"
2525
},
2626
{
2727
"Method": "MSTest",
2828
"Version": "4.0.2",
29-
"Mean": "626.0 ms",
30-
"Error": "8.79 ms",
31-
"StdDev": "8.22 ms",
32-
"Median": "625.3 ms"
29+
"Mean": "627.6 ms",
30+
"Error": "5.61 ms",
31+
"StdDev": "4.97 ms",
32+
"Median": "628.8 ms"
3333
},
3434
{
3535
"Method": "xUnit3",
3636
"Version": "3.2.0",
37-
"Mean": "701.3 ms",
38-
"Error": "8.08 ms",
39-
"StdDev": "7.16 ms",
40-
"Median": "698.3 ms"
37+
"Mean": "715.6 ms",
38+
"Error": "9.37 ms",
39+
"StdDev": "8.31 ms",
40+
"Median": "714.6 ms"
4141
},
4242
{
4343
"Method": "TUnit_AOT",
4444
"Version": "1.2.11",
45-
"Mean": "124.3 ms",
46-
"Error": "0.40 ms",
47-
"StdDev": "0.31 ms",
45+
"Mean": "124.2 ms",
46+
"Error": "0.38 ms",
47+
"StdDev": "0.35 ms",
4848
"Median": "124.3 ms"
4949
}
5050
]

0 commit comments

Comments
 (0)