[Perf] Windows/arm64: 12 Improvements on 3/17/2023 1:20:12 AM #14422
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 5bdc36e1d956fc39cd768b6cf59ac4b4bf5f56a5 |
| Compare | 34c0472fe5a237f85c39bfc45e586ac0fd6e2f11 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Count - Duration of single invocation | 2.47 ms | 2.35 ms | 0.95 | 0.01 | False | |||||
| Count - Duration of single invocation | 3.18 ms | 2.97 ms | 0.93 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Lu}", Options: None)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3471375166889183 < 2.3479669132847345.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/21/2023 1:51:00 AM, 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 31.367318300664138 (T) = (0 -2341864.8397515165) / Math.Sqrt((5767917368.292317 / (299)) + (6283669.226939396 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.056007872554886734 = (2480809.714048892 - 2341864.8397515165) / 2480809.714048892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.972581972789116 < 3.008762417657764.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/21/2023 1:51:00 AM, 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 28.659156847214152 (T) = (0 -2982788.514956502) / Math.Sqrt((6023706254.650907 / (299)) + (263020736.268447 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.05253033771170585 = (3148162.557260757 - 2982788.514956502) / 3148162.557260757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 5bdc36e1d956fc39cd768b6cf59ac4b4bf5f56a5 |
| Compare | 34c0472fe5a237f85c39bfc45e586ac0fd6e2f11 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_VectorConvert
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Narrow_uint - Duration of single invocation | 803.09 ns | 644.97 ns | 0.80 | 0.00 | False | |||||
| Narrow_double - Duration of single invocation | 804.08 ns | 712.24 ns | 0.89 | 0.01 | False | |||||
| Narrow_int - Duration of single invocation | 803.35 ns | 644.02 ns | 0.80 | 0.00 | False | |||||
| Narrow_long - Duration of single invocation | 803.40 ns | 644.65 ns | 0.80 | 0.00 | False | |||||
| Narrow_ulong - Duration of single invocation | 802.92 ns | 644.47 ns | 0.80 | 0.00 | False | |||||
| Narrow_ushort - Duration of single invocation | 803.13 ns | 644.27 ns | 0.80 | 0.00 | False | |||||
| Narrow_short - Duration of single invocation | 802.73 ns | 644.37 ns | 0.80 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'Payloads
Histogram
System.Numerics.Tests.Perf_VectorConvert.Narrow_uint
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 644.9749902444217 < 763.1553097233999.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 1444.287669594802 (T) = (0 -644.1617350847076) / Math.Sqrt((1.6178868154254362 / (299)) + (0.13911781293760714 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.19957625769833165 = (804.7758968673023 - 644.1617350847076) / 804.7758968673023 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_double
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 712.2382937791907 < 762.9473346402063.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 28.13203875201368 (T) = (0 -708.906108160503) / Math.Sqrt((2.1340223891896155 / (299)) + (232.68481147500776 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.11925216304567482 = (804.8911145918219 - 708.906108160503) / 804.8911145918219 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_int
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 644.024523780009 < 763.1286444168437.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 1446.9556021920494 (T) = (0 -643.9793272358414) / Math.Sqrt((1.5736072004645378 / (299)) + (0.1417410980230836 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.19980700738595186 = (804.780013296677 - 643.9793272358414) / 804.780013296677 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_long
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 644.6502185926237 < 762.9942423016076.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 1535.3880547911679 (T) = (0 -644.0911586633802) / Math.Sqrt((1.522392649240298 / (299)) + (0.11725720513991386 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.19967841610632886 = (804.7904387756117 - 644.0911586633802) / 804.7904387756117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_ulong
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 644.4665845518118 < 763.1640343038177.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 592.2432626478079 (T) = (0 -644.4340423181459) / Math.Sqrt((1.7684568028923553 / (299)) + (1.348610200310411 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.19928885992313491 = (804.8271218710443 - 644.4340423181459) / 804.8271218710443 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_ushort
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 644.2739219259013 < 762.9916236235478.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 1909.0654976350042 (T) = (0 -644.1268008301515) / Math.Sqrt((1.5748835511841293 / (299)) + (0.036293803042634354 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.19962562150648028 = (804.7818847506583 - 644.1268008301515) / 804.7818847506583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_short
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 644.3693711982521 < 763.1084805767628.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 1373.3549468819247 (T) = (0 -644.0387562297999) / Math.Sqrt((1.4257276337359595 / (299)) + (0.1785969850962931 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.19972826645848454 = (804.7750898056545 - 644.0387562297999) / 804.7750898056545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 5bdc36e1d956fc39cd768b6cf59ac4b4bf5f56a5 |
| Compare | 34c0472fe5a237f85c39bfc45e586ac0fd6e2f11 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Count - Duration of single invocation | 36.51 ms | 31.60 ms | 0.87 | 0.01 | False | |||||
| Count - Duration of single invocation | 36.48 ms | 31.65 ms | 0.87 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: None)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.60356979166667 < 34.69825966145833.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/21/2023 1:51:00 AM, 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 32.41590071846051 (T) = (0 -31622125.923992675) / Math.Sqrt((6535818161189.599 / (299)) + (576191148.4931653 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.13168721637108222 = (36417897.467586644 - 31622125.923992675) / 36417897.467586644 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\\p{Sm}", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.65235673076923 < 34.674318035714286.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/21/2023 1:51:00 AM, 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 32.72317101197216 (T) = (0 -31634871.17811355) / Math.Sqrt((6512048221618.324 / (299)) + (547237155.5069574 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.132510259163102 = (36467141.556733884 - 31634871.17811355) / 36467141.556733884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 5bdc36e1d956fc39cd768b6cf59ac4b4bf5f56a5 |
| Compare | 34c0472fe5a237f85c39bfc45e586ac0fd6e2f11 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| MultipleSerial - Duration of single invocation | 22.98 μs | 21.28 μs | 0.93 | 0.00 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Object>*'Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.284536857142857 < 21.847220268333334.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 55.681871885112095 (T) = (0 -21304.715574358976) / Math.Sqrt((12760.080179095674 / (299)) + (16048.32747964379 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.07061331284851302 = (22923.413761882708 - 21304.715574358976) / 22923.413761882708 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository