[Perf] Windows/arm64: 3 Improvements on 1/27/2023 2:26:56 PM #12368
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | f60d188f16f3843eddc0116ee47a7c0f85a08cb2 |
| Compare | ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7 |
| Diff | Diff |
Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| SingleParallel - Duration of single invocation | 1.25 μs | 1.08 μs | 0.86 | 0.05 | False | |||||
| ProducerConsumer - Duration of single invocation | 6.03 μs | 1.25 μs | 0.21 | 0.85 | False |
_1.png)
_2.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Object>*'Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: False)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0783099 < 1.176594839423077.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsImprovementStdDev: Marked as improvement because 11.048452513333782 (T) = (0 -1113.190206424314) / Math.Sqrt((519.9093590989374 / (299)) + (2378.97296941525 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.10301788362777992 = (1241.039465676899 - 1113.190206424314) / 1241.039465676899 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.ProducerConsumer(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2546614999999999 < 5.618360859375001.
IsChangePoint: Marked as a change because one of 1/26/2023 1:49:07 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsImprovementStdDev: Marked as improvement because 21.028594029182056 (T) = (0 -2109.8307592105257) / Math.Sqrt((2359716.570272231 / (299)) + (311842.77415460895 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.6004876843295637 = (5281.0155693697225 - 2109.8307592105257) / 5281.0155693697225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | f60d188f16f3843eddc0116ee47a7c0f85a08cb2 |
| Compare | ddfaf2750e8dcf127a8baf0a3e5a2bd4835f79c7 |
| Diff | Diff |
Improvements in Benchmark.GetChildKeysTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| AddChainedConfigurationEmpty - Duration of single invocation | 43.48 ms | 36.15 ms | 0.83 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'Payloads
Histogram
Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.151356666666665 < 41.31145737301587.
IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 1/13/2023 9:23:19 AM, 1/27/2023 3:43:27 AM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsImprovementStdDev: Marked as improvement because 111.15397692849194 (T) = (0 -36042054.27248677) / Math.Sqrt((2948099573548.437 / (299)) + (2074158675.319669 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.23548366108123034 = (47143602.35055259 - 36042054.27248677) / 47143602.35055259 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository