Closed
Description
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | 0712aebead42b6605fda32687b341de08f530a3c |
Compare | 86efc101ecd68734b957d5aee299cd1da1dd6b8d |
Diff | Diff |
Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ProducerConsumer - Duration of single invocation | 1.85 μs | 1.65 μs | 0.89 | 0.52 | False | |||||
SingleSerial - Duration of single invocation | 1.03 μs | 549.81 ns | 0.54 | 0.59 | False | |||||
SingleParallel - Duration of single invocation | 313.50 μs | 54.65 μs | 0.17 | 0.69 | False | |||||
SingleParallel - Duration of single invocation | 419.06 μs | 51.74 μs | 0.12 | 0.63 | False | |||||
SingleParallel - Duration of single invocation | 449.31 μs | 12.36 μs | 0.03 | 0.58 | False | |||||
SingleParallel - Duration of single invocation | 495.00 μs | 13.47 μs | 0.03 | 0.55 | False | |||||
SingleSerial - Duration of single invocation | 1.09 μs | 635.51 ns | 0.58 | 0.56 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Object>*'
Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Object>.ProducerConsumer(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.652064778 < 1.8116173188124998.
IsChangePoint: Marked as a change because one of 6/7/2022 6:41:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 11.89192312858722 (T) = (0 -1755.775179811182) / Math.Sqrt((11358.358712185533 / (169)) + (11109.886235783137 / (33))) is greater than 1.9718962236336928 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (169) + (33) - 2, .975) and 0.11980724648445512 = (1994.7621390865877 - 1755.775179811182) / 1994.7621390865877 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 549.8101633333334 < 907.5643154562499.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 45.139995323583555 (T) = (0 -569.7963587494845) / Math.Sqrt((7870.498370836751 / (168)) + (608.550537245055 / (34))) is greater than 1.9718962236336928 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (34) - 2, .975) and 0.38929690297998304 = (933.0169791668968 - 569.7963587494845) / 933.0169791668968 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.64642342857143 < 336.26342110642497.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 114.40359978693023 (T) = (0 -55403.55599156189) / Math.Sqrt((1274100484.8864343 / (168)) + (4153275.0234809434 / (33))) is greater than 1.9719565442516196 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (33) - 2, .975) and 0.8514898316883777 = (373062.3742564707 - 55403.55599156189) / 373062.3742564707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.737767077222216 < 406.84454215375.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 121.44187027481992 (T) = (0 -54570.09925799015) / Math.Sqrt((1826320732.7621918 / (168)) + (8068136.089025797 / (34))) is greater than 1.9718962236336928 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (34) - 2, .975) and 0.8811948349682582 = (459324.29994445416 - 54570.09925799015) / 459324.29994445416 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.358211316666665 < 396.70837832835707.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 160.11517296564043 (T) = (0 -11836.261662203258) / Math.Sqrt((1119166517.551617 / (168)) + (120177.09824139024 / (33))) is greater than 1.9719565442516196 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (33) - 2, .975) and 0.9721637889280844 = (425210.9466918448 - 11836.261662203258) / 425210.9466918448 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.473012203000001 < 474.60704211637494.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 163.53632870551303 (T) = (0 -13420.997851789958) / Math.Sqrt((1397693376.3764386 / (168)) + (11586.774988406878 / (34))) is greater than 1.9718962236336928 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (34) - 2, .975) and 0.9723352725304223 = (485130.3113883446 - 13420.997851789958) / 485130.3113883446 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 635.505301 < 1.0173012146218752.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 45.8136899985523 (T) = (0 -642.6101687082655) / Math.Sqrt((4969.3061608432745 / (168)) + (755.0170937876475 / (34))) is greater than 1.9718962236336928 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (34) - 2, .975) and 0.3390794100528466 = (972.2955805623305 - 642.6101687082655) / 972.2955805623305 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | 0712aebead42b6605fda32687b341de08f530a3c |
Compare | 86efc101ecd68734b957d5aee299cd1da1dd6b8d |
Diff | Diff |
Improvements in System.Threading.Tasks.Tests.Perf_AsyncMethods
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SingleYieldMethodInvocation - Duration of single invocation | 1.30 μs | 1.08 μs | 0.83 | 0.43 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tasks.Tests.Perf_AsyncMethods*'
Payloads
Histogram
System.Threading.Tasks.Tests.Perf_AsyncMethods.SingleYieldMethodInvocation
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0758320491666669 < 1.2410897045688656.
IsChangePoint: Marked as a change because one of 6/7/2022 6:41:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 22.253228961827986 (T) = (0 -1029.7138382237756) / Math.Sqrt((3342.4903971025633 / (113)) + (2199.2049174558865 / (33))) is greater than 1.9765750658302461 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (113) + (33) - 2, .975) and 0.17491022057140032 = (1248.0021736990664 - 1029.7138382237756) / 1248.0021736990664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository