Skip to content

[Perf] Changes at 6/7/2022 11:31:29 PM #6157

Closed
@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 0712aebead42b6605fda32687b341de08f530a3c
Compare 86efc101ecd68734b957d5aee299cd1da1dd6b8d
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ProducerConsumer - Duration of single invocation 1.85 μs 1.65 μs 0.89 0.52 False
SingleSerial - Duration of single invocation 1.03 μs 549.81 ns 0.54 0.59 False
SingleParallel - Duration of single invocation 313.50 μs 54.65 μs 0.17 0.69 False
SingleParallel - Duration of single invocation 419.06 μs 51.74 μs 0.12 0.63 False
SingleParallel - Duration of single invocation 449.31 μs 12.36 μs 0.03 0.58 False
SingleParallel - Duration of single invocation 495.00 μs 13.47 μs 0.03 0.55 False
SingleSerial - Duration of single invocation 1.09 μs 635.51 ns 0.58 0.56 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.ProducerConsumer(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.652064778 < 1.8116173188124998.
IsChangePoint: Marked as a change because one of 6/7/2022 6:41:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 11.89192312858722 (T) = (0 -1755.775179811182) / Math.Sqrt((11358.358712185533 / (169)) + (11109.886235783137 / (33))) is greater than 1.9718962236336928 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (169) + (33) - 2, .975) and 0.11980724648445512 = (1994.7621390865877 - 1755.775179811182) / 1994.7621390865877 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 549.8101633333334 < 907.5643154562499.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 45.139995323583555 (T) = (0 -569.7963587494845) / Math.Sqrt((7870.498370836751 / (168)) + (608.550537245055 / (34))) is greater than 1.9718962236336928 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (34) - 2, .975) and 0.38929690297998304 = (933.0169791668968 - 569.7963587494845) / 933.0169791668968 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.64642342857143 < 336.26342110642497.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 114.40359978693023 (T) = (0 -55403.55599156189) / Math.Sqrt((1274100484.8864343 / (168)) + (4153275.0234809434 / (33))) is greater than 1.9719565442516196 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (33) - 2, .975) and 0.8514898316883777 = (373062.3742564707 - 55403.55599156189) / 373062.3742564707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.737767077222216 < 406.84454215375.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 121.44187027481992 (T) = (0 -54570.09925799015) / Math.Sqrt((1826320732.7621918 / (168)) + (8068136.089025797 / (34))) is greater than 1.9718962236336928 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (34) - 2, .975) and 0.8811948349682582 = (459324.29994445416 - 54570.09925799015) / 459324.29994445416 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.358211316666665 < 396.70837832835707.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 160.11517296564043 (T) = (0 -11836.261662203258) / Math.Sqrt((1119166517.551617 / (168)) + (120177.09824139024 / (33))) is greater than 1.9719565442516196 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (33) - 2, .975) and 0.9721637889280844 = (425210.9466918448 - 11836.261662203258) / 425210.9466918448 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.473012203000001 < 474.60704211637494.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 163.53632870551303 (T) = (0 -13420.997851789958) / Math.Sqrt((1397693376.3764386 / (168)) + (11586.774988406878 / (34))) is greater than 1.9718962236336928 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (34) - 2, .975) and 0.9723352725304223 = (485130.3113883446 - 13420.997851789958) / 485130.3113883446 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 635.505301 < 1.0173012146218752.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 45.8136899985523 (T) = (0 -642.6101687082655) / Math.Sqrt((4969.3061608432745 / (168)) + (755.0170937876475 / (34))) is greater than 1.9718962236336928 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (168) + (34) - 2, .975) and 0.3390794100528466 = (972.2955805623305 - 642.6101687082655) / 972.2955805623305 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 0712aebead42b6605fda32687b341de08f530a3c
Compare 86efc101ecd68734b957d5aee299cd1da1dd6b8d
Diff Diff

Improvements in System.Threading.Tasks.Tests.Perf_AsyncMethods

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleYieldMethodInvocation - Duration of single invocation 1.30 μs 1.08 μs 0.83 0.43 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tasks.Tests.Perf_AsyncMethods*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tasks.Tests.Perf_AsyncMethods.SingleYieldMethodInvocation


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0758320491666669 < 1.2410897045688656.
IsChangePoint: Marked as a change because one of 6/7/2022 6:41:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 22.253228961827986 (T) = (0 -1029.7138382237756) / Math.Sqrt((3342.4903971025633 / (113)) + (2199.2049174558865 / (33))) is greater than 1.9765750658302461 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (113) + (33) - 2, .975) and 0.17491022057140032 = (1248.0021736990664 - 1029.7138382237756) / 1248.0021736990664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions