Skip to content

[Perf] Changes at 6/29/2022 1:11:29 AM #6568

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 845.41 μs 374.43 μs 0.44 0.06 False
ImmutableSortedSet - Duration of single invocation 956.19 μs 610.80 μs 0.64 0.02 False
SortedSet - Duration of single invocation 793.25 μs 429.70 μs 0.54 0.02 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.SortedList(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 374.43095089285714 < 810.1842635044642.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 227.08158743857135 (T) = (0 -376120.8686908124) / Math.Sqrt((1280438237.2741232 / (299)) + (8580211.19410225 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.5619362240605862 = (858598.4264145858 - 376120.8686908124) / 858598.4264145858 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 610.7988786858974 < 913.6935857438289.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 135.71477501611545 (T) = (0 -603033.4803686213) / Math.Sqrt((1593485354.912962 / (299)) + (67710030.94527641 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.3758488837878269 = (966165.8286029996 - 603033.4803686213) / 966165.8286029996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;Int32&gt;.SortedSet(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 429.69618665540537 < 750.4459532820513.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 165.9864520668582 (T) = (0 -433712.57497148484) / Math.Sqrt((1061502853.430323 / (299)) + (41811206.50282233 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.4529386046720944 = (792804.2056623643 - 433712.57497148484) / 792804.2056623643 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array - Duration of single invocation 1.12 ms 666.64 μs 0.59 0.01 False
List - Duration of single invocation 1.12 ms 655.80 μs 0.59 0.02 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<BigStruct>.Array(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 666.6414571857144 < 1.061881343935357.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 274.7782848669194 (T) = (0 -659457.7347074468) / Math.Sqrt((627177062.3770927 / (299)) + (27283344.202417687 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.41230670562078797 = (1122112.0625581418 - 659457.7347074468) / 1122112.0625581418 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;BigStruct&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 655.7970333666666 < 1.0672148167317617.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 232.7393370789237 (T) = (0 -661195.0490530015) / Math.Sqrt((637670761.042802 / (299)) + (66427849.71793315 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.4109447161896637 = (1122466.8842218427 - 661195.0490530015) / 1122466.8842218427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 654.66 μs 384.02 μs 0.59 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<Int32, Int32>.SortedList(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 384.0223575203252 < 621.7033102063301.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 175.3957116564391 (T) = (0 -387950.97146240016) / Math.Sqrt((467343209.99102044 / (299)) + (30141323.855679393 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.4107712965700513 = (658404.7403055989 - 387950.97146240016) / 658404.7403055989 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 943.30 μs 673.04 μs 0.71 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<Int32>.SortedList(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 673.0416666666666 < 892.3594314403884.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 29.145301352150714 (T) = (0 -680072.7497735185) / Math.Sqrt((24361500984.500095 / (299)) + (65468240.695813194 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.2811011689924607 = (945992.2877053425 - 680072.7497735185) / 945992.2877053425 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array - Duration of single invocation 685.61 μs 322.94 μs 0.47 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.Array(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 322.9357142428572 < 641.8007697805493.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 182.0415357038296 (T) = (0 -324233.5571360866) / Math.Sqrt((832855727.0093045 / (299)) + (38559100.3900093 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.5234554881781033 = (680384.6211479723 - 324233.5571360866) / 680384.6211479723 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.Sort<IntClass>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array - Duration of single invocation 398.94 μs 373.23 μs 0.94 0.04 False
List - Duration of single invocation 399.47 μs 369.82 μs 0.93 0.02 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;IntClass&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<IntClass>.Array(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 373.2328266800001 < 379.3071966476667.
IsChangePoint: Marked as a change because one of 5/25/2022 8:13:38 AM, 6/5/2022 6:02:14 PM, 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 20.72484217319039 (T) = (0 -368863.2388255292) / Math.Sqrt((732744999.9924022 / (299)) + (24780962.653483585 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.09028902754131651 = (405473.0018575014 - 368863.2388255292) / 405473.0018575014 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;IntClass&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 369.82161425714287 < 378.4546350633333.
IsChangePoint: Marked as a change because one of 5/25/2022 8:13:38 AM, 6/5/2022 6:02:14 PM, 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 16.381676553198403 (T) = (0 -371492.2990084542) / Math.Sqrt((682572693.6911192 / (299)) + (65103562.20679423 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.08236340466776616 = (404835.96763482824 - 371492.2990084542) / 404835.96763482824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 997.50 μs 588.41 μs 0.59 0.02 False
Array - Duration of single invocation 984.80 μs 582.39 μs 0.59 0.02 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<IntStruct>.List(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 588.4084614615384 < 941.9803140019762.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 243.59824595015624 (T) = (0 -589184.3876718277) / Math.Sqrt((635894048.9336479 / (299)) + (18484522.24153326 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.4012096354160041 = (983957.6962484326 - 589184.3876718277) / 983957.6962484326 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;IntStruct&gt;.Array(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 582.3940666666666 < 935.3735783396668.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 191.66902646780488 (T) = (0 -591887.1772980021) / Math.Sqrt((529495384.56694376 / (299)) + (91342581.68136826 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.40003703359212023 = (986539.5206670349 - 591887.1772980021) / 986539.5206670349 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 660.68 μs 394.71 μs 0.60 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.SortedList(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 394.70733125000004 < 627.3106090908788.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 194.8357346817769 (T) = (0 -389700.61163508013) / Math.Sqrt((532925110.9941157 / (299)) + (7362136.767769038 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.41305834638678834 = (663951.1938470952 - 389700.61163508013) / 663951.1938470952 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 947.64 μs 685.11 μs 0.72 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.SortedList(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 685.114518439441 < 900.9852560308259.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 129.93865396328425 (T) = (0 -687846.9860471555) / Math.Sqrt((1050122362.1761982 / (299)) + (17424229.97428867 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.27378931563510095 = (947172.7707348533 - 687846.9860471555) / 947172.7707348533 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 720.75 μs 435.91 μs 0.60 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.SortedList(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 435.9088540426587 < 686.1850001799243.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 139.17758297777064 (T) = (0 -425487.9812795426) / Math.Sqrt((698721821.3739876 / (299)) + (92196352.95938697 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.41819148263638456 = (731319.615614399 - 425487.9812795426) / 731319.615614399 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 1.81 ms 1.33 ms 0.73 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<Int32>.SortedList(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3293322930555556 < 1.721497779079861.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 118.77278547398564 (T) = (0 -1331814.0824537796) / Math.Sqrt((3766170384.820892 / (299)) + (141576701.52218297 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.2654629142709721 = (1813133.888443433 - 1331814.0824537796) / 1813133.888443433 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3fb9511bf78a5de5cb73bad8fd50a69c691d61cd
Compare 8b71701737f3edb7c9c8acccbf6656fdaa955ffa
Diff Diff

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 751.41 μs 430.99 μs 0.57 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 430.9919870459402 < 692.1729748965877.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 7/5/2022 1:19:49 PM falls between 6/26/2022 6:11:06 PM and 7/5/2022 1:19:49 PM.
IsImprovementStdDev: Marked as improvement because 177.05874027408177 (T) = (0 -428380.14577216416) / Math.Sqrt((609444895.4533415 / (299)) + (35244060.00736095 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.41684082315194076 = (734585.277535258 - 428380.14577216416) / 734585.277535258 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions