Skip to content

[Perf] Changes at 4/16/2022 10:12:41 PM #4874

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ForeachOverList100Elements - Duration of single invocation 6.15 ms 10.51 ms 1.71 0.13 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.ForeachOverList100Elements


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.508131235897439 > 6.460709094218751.
IsChangePoint: Marked as a change because one of 3/15/2022 11:34:41 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -14.468396769996337 (T) = (0 -10275307.406177158) / Math.Sqrt((334158393904.3621 / (46)) + (619183493689.1334 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.55032418984029 = (6627844.339599507 - 10275307.406177158) / 6627844.339599507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TrimStart - Duration of single invocation 10.28 ns 11.95 ns 1.16 0.18 False
Trim - Duration of single invocation 13.27 ns 14.80 ns 1.11 0.06 False
Trim - Duration of single invocation 10.76 ns 12.05 ns 1.12 0.10 False
Trim - Duration of single invocation 10.55 ns 11.94 ns 1.13 0.14 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.TrimStart(s: " Test")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.950812547755511 > 10.79309906278997.
IsChangePoint: Marked as a change because one of 2/26/2022 3:32:05 AM, 4/2/2022 3:40:16 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -14.889943993101932 (T) = (0 -11.59547902114433) / Math.Sqrt((0.05392207918309299 / (46)) + (0.1117709377822395 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.15835111471837598 = (10.010331818917859 - 11.59547902114433) / 10.010331818917859 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: " Te st  ")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.797898187448453 > 13.984379895623395.
IsChangePoint: Marked as a change because one of 2/25/2022 7:19:51 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -35.03087003076377 (T) = (0 -14.786953896362554) / Math.Sqrt((0.08810576309222704 / (46)) + (0.0034671695045072346 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.12598204985964162 = (13.132495227793205 - 14.786953896362554) / 13.132495227793205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: "Test ")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.053331851479438 > 11.433688825056453.
IsChangePoint: Marked as a change because one of 2/24/2022 9:06:13 PM, 3/30/2022 6:37:35 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -34.05403580405217 (T) = (0 -12.139535921343105) / Math.Sqrt((0.030276203231365307 / (46)) + (0.012064260147921113 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.13316478079228689 = (10.712948484734389 - 12.139535921343105) / 10.712948484734389 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: " Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.938674490618524 > 11.200035366850665.
IsChangePoint: Marked as a change because one of 2/25/2022 5:57:13 AM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -10.472758061236597 (T) = (0 -11.91862347115635) / Math.Sqrt((0.04283604934959887 / (46)) + (0.23237029675301318 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.15008060358461325 = (10.363294045659016 - 11.91862347115635) / 10.363294045659016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.IndexerSetReverse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 1.45 μs 1.79 μs 1.23 0.16 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<String>.IList(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7894518972830449 > 1.5267697155497022.
IsChangePoint: Marked as a change because one of 2/17/2022 3:48:53 PM, 2/22/2022 5:41:06 PM, 2/23/2022 9:24:01 AM, 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -195.83408055766438 (T) = (0 -1788.8803178898627) / Math.Sqrt((126.66793590438694 / (46)) + (1.116326386094395 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.22696089325172697 = (1457.9766378282202 - 1788.8803178898627) / 1457.9766378282202 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.TryGetValueTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 9.78 μs 12.01 μs 1.23 0.01 False
IDictionary - Duration of single invocation 10.38 μs 12.24 μs 1.18 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<String, String>.Dictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.012891595842882 > 10.333787910923453.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -42.75350088159367 (T) = (0 -11905.650683167747) / Math.Sqrt((29455.27181408835 / (46)) + (15361.080076496253 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.19341150351488232 = (9976.148753470834 - 11905.650683167747) / 9976.148753470834 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueTrue&lt;String, String&gt;.IDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.239603924662124 > 11.164492989068425.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -35.5965631259579 (T) = (0 -12387.271198156424) / Math.Sqrt((15828.184885936735 / (46)) + (25104.080690521205 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.17269885012258407 = (10563.045403226552 - 12387.271198156424) / 10563.045403226552 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 8.11 μs 8.88 μs 1.09 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.HashSet(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.876139627833837 > 8.577192635751029.
IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 2/24/2022 10:17:16 AM, 3/30/2022 6:37:35 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -23.196036177514152 (T) = (0 -8800.573426243192) / Math.Sqrt((4958.130518759598 / (46)) + (6837.403909346844 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.07663830165230878 = (8174.122556049713 - 8800.573426243192) / 8174.122556049713 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array - Duration of single invocation 15.60 μs 18.12 μs 1.16 0.20 False
ImmutableArray - Duration of single invocation 16.24 μs 18.22 μs 1.12 0.40 False
Span - Duration of single invocation 14.17 μs 17.16 μs 1.21 0.18 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.Array(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.120819354204603 > 16.47811771003948.
IsChangePoint: Marked as a change because one of 2/12/2022 1:03:18 PM, 2/19/2022 11:51:48 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -3.754539523158522 (T) = (0 -19692.48542669895) / Math.Sqrt((67524.00555975518 / (46)) + (12301418.98113097 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.25274672042341356 = (15719.446800900922 - 19692.48542669895) / 15719.446800900922 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.216926046511627 > 17.045126678226854.
IsChangePoint: Marked as a change because one of 4/1/2022 5:08:23 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -16.277176572696742 (T) = (0 -18933.13938566031) / Math.Sqrt((387230.7914233105 / (46)) + (323884.6124397518 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.2008921081342669 = (15765.895418427941 - 18933.13938566031) / 15765.895418427941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.162787104166668 > 14.860787836402583.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -56.37745256859708 (T) = (0 -17309.477641302343) / Math.Sqrt((137420.62176312337 / (45)) + (3240.0922790222303 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.23223663543621278 = (14047.202577429272 - 17309.477641302343) / 14047.202577429272 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array - Duration of single invocation 485.89 μs 535.57 μs 1.10 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.Array(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 535.5687054166667 > 510.8680233637456.
IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 2/19/2022 11:51:48 PM, 2/23/2022 6:59:06 PM, 2/26/2022 10:27:55 AM, 3/7/2022 10:45:01 PM, 3/14/2022 4:00:26 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -36.68432197211849 (T) = (0 -538645.4987413066) / Math.Sqrt((6740271.210843837 / (46)) + (19317703.938823342 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.10368232184669923 = (488043.9670720069 - 538645.4987413066) / 488043.9670720069 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableArray - Duration of single invocation 19.85 μs 21.74 μs 1.09 0.32 False
ICollection - Duration of single invocation 18.27 μs 20.37 μs 1.11 0.34 False
List - Duration of single invocation 17.85 μs 19.15 μs 1.07 0.37 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.ImmutableArray(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.735764957636952 > 20.94848438994739.
IsChangePoint: Marked as a change because one of 2/11/2022 3:03:06 PM, 3/14/2022 9:50:33 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -35.31725186652747 (T) = (0 -21760.64298565448) / Math.Sqrt((125990.85874708007 / (46)) + (694.0362929522713 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.09398596463551001 = (19891.153715947905 - 21760.64298565448) / 19891.153715947905 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.370499289905013 > 19.210780087584983.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -30.997225024200286 (T) = (0 -20319.064489712433) / Math.Sqrt((199191.7420539108 / (46)) + (3559.246483247914 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.11615817750249463 = (18204.46680342224 - 20319.064489712433) / 18204.46680342224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.152400795474442 > 18.782756266442927.
IsChangePoint: Marked as a change because one of 2/2/2022 4:36:37 PM, 2/19/2022 11:51:48 PM, 2/22/2022 2:43:53 PM, 4/14/2022 8:50:18 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -9.945159245880799 (T) = (0 -19819.327255541608) / Math.Sqrt((89074.68256826104 / (37)) + (735952.3496783499 / (20))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (20) - 2, .025) and -0.11031653104581432 = (17850.159572850505 - 19819.327255541608) / 17850.159572850505 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 2.95 ns 4.15 ns 1.41 0.01 False
TryParse - Duration of single invocation 2.95 ns 4.16 ns 1.41 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Boolean*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Boolean.TryParse(value: "false")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.148303603474224 > 3.098015135407483.
IsChangePoint: Marked as a change because one of 2/8/2022 11:58:21 PM, 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -1136.0020196335684 (T) = (0 -4.153779206131033) / Math.Sqrt((5.770776123963784E-06 / (46)) + (1.0966556572170448E-05 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.40793943806305827 = (2.950254175595439 - 4.153779206131033) / 2.950254175595439 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Boolean.TryParse(value: "False")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.155272208792124 > 3.099267660750911.
IsChangePoint: Marked as a change because one of 2/3/2022 4:53:52 PM, 2/8/2022 11:58:21 PM, 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -620.2968343528779 (T) = (0 -4.155860483121808) / Math.Sqrt((0.00014166940770667279 / (46)) + (7.555505035106985E-06 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.4078119582643355 = (2.951999703316549 - 4.155860483121808) / 2.951999703316549 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.12
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare a39ddca87f9d8e99d74ef44f280761bde578aca2
Diff Diff

Regressions in System.Collections.TryGetValueFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IDictionary - Duration of single invocation 8.33 μs 10.15 μs 1.22 0.01 False
Dictionary - Duration of single invocation 7.61 μs 9.04 μs 1.19 0.00 True

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<String, String>.IDictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.152635144106219 > 8.782365383657185.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -57.398790659994624 (T) = (0 -10085.722866941116) / Math.Sqrt((3745.060874051329 / (46)) + (9210.274335290123 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.2084570769211336 = (8345.950435109522 - 10085.722866941116) / 8345.950435109522 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueFalse&lt;String, String&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.043817840713052 > 8.010285376607378.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/19/2022 4:39:14 PM falls between 4/10/2022 11:44:05 PM and 4/19/2022 4:39:14 PM.
IsRegressionStdDev: Marked as regression because -40.4058070579155 (T) = (0 -9008.503563649987) / Math.Sqrt((5794.528993533562 / (46)) + (10930.488841572553 / (11))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (11) - 2, .025) and -0.1765857382257541 = (7656.478632177254 - 9008.503563649987) / 7656.478632177254 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions