Skip to content

[Perf] Alpine/x64: 24 Regressions on 10/25/2022 7:09:53 PM #9390

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS alpine 3.15
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 16.01 μs 17.27 μs 1.08 0.03 False
ImmutableSortedDictionary - Duration of single invocation 26.96 μs 28.52 μs 1.06 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableDictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 17.26624631339581 > 16.820055925916048.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/5/2022 10:47:59 PM, 9/13/2022 12:23:36 PM, 9/24/2022 1:05:07 PM, 10/2/2022 4:08:26 PM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -21.266356146433843 (T) = (0 -17151.28504846523) / Math.Sqrt((19598.623493079685 / (21)) + (53800.63904831941 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.06922756653837418 = (16040.818236657085 - 17151.28504846523) / 16040.818236657085 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;.ImmutableSortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 28.519856397727267 > 28.319605542017023.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 10/2/2022 4:08:26 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -44.41394702117482 (T) = (0 -28394.52062879577) / Math.Sqrt((962.3143432970785 / (21)) + (28862.01685902893 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.052253090018418144 = (26984.497264150363 - 28394.52062879577) / 26984.497264150363 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS alpine 3.15
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 8.18 μs 11.03 μs 1.35 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.CtorFromCollection<Int32>.SortedSet(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 11.034206963538555 > 8.424936107126076.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 8/4/2022 6:25:19 PM, 9/9/2022 3:12:19 AM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -12.246298592720782 (T) = (0 -10465.771201668931) / Math.Sqrt((20675.360272373644 / (21)) + (1144821.9705172256 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.30125628225947776 = (8042.820883443772 - 10465.771201668931) / 8042.820883443772 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
StaticDelegate - Duration of single invocation 195.69 μs 217.42 μs 1.11 0.01 False
ForeachOverList100Elements - Duration of single invocation 9.33 ms 10.67 ms 1.14 0.22 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

PerfLabTests.LowLevelPerf.StaticDelegate


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 217.41607064636753 > 205.46993679086538.
IsChangePoint: Marked as a change because one of 10/6/2022 2:13:23 PM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -1011.5122623777582 (T) = (0 -217470.69884334513) / Math.Sqrt((7907.118038470175 / (21)) + (2606.192783230411 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.11126746638841162 = (195696.09065414185 - 217470.69884334513) / 195696.09065414185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.ForeachOverList100Elements

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 10.665857185714287 > 9.9811385825.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 9/23/2022 1:16:35 AM, 10/25/2022 7:09:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -17.040466961409106 (T) = (0 -10622243.145008033) / Math.Sqrt((55401256809.373474 / (22)) + (50524392840.987434 / (29))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (29) - 2, .025) and -0.11695811374344982 = (9509974.469327163 - 10622243.145008033) / 9509974.469327163 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 3.96 μs 4.26 μs 1.07 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.AddGivenSize<Int32>.Dictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.258074490878714 > 4.149980498451899.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -24.814340809971956 (T) = (0 -4237.464828569728) / Math.Sqrt((261.41099759302926 / (21)) + (3785.390458900583 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.07405389662081163 = (3945.2999909051487 - 4237.464828569728) / 3945.2999909051487 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 634.69 μs 710.56 μs 1.12 0.04 False
Span - Duration of single invocation 619.18 μs 680.70 μs 1.10 0.02 False
ImmutableArray - Duration of single invocation 637.14 μs 710.38 μs 1.11 0.06 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsFalse<String>.Queue(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 710.5600288825757 > 666.02489971875.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -102.06389270420716 (T) = (0 -712209.9616748443) / Math.Sqrt((1165308.06310645 / (21)) + (15633160.763515335 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.12210500916859217 = (634708.8337147217 - 712209.9616748443) / 634708.8337147217 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 680.6986940993788 > 650.5312406596707.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/5/2022 10:47:59 PM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -654.6084844052551 (T) = (0 -680452.8991740524) / Math.Sqrt((69393.45127181447 / (21)) + (161051.02304834814 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.09840724006518355 = (619490.5444484068 - 680452.8991740524) / 619490.5444484068 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 710.3841102272727 > 666.2685437019231.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/5/2022 10:47:59 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -13.05344272050602 (T) = (0 -710110.7891486955) / Math.Sqrt((548223287.8169979 / (21)) + (927883.3029728808 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.10372568604779928 = (643376.155982604 - 710110.7891486955) / 643376.155982604 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 15.59 μs 16.79 μs 1.08 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableDictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 16.79487459109452 > 16.37365980671773.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/13/2022 12:23:36 PM, 9/24/2022 1:05:07 PM, 10/2/2022 4:08:26 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -18.667739128833677 (T) = (0 -16736.787580438922) / Math.Sqrt((49555.68641412697 / (21)) + (38298.99927759474 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.07211017106242548 = (15611.070608399623 - 16736.787580438922) / 15611.070608399623 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArraySetLengthShrink - Duration of single invocation 132.37 ns 145.74 ns 1.10 0.03 False
BitArrayByteArrayCtor - Duration of single invocation 130.30 ns 144.49 ns 1.11 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 145.74255713214933 > 139.30988840589552.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 10/6/2022 2:13:23 PM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -83.02693115859027 (T) = (0 -145.84636635797926) / Math.Sqrt((0.40295412237456174 / (21)) + (0.13152903795879373 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.09577410024581234 = (133.09893556095358 - 145.84636635797926) / 133.09893556095358 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 144.4904406014503 > 136.82610008745547.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -117.25855355383182 (T) = (0 -144.24843368329192) / Math.Sqrt((0.18786022365214766 / (21)) + (0.13798278760601831 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.10449329891318188 = (130.60145663647933 - 144.24843368329192) / 130.60145663647933 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 10.47 μs 11.63 μs 1.11 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 11.629603707002632 > 10.902490004190991.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/1/2022 8:26:50 AM, 8/4/2022 6:25:19 PM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -79.43265493200644 (T) = (0 -11626.34652032774) / Math.Sqrt((4117.4096560040125 / (21)) + (1336.9230798087037 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.11854561985713298 = (10394.163916007934 - 11626.34652032774) / 10394.163916007934 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 312.16 μs 343.17 μs 1.10 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsTrue<String>.Span(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 343.1653813082298 > 327.63041975395925.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/5/2022 10:47:59 PM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -387.0462966001081 (T) = (0 -343254.7442769466) / Math.Sqrt((62656.85710927163 / (21)) + (104050.15806702842 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.0995921545860681 = (312165.50868004496 - 343254.7442769466) / 312165.50868004496 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions