Skip to content

[Perf] Linux/x64: 32 Regressions on 10/25/2022 7:09:53 PM #9302

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 329.75 μs 348.21 μs 1.06 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsTrue<String>.Span(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 348.2055614087301 > 345.6457712109375.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -206.85114371288367 (T) = (0 -348224.7859634886) / Math.Sqrt((74161.45389414101 / (21)) + (150050.10647381173 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.05805811303646832 = (329116.880890536 - 348224.7859634886) / 329116.880890536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 4.16 ms 4.49 ms 1.08 0.01 False
Count - Duration of single invocation 1.30 ms 1.55 ms 1.19 0.26 False
Count - Duration of single invocation 190.22 μs 209.26 μs 1.10 0.01 False
Count - Duration of single invocation 971.93 μs 1.03 ms 1.06 0.03 False
Count - Duration of single invocation 2.67 ms 3.19 ms 1.19 0.04 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: None)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.493463667132867 > 4.378656754727402.
IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -17.066343101010023 (T) = (0 -4410648.13644602) / Math.Sqrt((827331810.7820512 / (21)) + (4424998451.8592415 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.055852433861574714 = (4177333.872608441 - 4410648.13644602) / 4177333.872608441 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.547016403409091 > 1.361408795984456.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 10/6/2022 2:13:23 PM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -18.310968982104516 (T) = (0 -1482803.5309941685) / Math.Sqrt((474278.6164436224 / (21)) + (3105909146.1308436 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.14372428967482215 = (1296469.3889781353 - 1482803.5309941685) / 1296469.3889781353 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 209.26475935897435 > 199.8091286017693.
IsChangePoint: Marked as a change because one of 8/17/2022 1:34:50 AM, 10/26/2022 8:56:26 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -12.540616586135576 (T) = (0 -202689.58903467935) / Math.Sqrt((13179114.31792569 / (29)) + (9335908.352595696 / (22))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (22) - 2, .025) and -0.06157223396348942 = (190933.39346105244 - 202689.58903467935) / 190933.39346105244 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.0336038518229167 > 1023.2463060107423.
IsChangePoint: Marked as a change because one of 8/15/2022 9:13:40 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -24.579952971041244 (T) = (0 -1054510.263536773) / Math.Sqrt((25627955.00179588 / (21)) + (263072765.7990181 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.07953103099040411 = (976822.55837456 - 1054510.263536773) / 976822.55837456 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.1887315467032966 > 2.802750828446228.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/27/2022 11:22:50 AM, 10/2/2022 4:08:26 PM, 10/27/2022 12:34:03 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -44.270615550995615 (T) = (0 -3192920.87465036) / Math.Sqrt((3433858303.7520585 / (29)) + (26149832.85385392 / (21))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (21) - 2, .025) and -0.17878246986091595 = (2708659.9574449826 - 3192920.87465036) / 2708659.9574449826 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
StaticDelegate - Duration of single invocation 195.82 μs 217.66 μs 1.11 0.02 False
ForeachOverList100Elements - Duration of single invocation 9.45 ms 10.53 ms 1.11 0.24 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

PerfLabTests.LowLevelPerf.StaticDelegate


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 217.6633865451389 > 205.61560286914062.
IsChangePoint: Marked as a change because one of 10/6/2022 2:13:23 PM, 10/11/2022 8:42:28 PM, 10/17/2022 12:47:34 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -491.62062971113295 (T) = (0 -217558.02159717455) / Math.Sqrt((31145.434867629356 / (21)) + (14040.43149598162 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.11088342291765999 = (195842.351329515 - 217558.02159717455) / 195842.351329515 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.ForeachOverList100Elements

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 10.529357680952382 > 9.902362920000002.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -13.800848561364147 (T) = (0 -10508280.02758456) / Math.Sqrt((75134388575.36307 / (21)) + (55798726029.99199 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.10723183982890894 = (9490586.930022094 - 10508280.02758456) / 9490586.930022094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 643.11 μs 719.90 μs 1.12 0.06 False
Span - Duration of single invocation 653.35 μs 692.32 μs 1.06 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsFalse<String>.Queue(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 719.8986818181819 > 675.5438804625.
IsChangePoint: Marked as a change because one of 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -24.991900567335207 (T) = (0 -720077.9707410646) / Math.Sqrt((174888517.99509206 / (20)) + (5003414.553383169 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.11558179327996818 = (645473.0393402474 - 720077.9707410646) / 645473.0393402474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 692.3166888586956 > 686.176015219351.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -177.07157018193342 (T) = (0 -690190.933364472) / Math.Sqrt((530786.6195511797 / (21)) + (549137.5919525089 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.056588719573394 = (653225.7259410663 - 690190.933364472) / 653225.7259410663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 16.03 μs 17.01 μs 1.06 0.04 False
ConcurrentDictionary - Duration of single invocation 2.35 μs 2.56 μs 1.09 0.12 False
ImmutableSortedDictionary - Duration of single invocation 28.24 μs 30.47 μs 1.08 0.02 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableDictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 17.012081173007246 > 16.805652585642605.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/24/2022 1:05:07 PM, 10/2/2022 4:08:26 PM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -47.50671320010254 (T) = (0 -17001.18709632753) / Math.Sqrt((6150.357983167147 / (21)) + (3779.6584697791754 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.06065784020719337 = (16028.908147236667 - 17001.18709632753) / 16028.908147236667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;.ConcurrentDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.5610707569596776 > 2.4665053247981485.
IsChangePoint: Marked as a change because one of 8/15/2022 9:13:40 PM, 10/26/2022 7:11:30 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -8.772192313374665 (T) = (0 -2559.366384941995) / Math.Sqrt((8025.478223080907 / (25)) + (830.935643360025 / (26))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (26) - 2, .025) and -0.06882669439386868 = (2394.556945823112 - 2559.366384941995) / 2394.556945823112 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;.ImmutableSortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 30.465889662991117 > 29.639150447314986.
IsChangePoint: Marked as a change because one of 8/17/2022 1:34:50 AM, 10/11/2022 8:42:28 PM, 10/17/2022 12:47:34 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -26.32998319617529 (T) = (0 -30272.59817068933) / Math.Sqrt((79354.79165545886 / (21)) + (48156.74285734917 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.06817024755190104 = (28340.611658178976 - 30272.59817068933) / 28340.611658178976 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 10.73 μs 12.00 μs 1.12 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 11.996411522239264 > 11.228035864579732.
IsChangePoint: Marked as a change because one of 8/15/2022 9:13:40 PM, 9/23/2022 1:16:35 AM, 9/24/2022 1:05:07 PM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -130.3921189055388 (T) = (0 -12013.759514239659) / Math.Sqrt((1167.9439779562194 / (21)) + (1384.6536996364002 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.12295552849088558 = (10698.339524081312 - 12013.759514239659) / 10698.339524081312 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.TryAddGiventSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 3.65 μs 4.39 μs 1.20 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddGiventSize&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.TryAddGiventSize<Int32>.Dictionary(Count: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.385938354224699 > 3.8692905314237347.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/3/2022 11:44:47 AM, 9/26/2022 8:26:31 AM, 9/27/2022 11:22:50 AM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -106.77578734580163 (T) = (0 -4400.73025512728) / Math.Sqrt((466.81927228543293 / (21)) + (699.5872875016653 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.19581813177207344 = (3680.099956843665 - 4400.73025512728) / 3680.099956843665 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentStack - Duration of single invocation 1.87 μs 2.04 μs 1.09 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEach<Int32>.ConcurrentStack(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.03547957090767 > 1.9664464559673966.
IsChangePoint: Marked as a change because one of 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 10/6/2022 2:13:23 PM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -216.10574352854545 (T) = (0 -2032.0408076969043) / Math.Sqrt((6.494821879483919 / (21)) + (6.8336840647786 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.0845255035453215 = (1873.6680705563388 - 2032.0408076969043) / 1873.6680705563388 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7175e861011e323295f5fcfc0a5fac648974b80d
Compare 54b12a83a329dbc4079c254dc7071cb528e1f2f1
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetDateTime - Duration of single invocation 3.44 μs 3.72 μs 1.08 0.08 False
GetDateTimeOffset - Duration of single invocation 4.57 μs 4.96 μs 1.09 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Get.GetDateTime


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.71621226471289 > 3.491275978957627.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -45.98899885496576 (T) = (0 -3762.501630236339) / Math.Sqrt((698.2971355265902 / (21)) + (1664.5884953435989 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.13012469791721704 = (3329.280067208961 - 3762.501630236339) / 3329.280067208961 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetDateTimeOffset

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.964670451719576 > 4.799204002144576.
IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -13.359149183682042 (T) = (0 -4929.35426902658) / Math.Sqrt((7849.8676678818565 / (21)) + (1074.5974411057664 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.05803373507887121 = (4658.976463221299 - 4929.35426902658) / 4658.976463221299 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions