Skip to content

Regressions in Burgers.Test3 #80129

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 50.00 ns 57.82 ns 1.16 0.05 False
EncodeUtf8 - Duration of single invocation 71.39 ns 81.55 ns 1.14 0.04 False
EncodeUtf8 - Duration of single invocation 46.24 ns 54.00 ns 1.17 0.04 False
EncodeUtf8 - Duration of single invocation 46.55 ns 54.03 ns 1.16 0.04 False
EncodeUtf16 - Duration of single invocation 54.81 ns 62.79 ns 1.15 0.05 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.81583277258387 > 52.417481129298075.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 11/22/2022 2:20:42 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -25.145786383702696 (T) = (0 -57.933217934322485) / Math.Sqrt((2.3623964474246115 / (29)) + (0.1613397566644288 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.15254773986311304 = (50.265352080949945 - 57.933217934322485) / 50.265352080949945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,©2020,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.54861659265462 > 75.32232075944864.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -114.0821565588086 (T) = (0 -81.65124012302711) / Math.Sqrt((0.06638959609032156 / (29)) + (0.07100614020406576 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.1362020840470246 = (71.86330783006007 - 81.65124012302711) / 71.86330783006007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no <escaping /> required,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.00301563830678 > 48.5118806196726.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -105.69282709382192 (T) = (0 -54.18273969123065) / Math.Sqrt((0.08034202410386392 / (29)) + (0.04128019098655086 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.1730450971989896 = (46.1898181243064 - 54.18273969123065) / 46.1898181243064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.02755372264943 > 48.69273409220534.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -170.4406772588238 (T) = (0 -54.13547871090059) / Math.Sqrt((0.023406243279603695 / (29)) + (0.018393288510694467 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.16958389434541218 = (46.28610138411568 - 54.13547871090059) / 46.28610138411568 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.78543086769447 > 57.4718640789619.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 11/22/2022 2:20:42 AM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -49.258615720152235 (T) = (0 -62.81305937612173) / Math.Sqrt((0.7287449900611545 / (29)) + (0.0011130533439879145 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.14221852718437203 = (54.99215595019211 - 62.81305937612173) / 54.99215595019211 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScalarValueTypeObj - Duration of single invocation 167.08 μs 250.53 μs 1.50 0.02 False
IFooObjIsIFooInterAlia - Duration of single invocation 417.88 μs 445.61 μs 1.07 0.05 False
ObjObjrefValueType - Duration of single invocation 832.79 μs 947.79 μs 1.14 0.12 False
ObjrefValueTypeObj - Duration of single invocation 779.82 μs 946.65 μs 1.21 0.08 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 250.52506720430105 > 177.88717126125206.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -157.34889274663954 (T) = (0 -250698.64349911804) / Math.Sqrt((7772358.433191751 / (29)) + (10568.652988295244 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.4823308478022079 = (169124.6214506153 - 250698.64349911804) / 169124.6214506153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFooInterAlia

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 445.60746527777786 > 438.70099191337715.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.460168000145664 (T) = (0 -445711.48518582113) / Math.Sqrt((51528625.10230869 / (29)) + (59427.090821034675 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.061874146213374455 = (419740.4059372016 - 445711.48518582113) / 419740.4059372016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.ObjObjrefValueType

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 947.7931843891404 > 874.5227331098178.
IsChangePoint: Marked as a change because one of 12/6/2022 6:34:01 PM, 12/23/2022 9:40:21 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.770505381122266 (T) = (0 -937213.1517214874) / Math.Sqrt((353415415.0537492 / (17)) + (188433114.7229348 / (26))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (26) - 2, .025) and -0.1186275434229421 = (837824.1330028974 - 937213.1517214874) / 837824.1330028974 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.ObjrefValueTypeObj

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 946.6478789592763 > 818.8279194078948.
IsChangePoint: Marked as a change because one of 12/9/2022 7:57:12 PM, 12/10/2022 11:25:25 PM, 12/23/2022 9:40:21 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -26.85181093902272 (T) = (0 -925516.8672099016) / Math.Sqrt((32723.634549815342 / (17)) + (766296265.8028983 / (26))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (26) - 2, .025) and -0.1869612719955682 = (779736.3646531487 - 925516.8672099016) / 779736.3646531487 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 111.86 ns 118.54 ns 1.06 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortSingleLine)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 118.54351975679164 > 118.10710017101991.
IsChangePoint: Marked as a change because one of 10/26/2022 2:40:44 AM, 10/31/2022 10:23:25 PM, 11/7/2022 4:10:19 PM, 11/14/2022 1:59:26 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -43.09754038158343 (T) = (0 -119.18940387461092) / Math.Sqrt((0.29301881287536213 / (29)) + (0.18750842099727472 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.05868011221014512 = (112.58301964866999 - 119.18940387461092) / 112.58301964866999 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WhereLast_LastElementMatches - Duration of single invocation 650.22 ns 724.73 ns 1.11 0.05 False
WhereFirst_LastElementMatches - Duration of single invocation 639.30 ns 702.08 ns 1.10 0.14 False
WhereSelect - Duration of single invocation 838.32 ns 894.79 ns 1.07 0.05 False
Min - Duration of single invocation 440.62 ns 468.21 ns 1.06 0.07 False
Where - Duration of single invocation 689.69 ns 736.30 ns 1.07 0.04 False
WhereSelect - Duration of single invocation 1.25 μs 1.34 μs 1.07 0.11 False
Repeat - Duration of single invocation 382.25 ns 410.60 ns 1.07 0.05 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: IEnumerable)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 724.7254658714982 > 682.5974208911517.
IsChangePoint: Marked as a change because one of 10/11/2022 5:13:36 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.36085607764159 (T) = (0 -716.0102863111845) / Math.Sqrt((183.58905449092907 / (29)) + (70.05995345922597 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.0944921946076017 = (654.194054410675 - 716.0102863111845) / 654.194054410675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 702.0825666457229 > 671.7574198262187.
IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.88758054547226 (T) = (0 -698.0520999185754) / Math.Sqrt((221.8120470681958 / (29)) + (3.2052738842730317 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.08218407656808165 = (645.0400768530067 - 698.0520999185754) / 645.0400768530067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 894.7945473488121 > 881.9290342422098.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -12.710372271437192 (T) = (0 -899.8308587803176) / Math.Sqrt((265.31080182033116 / (29)) + (150.9514640344226 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.06730533284694304 = (843.0866323698561 - 899.8308587803176) / 843.0866323698561 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 468.213976760022 > 462.9503218579129.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 8:57:30 PM, 11/10/2022 9:13:35 AM, 12/3/2022 7:24:45 AM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/28/2022 7:19:51 AM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.29721173302842 (T) = (0 -467.92709626144557) / Math.Sqrt((47.436089684565395 / (28)) + (0.049884560962766014 / (15))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (15) - 2, .025) and -0.05678094445247723 = (442.7853271936888 - 467.92709626144557) / 442.7853271936888 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Where(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 736.2994828147459 > 716.8184132859758.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -20.26576807083207 (T) = (0 -738.2921684769659) / Math.Sqrt((189.62581591898518 / (29)) + (3.4337103742602078 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.07700104364540426 = (685.5073844478501 - 738.2921684769659) / 685.5073844478501 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3438325908230024 > 1.3145043872684157.
IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 12/5/2022 3:14:07 PM, 12/10/2022 11:25:25 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -9.562000836067062 (T) = (0 -1397.2041165898495) / Math.Sqrt((979.4742107775817 / (29)) + (1714.605452420122 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.09354748590304915 = (1277.6803335943307 - 1397.2041165898495) / 1277.6803335943307 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Repeat

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 410.59924185730245 > 401.7604620467154.
IsChangePoint: Marked as a change because one of 10/7/2022 9:25:17 PM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.516430850153665 (T) = (0 -410.7379330275285) / Math.Sqrt((58.546976033624375 / (29)) + (0.04626765149313623 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.06849744333201213 = (384.40703399970675 - 410.7379330275285) / 384.40703399970675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Add - Duration of single invocation 51.38 ns 57.43 ns 1.12 0.05 False
Subtract - Duration of single invocation 53.29 ns 58.68 ns 1.10 0.05 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Add(arguments: 1024,1024 bits)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.42732197836272 > 53.48368915016196.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/18/2022 8:48:12 PM, 12/10/2022 11:25:25 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -60.00067824227273 (T) = (0 -56.74103679987769) / Math.Sqrt((0.12083809240293349 / (29)) + (0.08004781347707045 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.11748385839528891 = (50.775710426241 - 56.74103679987769) / 50.775710426241 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 1024,1024 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.680051163526294 > 55.93998442951785.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 11/18/2022 8:48:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -51.453698525063 (T) = (0 -58.539282567362946) / Math.Sqrt((0.2313144713726593 / (29)) + (0.0291126567627274 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.09666085854620245 = (53.37956772248262 - 58.539282567362946) / 53.37956772248262 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ChangeType - Duration of single invocation 23.38 ns 26.97 ns 1.15 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ChangeType


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.96547629232403 > 24.831430125103818.
IsChangePoint: Marked as a change because one of 11/1/2022 6:58:35 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.40126556261769 (T) = (0 -26.82030325896489) / Math.Sqrt((0.140839807192652 / (29)) + (0.24387579328379141 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.12103361363189857 = (23.924620040672195 - 26.82030325896489) / 23.924620040672195 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 1.74 μs 2.16 μs 1.24 0.04 False
ImmutableQueue - Duration of single invocation 3.73 μs 4.02 μs 1.08 0.10 False
ConcurrentStack - Duration of single invocation 3.99 μs 4.88 μs 1.22 0.41 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<String>.List(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.158741014574434 > 1.8229782976610758.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/3/2022 7:24:45 AM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -564.4070078001165 (T) = (0 -2159.2521116215617) / Math.Sqrt((15.50447013384292 / (29)) + (0.3182434813910868 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.2424618037325781 = (1737.8820863021954 - 2159.2521116215617) / 1737.8820863021954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableQueue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.01852280644236 > 3.919239245106011.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/13/2022 5:34:08 PM, 12/10/2022 11:25:25 PM, 12/12/2022 10:47:19 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -6.4175585805493265 (T) = (0 -4093.2681310405783) / Math.Sqrt((1537.6210689605634 / (29)) + (39587.399157815475 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.0918804722671955 = (3748.824376848924 - 4093.2681310405783) / 3748.824376848924 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ConcurrentStack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.877410345631326 > 3.969456488386927.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -8.744637020481978 (T) = (0 -4509.761731346228) / Math.Sqrt((36053.47192347738 / (29)) + (73981.46956850182 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.1857650776947188 = (3803.250590002018 - 4509.761731346228) / 3803.250590002018 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions