Skip to content

[Perf] Changes at 4/8/2022 8:59:46 PM #4576

Closed
@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 24.17 μs 26.22 μs 1.08 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableDictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.222932209930544 > 25.25924379029418.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/21/2022 1:28:14 PM, 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -33.13322634367094 (T) = (0 -26194.832335236242) / Math.Sqrt((16980.837561018365 / (39)) + (65952.09843920812 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.0881224660590953 = (24073.42293933817 - 26194.832335236242) / 24073.42293933817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentBag - Duration of single invocation 2.14 μs 2.49 μs 1.16 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ConcurrentBag(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4930842014102157 > 2.2449272418560535.
IsChangePoint: Marked as a change because one of 3/3/2022 7:35:51 PM, 3/11/2022 8:42:43 PM, 3/23/2022 6:35:24 PM, 3/29/2022 11:07:04 AM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -70.44286894837911 (T) = (0 -2495.333820793321) / Math.Sqrt((968.7619655387663 / (39)) + (3.1703155392944633 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.16440848707366165 = (2143.005524688746 - 2495.333820793321) / 2143.005524688746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateGetPositionSingleSegment - Duration of single invocation 32.93 ns 35.27 ns 1.07 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionSingleSegment


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.27249963223523 > 34.58420222384465.
IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -15.834575901314098 (T) = (0 -34.85541144276243) / Math.Sqrt((0.2389035291315352 / (39)) + (0.10070296015102535 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.05172600629783976 = (33.141152005412785 - 34.85541144276243) / 33.141152005412785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Char_ToLowerInvariant - Duration of single invocation 22.45 ns 29.28 ns 1.30 0.08 False
Char_ToUpperInvariant - Duration of single invocation 33.09 ns 37.04 ns 1.12 0.10 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.27720794595902 > 23.783293586902115.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/24/2022 10:17:16 AM, 3/4/2022 3:52:45 PM, 3/11/2022 8:42:43 PM, 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -122.19227935359008 (T) = (0 -29.33356670733879) / Math.Sqrt((0.07683104693362362 / (39)) + (0.01596188379132147 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.2864184013977807 = (22.802508635966248 - 29.33356670733879) / 22.802508635966248 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.Char_ToUpperInvariant(input: "Hello World!")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.03828763176484 > 34.81715721542249.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/23/2022 5:12:02 AM, 3/16/2022 12:48:00 PM, 3/21/2022 1:28:14 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -133.42685412240755 (T) = (0 -37.0531314665092) / Math.Sqrt((0.016902597752947876 / (39)) + (0.007777958102332498 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.11849136122943592 = (33.12777617323814 - 37.0531314665092) / 33.12777617323814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.CtorFromCollectionNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 29.17 μs 31.40 μs 1.08 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollectionNonGeneric<Int32>.Queue(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.402713462639568 > 30.442382035450507.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -11.96348823909402 (T) = (0 -31390.121849613748) / Math.Sqrt((1219808.6472099978 / (39)) + (116715.47999533376 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.0799845400953987 = (29065.343700976453 - 31390.121849613748) / 29065.343700976453 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Linq.Tests.Perf_OrderBy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OrderByValueType - Duration of single invocation 54.44 μs 57.48 μs 1.06 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_OrderBy*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_OrderBy.OrderByValueType(NumberOfPeople: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.482879792542015 > 57.25308219410163.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 3/29/2022 11:07:04 AM, 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -60.71499059062252 (T) = (0 -57433.543226083646) / Math.Sqrt((57904.73452049728 / (39)) + (10126.865132610692 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.050235659808586006 = (54686.33890849924 - 57433.543226083646) / 54686.33890849924 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.IterateForNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayList - Duration of single invocation 1.45 μs 1.61 μs 1.11 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForNonGeneric&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForNonGeneric<Int32>.ArrayList(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6085317357895048 > 1.524575460310654.
IsChangePoint: Marked as a change because one of 2/4/2022 6:00:25 PM, 2/12/2022 12:09:13 AM, 2/19/2022 11:51:48 PM, 3/4/2022 3:52:45 PM, 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/23/2022 3:13:36 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -12.047675136062567 (T) = (0 -1608.1628730034784) / Math.Sqrt((5235.722749420881 / (39)) + (0.24785821064339852 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.0950579892306685 = (1468.5641206392102 - 1608.1628730034784) / 1468.5641206392102 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 2.84 μs 3.31 μs 1.17 0.26 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.Stack(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3129564521741752 > 2.983069160348327.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -19.149407454553266 (T) = (0 -3283.4104539573236) / Math.Sqrt((6786.216701386223 / (39)) + (3636.233110815095 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.12751361846411774 = (2912.0805285083266 - 3283.4104539573236) / 2912.0805285083266 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 83.66 ns 90.31 ns 1.08 0.08 False
EncodeUtf16 - Duration of single invocation 108.58 ns 115.86 ns 1.07 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,©2020,512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.31272499811051 > 87.74520586030843.
IsChangePoint: Marked as a change because one of 2/21/2022 2:16:42 PM, 3/3/2022 7:35:51 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -12.321904327319015 (T) = (0 -92.1991133668373) / Math.Sqrt((2.994037618511004 / (39)) + (4.64236409287772 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.08379386223687786 = (85.0707100117217 - 92.1991133668373) / 85.0707100117217 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 115.86048335006498 > 114.39500547051426.
IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/29/2022 11:07:04 AM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -9.689921666491882 (T) = (0 -116.15381376850921) / Math.Sqrt((1.4565102635441782 / (39)) + (8.543542226116443 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.06348139040908804 = (109.2203538454288 - 116.15381376850921) / 109.2203538454288 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLineAsync - Duration of single invocation 21.53 μs 27.05 μs 1.26 0.01 False
ReadLineAsync - Duration of single invocation 21.46 μs 25.90 μs 1.21 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024])


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.047377409297052 > 22.556814918478338.
IsChangePoint: Marked as a change because one of 2/24/2022 10:17:16 AM, 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -51.2063026531744 (T) = (0 -26141.089020456988) / Math.Sqrt((45832.41946916701 / (39)) + (144546.44028967695 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.23143960805798175 = (21228.072289864293 - 26141.089020456988) / 21228.072289864293 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.89828390993959 > 22.658561493612716.
IsChangePoint: Marked as a change because one of 2/24/2022 10:17:16 AM, 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -69.48965885335853 (T) = (0 -25904.05210476315) / Math.Sqrt((44912.81545588085 / (39)) + (49328.05893952455 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.2009939397796785 = (21568.844976449447 - 25904.05210476315) / 21568.844976449447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 13.03 ns 15.44 ns 1.18 0.49 False
SequenceEqual - Duration of single invocation 13.22 ns 14.74 ns 1.12 0.10 False
BinarySearch - Duration of single invocation 13.13 ns 16.35 ns 1.25 0.09 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.LastIndexOfValue(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.439962444559955 > 14.900140201401067.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/24/2022 10:17:16 AM, 3/11/2022 8:42:43 PM, 3/21/2022 1:28:14 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -10.24844448484873 (T) = (0 -15.30584233759866) / Math.Sqrt((1.4595934210259047 / (39)) + (0.32538490567838196 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.1872894185671829 = (12.891416446774791 - 15.30584233759866) / 12.891416446774791 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.SequenceEqual(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.73969601972988 > 13.902668539397569.
IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 4/4/2022 8:14:13 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -17.571482898430887 (T) = (0 -14.57997749063767) / Math.Sqrt((0.03204914487118026 / (17)) + (0.14470823956598625 / (40))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (40) - 2, .025) and -0.09817995995997533 = (13.27649203430106 - 14.57997749063767) / 13.27649203430106 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.BinarySearch(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.352312156389807 > 13.360557518355241.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 2/19/2022 11:51:48 PM, 2/25/2022 11:02:46 PM, 3/17/2022 6:54:53 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -78.98468015637509 (T) = (0 -16.154763871463985) / Math.Sqrt((0.027376226211249174 / (39)) + (0.019334620340563147 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.2595276796908698 = (12.826049107097752 - 16.154763871463985) / 12.826049107097752 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HeapSort - Duration of single invocation 255.00 ns 277.85 ns 1.09 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 10)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 277.848078601306 > 268.22308977882767.
IsChangePoint: Marked as a change because one of 2/10/2022 7:02:09 PM, 2/25/2022 7:19:51 AM, 3/17/2022 6:54:53 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -49.29452651722794 (T) = (0 -278.60089842069203) / Math.Sqrt((6.220631268190085 / (39)) + (0.7268071445043258 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.08589982822028323 = (256.56224559617084 - 278.60089842069203) / 256.56224559617084 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions