Skip to content

[Perf] Changes at 7/6/2022 7:05:00 PM #6723

Closed
@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9534947f2918c4721e4b1dabfa9402057c6e1246
Compare 11bfa9b6fac2a6d50f09c3c8057bbccee9908383
Diff Diff

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateForEachTenSegments - Duration of single invocation 52.63 ns 47.85 ns 0.91 0.02 False
FirstSpanTenSegments - Duration of single invocation 17.42 ns 4.81 ns 0.28 0.02 False
FirstSpanSingleSegment - Duration of single invocation 17.04 ns 5.05 ns 0.30 0.02 False
FirstSpanArray - Duration of single invocation 23.86 ns 4.51 ns 0.19 0.03 False
IterateTryGetTenSegments - Duration of single invocation 56.50 ns 43.74 ns 0.77 0.03 False
FirstSpanMemory - Duration of single invocation 24.09 ns 5.29 ns 0.22 0.09 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachTenSegments


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.85022714647945 < 49.88965038934605.
IsChangePoint: Marked as a change because one of 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 20.79528296387366 (T) = (0 -47.69524109815917) / Math.Sqrt((3.326649326782174 / (299)) + (0.40974911681378595 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06026257805344075 = (50.75379567130987 - 47.69524109815917) / 50.75379567130987 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;.FirstSpanTenSegments

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.809525537268655 < 10.577835054204689.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 3.564871175321872 (T) = (0 -4.830072368133726) / Math.Sqrt((7.89347299398933 / (299)) + (0.00602970928504505 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.1073379774600983 = (5.410863514043831 - 4.830072368133726) / 5.410863514043831 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;.FirstSpanSingleSegment

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.047865548257078 < 10.741854003143859.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 2.6199789960233977 (T) = (0 -5.10966107479831) / Math.Sqrt((8.34546507200745 / (299)) + (0.008957029757272772 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.07918125882368733 = (5.549041137315366 - 5.10966107479831) / 5.549041137315366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;.FirstSpanArray

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.50739577862188 < 13.46105435368254.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 4.899783241451749 (T) = (0 -4.487798583377821) / Math.Sqrt((19.540133638216897 / (299)) + (0.0004936817752485261 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.2182202903222841 = (5.740489971564867 - 4.487798583377821) / 5.740489971564867 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;.IterateTryGetTenSegments

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.74208268747402 < 53.684787670237796.
IsChangePoint: Marked as a change because one of 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 9.018030282212488 (T) = (0 -44.17334117461784) / Math.Sqrt((51.43181498329463 / (299)) + (0.8164095089120201 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.08229488806753751 = (48.13457024511892 - 44.17334117461784) / 48.13457024511892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;.FirstSpanMemory

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.289433883544825 < 14.036889124813888.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 4.510274785990649 (T) = (0 -5.2854095505170635) / Math.Sqrt((18.349989049347837 / (299)) + (0.0006998500780451758 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.1745294230851334 = (6.402904837954224 - 5.2854095505170635) / 6.402904837954224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 9534947f2918c4721e4b1dabfa9402057c6e1246
Compare 11bfa9b6fac2a6d50f09c3c8057bbccee9908383
Diff Diff

Improvements in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Slice_StartPosition_And_EndPosition - Duration of single invocation 12.25 ns 10.88 ns 0.89 0.08 False
Slice_Repeat_StartPosition_And_EndPosition - Duration of single invocation 52.24 ns 43.99 ns 0.84 0.07 False
Slice_StartPosition - Duration of single invocation 10.76 ns 9.09 ns 0.84 0.14 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySequence*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment: Multiple)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.88318230268836 < 11.571529122072073.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 8.342216177242427 (T) = (0 -10.654523865781897) / Math.Sqrt((1.1486502132323444 / (299)) + (0.06512927818055393 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.054934954231993696 = (11.273852433219036 - 10.654523865781897) / 11.273852433219036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Multiple)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.98769411223676 < 49.567034982380235.
IsChangePoint: Marked as a change because one of 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 12.073880668724026 (T) = (0 -44.20248253432354) / Math.Sqrt((18.629107726614873 / (299)) + (0.2092976384993568 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.0663422547655363 = (47.34334691694037 - 44.20248253432354) / 47.34334691694037 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySequence.Slice_StartPosition(Segment: Multiple)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.085826238122166 < 10.233667286212563.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 8.967535925010301 (T) = (0 -9.322904032308966) / Math.Sqrt((0.9543112230100996 / (299)) + (0.1718964851402751 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.0773639296973887 = (10.10463858111594 - 9.322904032308966) / 10.10463858111594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9534947f2918c4721e4b1dabfa9402057c6e1246
Compare 11bfa9b6fac2a6d50f09c3c8057bbccee9908383
Diff Diff

Improvements in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 771.76 μs 712.96 μs 0.92 0.02 False
Span - Duration of single invocation 701.94 μs 654.08 μs 0.93 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.Stack(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 712.9646075174825 < 732.0537375291419.
IsChangePoint: Marked as a change because one of 5/9/2022 5:23:46 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 45.01590531986784 (T) = (0 -719463.3639852717) / Math.Sqrt((24935116.682499558 / (299)) + (47070755.05738 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06635743685552348 = (770598.2914512203 - 719463.3639852717) / 770598.2914512203 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 654.078984375 < 666.8634452219203.
IsChangePoint: Marked as a change because one of 5/9/2022 5:23:46 PM, 7/6/2022 12:32:56 PM, 7/7/2022 8:07:51 AM, 7/10/2022 1:04:01 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 82.12501126335444 (T) = (0 -654298.7960412852) / Math.Sqrt((111782469.40797868 / (299)) + (191174.78450194743 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07256263538932001 = (705491.088679554 - 654298.7960412852) / 705491.088679554 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9534947f2918c4721e4b1dabfa9402057c6e1246
Compare 11bfa9b6fac2a6d50f09c3c8057bbccee9908383
Diff Diff

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateTryGetTenSegments - Duration of single invocation 47.57 ns 37.29 ns 0.78 0.03 False
FirstSpanArray - Duration of single invocation 21.38 ns 4.02 ns 0.19 0.01 False
FirstSpanMemory - Duration of single invocation 22.08 ns 4.72 ns 0.21 0.06 False
FirstSpanSingleSegment - Duration of single invocation 23.45 ns 4.33 ns 0.18 0.02 False
IterateGetPositionTenSegments - Duration of single invocation 61.06 ns 52.16 ns 0.85 0.16 False
FirstSpanTenSegments - Duration of single invocation 17.72 ns 3.96 ns 0.22 0.02 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetTenSegments


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.28752837533343 < 45.13565955359117.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 6.456006996437721 (T) = (0 -37.670964531423245) / Math.Sqrt((44.62432733987978 / (299)) + (0.2386713141214268 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06327688766560523 = (40.21568811037869 - 37.670964531423245) / 40.21568811037869 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;.FirstSpanArray

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.0211744500890525 < 12.110479312310813.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 4.506504181290078 (T) = (0 -4.028518773795401) / Math.Sqrt((15.58907037526811 / (299)) + (0.00016196350869950214 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.20346504193801987 = (5.057554264281183 - 4.028518773795401) / 5.057554264281183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;.FirstSpanMemory

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.723696401092939 < 12.785651697441923.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 4.750677751947787 (T) = (0 -4.719691582840211) / Math.Sqrt((15.268488013293709 / (299)) + (0.002182288168434619 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.18538605115688195 = (5.79377702719543 - 4.719691582840211) / 5.79377702719543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;.FirstSpanSingleSegment

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.329930715408517 < 10.680641833245263.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 4.797603551000855 (T) = (0 -4.365715810966702) / Math.Sqrt((9.158885276543542 / (299)) + (0.01054246269347359 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.1618611870727967 = (5.208821908293951 - 4.365715810966702) / 5.208821908293951 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;.IterateGetPositionTenSegments

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.15516701397933 < 57.83479757780258.
IsChangePoint: Marked as a change because one of 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 6.460983363135021 (T) = (0 -53.98717587953612) / Math.Sqrt((11.885179401729124 / (299)) + (6.238065123979034 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.05076572820772709 = (56.87444868335989 - 53.98717587953612) / 56.87444868335989 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;.FirstSpanTenSegments

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.9646123377071527 < 10.487148991837337.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 5.3155518222340215 (T) = (0 -3.993765497848496) / Math.Sqrt((8.821648867682624 / (299)) + (0.008503671059818826 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.18659394838628068 = (4.909928429872448 - 3.993765497848496) / 4.909928429872448 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9534947f2918c4721e4b1dabfa9402057c6e1246
Compare 11bfa9b6fac2a6d50f09c3c8057bbccee9908383
Diff Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBytes - Duration of single invocation 86.84 ns 75.61 ns 0.87 0.20 False
GetBytes - Duration of single invocation 62.88 ns 53.64 ns 0.85 0.20 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 75.61092476091405 < 83.73585615615234.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/30/2022 12:24:00 AM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 10.918664756238378 (T) = (0 -75.79356971236683) / Math.Sqrt((41.45518554149039 / (299)) + (1.3115714611469165 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.05641943429183503 = (80.3254883227519 - 75.79356971236683) / 80.3254883227519 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.64080790930516 < 59.81162860437861.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/30/2022 12:24:00 AM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 14.73566300273919 (T) = (0 -53.15099139644068) / Math.Sqrt((27.40228244755266 / (299)) + (1.38031195831952 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.08992945810865859 = (58.40315552460403 - 53.15099139644068) / 58.40315552460403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9534947f2918c4721e4b1dabfa9402057c6e1246
Compare 11bfa9b6fac2a6d50f09c3c8057bbccee9908383
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleSerial - Duration of single invocation 400.54 ns 364.00 ns 0.91 0.09 False
SingleSerial - Duration of single invocation 402.90 ns 369.40 ns 0.92 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 364.0000363333333 < 381.5155381666667.
IsChangePoint: Marked as a change because one of 6/9/2022 8:09:33 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 14.209724526363214 (T) = (0 -363.33495390089035) / Math.Sqrt((442.1831769227245 / (299)) + (59.18543055296298 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06340639827764075 = (387.9323467859822 - 363.33495390089035) / 387.9323467859822 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 369.39764600000007 < 385.3226867055556.
IsChangePoint: Marked as a change because one of 6/9/2022 8:09:33 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 17.621380214510793 (T) = (0 -370.6682031287842) / Math.Sqrt((325.4210207206278 / (299)) + (15.484404157892575 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.05476656054930557 = (392.1446149262255 - 370.6682031287842) / 392.1446149262255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9534947f2918c4721e4b1dabfa9402057c6e1246
Compare 11bfa9b6fac2a6d50f09c3c8057bbccee9908383
Diff Diff

Improvements in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsciiCharArray - Duration of single invocation 503.91 ns 330.61 ns 0.66 0.05 False
WriteAsciiString - Duration of single invocation 510.83 ns 341.73 ns 0.67 0.06 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 330.6084216169165 < 468.6155509110307.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/30/2022 12:24:00 AM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 16.219090745405545 (T) = (0 -323.06653921130726) / Math.Sqrt((7222.378701508694 / (299)) + (34.06836994758239 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.2007427664260866 = (404.20846460994943 - 323.06653921130726) / 404.20846460994943 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 341.73327005741896 < 473.12778072319463.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/30/2022 12:24:00 AM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 15.43810353167217 (T) = (0 -335.1713631608047) / Math.Sqrt((7431.694602816673 / (299)) + (40.16465864537622 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.1898521977743595 = (413.7163147761691 - 335.1713631608047) / 413.7163147761691 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions