Closed
Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9534947f2918c4721e4b1dabfa9402057c6e1246 |
Compare | 11bfa9b6fac2a6d50f09c3c8057bbccee9908383 |
Diff | Diff |
Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IterateForEachTenSegments - Duration of single invocation | 52.63 ns | 47.85 ns | 0.91 | 0.02 | False | |||||
FirstSpanTenSegments - Duration of single invocation | 17.42 ns | 4.81 ns | 0.28 | 0.02 | False | |||||
FirstSpanSingleSegment - Duration of single invocation | 17.04 ns | 5.05 ns | 0.30 | 0.02 | False | |||||
FirstSpanArray - Duration of single invocation | 23.86 ns | 4.51 ns | 0.19 | 0.03 | False | |||||
IterateTryGetTenSegments - Duration of single invocation | 56.50 ns | 43.74 ns | 0.77 | 0.03 | False | |||||
FirstSpanMemory - Duration of single invocation | 24.09 ns | 5.29 ns | 0.22 | 0.09 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Char>*'
Payloads
Histogram
System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachTenSegments
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.85022714647945 < 49.88965038934605.
IsChangePoint: Marked as a change because one of 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 20.79528296387366 (T) = (0 -47.69524109815917) / Math.Sqrt((3.326649326782174 / (299)) + (0.40974911681378595 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06026257805344075 = (50.75379567130987 - 47.69524109815917) / 50.75379567130987 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanTenSegments
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.809525537268655 < 10.577835054204689.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 3.564871175321872 (T) = (0 -4.830072368133726) / Math.Sqrt((7.89347299398933 / (299)) + (0.00602970928504505 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.1073379774600983 = (5.410863514043831 - 4.830072368133726) / 5.410863514043831 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanSingleSegment
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.047865548257078 < 10.741854003143859.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 2.6199789960233977 (T) = (0 -5.10966107479831) / Math.Sqrt((8.34546507200745 / (299)) + (0.008957029757272772 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.07918125882368733 = (5.549041137315366 - 5.10966107479831) / 5.549041137315366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.50739577862188 < 13.46105435368254.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 4.899783241451749 (T) = (0 -4.487798583377821) / Math.Sqrt((19.540133638216897 / (299)) + (0.0004936817752485261 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.2182202903222841 = (5.740489971564867 - 4.487798583377821) / 5.740489971564867 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateTryGetTenSegments
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.74208268747402 < 53.684787670237796.
IsChangePoint: Marked as a change because one of 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 9.018030282212488 (T) = (0 -44.17334117461784) / Math.Sqrt((51.43181498329463 / (299)) + (0.8164095089120201 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.08229488806753751 = (48.13457024511892 - 44.17334117461784) / 48.13457024511892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanMemory
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.289433883544825 < 14.036889124813888.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 4.510274785990649 (T) = (0 -5.2854095505170635) / Math.Sqrt((18.349989049347837 / (299)) + (0.0006998500780451758 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.1745294230851334 = (6.402904837954224 - 5.2854095505170635) / 6.402904837954224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9534947f2918c4721e4b1dabfa9402057c6e1246 |
Compare | 11bfa9b6fac2a6d50f09c3c8057bbccee9908383 |
Diff | Diff |
Improvements in System.Memory.ReadOnlySequence
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Slice_StartPosition_And_EndPosition - Duration of single invocation | 12.25 ns | 10.88 ns | 0.89 | 0.08 | False | |||||
Slice_Repeat_StartPosition_And_EndPosition - Duration of single invocation | 52.24 ns | 43.99 ns | 0.84 | 0.07 | False | |||||
Slice_StartPosition - Duration of single invocation | 10.76 ns | 9.09 ns | 0.84 | 0.14 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySequence*'
Payloads
Histogram
System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment: Multiple)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.88318230268836 < 11.571529122072073.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 8.342216177242427 (T) = (0 -10.654523865781897) / Math.Sqrt((1.1486502132323444 / (299)) + (0.06512927818055393 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.054934954231993696 = (11.273852433219036 - 10.654523865781897) / 11.273852433219036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Multiple)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.98769411223676 < 49.567034982380235.
IsChangePoint: Marked as a change because one of 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 12.073880668724026 (T) = (0 -44.20248253432354) / Math.Sqrt((18.629107726614873 / (299)) + (0.2092976384993568 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.0663422547655363 = (47.34334691694037 - 44.20248253432354) / 47.34334691694037 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySequence.Slice_StartPosition(Segment: Multiple)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.085826238122166 < 10.233667286212563.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 8.967535925010301 (T) = (0 -9.322904032308966) / Math.Sqrt((0.9543112230100996 / (299)) + (0.1718964851402751 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.0773639296973887 = (10.10463858111594 - 9.322904032308966) / 10.10463858111594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9534947f2918c4721e4b1dabfa9402057c6e1246 |
Compare | 11bfa9b6fac2a6d50f09c3c8057bbccee9908383 |
Diff | Diff |
Improvements in System.Collections.ContainsFalse<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Stack - Duration of single invocation | 771.76 μs | 712.96 μs | 0.92 | 0.02 | False | |||||
Span - Duration of single invocation | 701.94 μs | 654.08 μs | 0.93 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse<String>*'
Payloads
Histogram
System.Collections.ContainsFalse<String>.Stack(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 712.9646075174825 < 732.0537375291419.
IsChangePoint: Marked as a change because one of 5/9/2022 5:23:46 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 45.01590531986784 (T) = (0 -719463.3639852717) / Math.Sqrt((24935116.682499558 / (299)) + (47070755.05738 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06635743685552348 = (770598.2914512203 - 719463.3639852717) / 770598.2914512203 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsFalse<String>.Span(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 654.078984375 < 666.8634452219203.
IsChangePoint: Marked as a change because one of 5/9/2022 5:23:46 PM, 7/6/2022 12:32:56 PM, 7/7/2022 8:07:51 AM, 7/10/2022 1:04:01 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 82.12501126335444 (T) = (0 -654298.7960412852) / Math.Sqrt((111782469.40797868 / (299)) + (191174.78450194743 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07256263538932001 = (705491.088679554 - 654298.7960412852) / 705491.088679554 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9534947f2918c4721e4b1dabfa9402057c6e1246 |
Compare | 11bfa9b6fac2a6d50f09c3c8057bbccee9908383 |
Diff | Diff |
Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IterateTryGetTenSegments - Duration of single invocation | 47.57 ns | 37.29 ns | 0.78 | 0.03 | False | |||||
FirstSpanArray - Duration of single invocation | 21.38 ns | 4.02 ns | 0.19 | 0.01 | False | |||||
FirstSpanMemory - Duration of single invocation | 22.08 ns | 4.72 ns | 0.21 | 0.06 | False | |||||
FirstSpanSingleSegment - Duration of single invocation | 23.45 ns | 4.33 ns | 0.18 | 0.02 | False | |||||
IterateGetPositionTenSegments - Duration of single invocation | 61.06 ns | 52.16 ns | 0.85 | 0.16 | False | |||||
FirstSpanTenSegments - Duration of single invocation | 17.72 ns | 3.96 ns | 0.22 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Byte>*'
Payloads
Histogram
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetTenSegments
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.28752837533343 < 45.13565955359117.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 6.456006996437721 (T) = (0 -37.670964531423245) / Math.Sqrt((44.62432733987978 / (299)) + (0.2386713141214268 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06327688766560523 = (40.21568811037869 - 37.670964531423245) / 40.21568811037869 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.0211744500890525 < 12.110479312310813.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 4.506504181290078 (T) = (0 -4.028518773795401) / Math.Sqrt((15.58907037526811 / (299)) + (0.00016196350869950214 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.20346504193801987 = (5.057554264281183 - 4.028518773795401) / 5.057554264281183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanMemory
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.723696401092939 < 12.785651697441923.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 4.750677751947787 (T) = (0 -4.719691582840211) / Math.Sqrt((15.268488013293709 / (299)) + (0.002182288168434619 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.18538605115688195 = (5.79377702719543 - 4.719691582840211) / 5.79377702719543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanSingleSegment
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.329930715408517 < 10.680641833245263.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 4.797603551000855 (T) = (0 -4.365715810966702) / Math.Sqrt((9.158885276543542 / (299)) + (0.01054246269347359 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.1618611870727967 = (5.208821908293951 - 4.365715810966702) / 5.208821908293951 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionTenSegments
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.15516701397933 < 57.83479757780258.
IsChangePoint: Marked as a change because one of 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 6.460983363135021 (T) = (0 -53.98717587953612) / Math.Sqrt((11.885179401729124 / (299)) + (6.238065123979034 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.05076572820772709 = (56.87444868335989 - 53.98717587953612) / 56.87444868335989 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanTenSegments
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.9646123377071527 < 10.487148991837337.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 5.3155518222340215 (T) = (0 -3.993765497848496) / Math.Sqrt((8.821648867682624 / (299)) + (0.008503671059818826 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.18659394838628068 = (4.909928429872448 - 3.993765497848496) / 4.909928429872448 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9534947f2918c4721e4b1dabfa9402057c6e1246 |
Compare | 11bfa9b6fac2a6d50f09c3c8057bbccee9908383 |
Diff | Diff |
Improvements in System.Text.Tests.Perf_Encoding
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetBytes - Duration of single invocation | 86.84 ns | 75.61 ns | 0.87 | 0.20 | False | |||||
GetBytes - Duration of single invocation | 62.88 ns | 53.64 ns | 0.85 | 0.20 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'
Payloads
Histogram
System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 75.61092476091405 < 83.73585615615234.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/30/2022 12:24:00 AM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 10.918664756238378 (T) = (0 -75.79356971236683) / Math.Sqrt((41.45518554149039 / (299)) + (1.3115714611469165 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.05641943429183503 = (80.3254883227519 - 75.79356971236683) / 80.3254883227519 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "ascii")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.64080790930516 < 59.81162860437861.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/30/2022 12:24:00 AM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 14.73566300273919 (T) = (0 -53.15099139644068) / Math.Sqrt((27.40228244755266 / (299)) + (1.38031195831952 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.08992945810865859 = (58.40315552460403 - 53.15099139644068) / 58.40315552460403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9534947f2918c4721e4b1dabfa9402057c6e1246 |
Compare | 11bfa9b6fac2a6d50f09c3c8057bbccee9908383 |
Diff | Diff |
Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SingleSerial - Duration of single invocation | 400.54 ns | 364.00 ns | 0.91 | 0.09 | False | |||||
SingleSerial - Duration of single invocation | 402.90 ns | 369.40 ns | 0.92 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Byte>*'
Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 364.0000363333333 < 381.5155381666667.
IsChangePoint: Marked as a change because one of 6/9/2022 8:09:33 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 14.209724526363214 (T) = (0 -363.33495390089035) / Math.Sqrt((442.1831769227245 / (299)) + (59.18543055296298 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06340639827764075 = (387.9323467859822 - 363.33495390089035) / 387.9323467859822 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 369.39764600000007 < 385.3226867055556.
IsChangePoint: Marked as a change because one of 6/9/2022 8:09:33 PM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 17.621380214510793 (T) = (0 -370.6682031287842) / Math.Sqrt((325.4210207206278 / (299)) + (15.484404157892575 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.05476656054930557 = (392.1446149262255 - 370.6682031287842) / 392.1446149262255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9534947f2918c4721e4b1dabfa9402057c6e1246 |
Compare | 11bfa9b6fac2a6d50f09c3c8057bbccee9908383 |
Diff | Diff |
Improvements in System.IO.Tests.BinaryWriterExtendedTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
WriteAsciiCharArray - Duration of single invocation | 503.91 ns | 330.61 ns | 0.66 | 0.05 | False | |||||
WriteAsciiString - Duration of single invocation | 510.83 ns | 341.73 ns | 0.67 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'
Payloads
Histogram
System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 330.6084216169165 < 468.6155509110307.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/30/2022 12:24:00 AM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 16.219090745405545 (T) = (0 -323.06653921130726) / Math.Sqrt((7222.378701508694 / (299)) + (34.06836994758239 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.2007427664260866 = (404.20846460994943 - 323.06653921130726) / 404.20846460994943 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 341.73327005741896 < 473.12778072319463.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/30/2022 12:24:00 AM, 7/6/2022 12:32:56 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 15.43810353167217 (T) = (0 -335.1713631608047) / Math.Sqrt((7431.694602816673 / (299)) + (40.16465864537622 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.1898521977743595 = (413.7163147761691 - 335.1713631608047) / 413.7163147761691 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository