Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.12 |
| Baseline | dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053 |
| Compare | 285b8259af30e255980b1ebfb2ddd60ca291a83a |
| Diff | Diff |
Regressions in System.Linq.Tests.Perf_Enumerable
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Min - Duration of single invocation | 344.72 ns | 394.62 ns | 1.14 | 0.15 | False | |||||
| Select - Duration of single invocation | 516.14 ns | 543.85 ns | 1.05 | 0.06 | False | |||||
| WhereLast_LastElementMatches - Duration of single invocation | 189.00 ns | 211.84 ns | 1.12 | 0.16 | False | |||||
| ToArray - Duration of single invocation | 538.21 ns | 587.94 ns | 1.09 | 0.15 | False | |||||
| Max - Duration of single invocation | 326.12 ns | 345.08 ns | 1.06 | 0.18 | False | |||||
| WhereSingleOrDefault_LastElementMatches - Duration of single invocation | 188.09 ns | 208.33 ns | 1.11 | 0.17 | False | |||||
| WhereSingle_LastElementMatches - Duration of single invocation | 241.01 ns | 275.88 ns | 1.14 | 0.09 | False | |||||
| WhereSingle_LastElementMatches - Duration of single invocation | 187.60 ns | 209.41 ns | 1.12 | 0.16 | False | |||||
| Concat_Once - Duration of single invocation | 1.39 μs | 1.50 μs | 1.08 | 0.10 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'Payloads
Histogram
System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 394.61806294713875 > 362.05982889539916.
IsChangePoint: Marked as a change because one of 2/2/2022 4:36:37 PM, 2/17/2022 3:48:53 PM, 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -99.13838721792497 (T) = (0 -388.6858653503989) / Math.Sqrt((0.06605368871080244 / (41)) + (2.8981968010071477 / (15))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (15) - 2, .025) and -0.1268632234122872 = (344.92727890560485 - 388.6858653503989) / 344.92727890560485 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Select(input: Array)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 543.8486089348033 > 541.8632477060415.
IsChangePoint: Marked as a change because one of 2/2/2022 4:36:37 PM, 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -91.33155035845984 (T) = (0 -544.4831203660419) / Math.Sqrt((1.3619256284845913 / (42)) + (0.9522971978068583 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.05479538769554284 = (516.1978585776695 - 544.4831203660419) / 516.1978585776695 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: Array)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 211.8397032049015 > 198.47407088024477.
IsChangePoint: Marked as a change because one of 2/12/2022 1:03:18 PM, 3/4/2022 1:10:06 AM, 3/7/2022 6:10:39 PM, 3/11/2022 8:42:43 PM, 3/21/2022 1:28:14 PM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -92.3480253125493 (T) = (0 -212.04095446475998) / Math.Sqrt((0.30497366979072915 / (42)) + (0.805795091292106 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.12050925633637162 = (189.2362363502924 - 212.04095446475998) / 189.2362363502924 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 587.9408925094091 > 564.7677348619952.
IsChangePoint: Marked as a change because one of 2/10/2022 10:02:40 PM, 2/17/2022 3:48:53 PM, 3/21/2022 1:28:14 PM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -51.82444215521497 (T) = (0 -588.2956558538465) / Math.Sqrt((9.63356249153452 / (42)) + (10.169338297592477 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.09159742758017234 = (538.930965748029 - 588.2956558538465) / 538.930965748029 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 345.07754686518666 > 342.0248002740456.
IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -139.34082000925326 (T) = (0 -345.197599340174) / Math.Sqrt((0.5755955163133378 / (42)) + (0.10429400322464001 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.06158955182995014 = (325.1704943262942 - 345.197599340174) / 325.1704943262942 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: Array)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 208.32933835851554 > 197.3437324453004.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/26/2022 10:27:55 AM, 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -101.23452382617847 (T) = (0 -209.62666586137888) / Math.Sqrt((0.7794940838382208 / (42)) + (0.45340920531385925 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.11940420136927643 = (187.26628469408956 - 209.62666586137888) / 187.26628469408956 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: List)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 275.8796801489393 > 252.88159557032284.
IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -53.3125925874796 (T) = (0 -283.40697332734703) / Math.Sqrt((1.2439723975903147 / (42)) + (9.212186628477783 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.17776261706336152 = (240.63165974311124 - 283.40697332734703) / 240.63165974311124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: Array)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 209.4082840855427 > 197.10058986883456.
IsChangePoint: Marked as a change because one of 2/1/2022 1:24:52 PM, 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -117.08418979343334 (T) = (0 -209.53530180583914) / Math.Sqrt((0.1663522930094863 / (42)) + (0.4657993240282166 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.11676841756592894 = (187.62645729410514 - 209.53530180583914) / 187.62645729410514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4960792177155142 > 1.454677141392647.
IsChangePoint: Marked as a change because one of 2/10/2022 10:02:40 PM, 3/21/2022 1:28:14 PM, 4/8/2022 2:40:20 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -18.7239373644994 (T) = (0 -1537.2144220094967) / Math.Sqrt((1224.5197593867324 / (41)) + (385.2264574045426 / (16))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (16) - 2, .025) and -0.09824972335700226 = (1399.6947955613575 - 1537.2144220094967) / 1399.6947955613575 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | alpine 3.12 |
| Baseline | dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053 |
| Compare | 285b8259af30e255980b1ebfb2ddd60ca291a83a |
| Diff | Diff |
Regressions in System.Memory.Span<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| LastIndexOfAnyValues - Duration of single invocation | 9.68 ns | 14.16 ns | 1.46 | 0.54 | False | |||||
| BinarySearch - Duration of single invocation | 9.03 ns | 10.39 ns | 1.15 | 0.14 | False |
_1.png)
_2.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span<Byte>*'Payloads
Histogram
System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.159845620731966 > 11.656600717395374.
IsChangePoint: Marked as a change because one of 3/16/2022 3:05:50 AM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -19.357725869138704 (T) = (0 -14.196068057031201) / Math.Sqrt((1.1733959313355158 / (42)) + (0.016550021977281348 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.3027240891349547 = (10.897217741983868 - 14.196068057031201) / 10.897217741983868 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Byte>.BinarySearch(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.393370663094947 > 9.473804526883557.
IsChangePoint: Marked as a change because one of 1/31/2022 7:28:13 PM, 2/1/2022 2:18:14 AM, 3/17/2022 6:54:53 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -19.952848366603714 (T) = (0 -10.240215896289447) / Math.Sqrt((0.00018233033663987982 / (35)) + (0.08278754013758374 / (22))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (22) - 2, .025) and -0.13586019021992402 = (9.015384097849884 - 10.240215896289447) / 9.015384097849884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.12 |
| Baseline | dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053 |
| Compare | 285b8259af30e255980b1ebfb2ddd60ca291a83a |
| Diff | Diff |
Regressions in System.Collections.Tests.Perf_BitArray
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| BitArrayNot - Duration of single invocation | 7.85 ns | 10.85 ns | 1.38 | 0.23 | False | |||||
| BitArrayOr - Duration of single invocation | 8.83 ns | 11.73 ns | 1.33 | 0.61 | False | |||||
| BitArrayAnd - Duration of single invocation | 8.74 ns | 11.74 ns | 1.34 | 0.59 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'Payloads
Histogram
System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.845571805248149 > 7.709763432514391.
IsChangePoint: Marked as a change because one of 2/2/2022 7:54:56 PM, 2/15/2022 2:06:03 PM, 2/23/2022 4:28:51 PM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -13.674706395032752 (T) = (0 -10.67244949209505) / Math.Sqrt((0.02248767927592186 / (42)) + (0.8856408150250037 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.45507466613645986 = (7.3346404418082045 - 10.67244949209505) / 7.3346404418082045 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.728589380002509 > 9.498874987226355.
IsChangePoint: Marked as a change because one of 2/12/2022 6:41:57 AM, 2/22/2022 5:41:06 PM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -5.717326926706129 (T) = (0 -12.748922818187394) / Math.Sqrt((0.23028461872526312 / (42)) + (6.360176386981104 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.41622845710003387 = (9.00202418209627 - 12.748922818187394) / 9.00202418209627 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.741024619579484 > 9.77697683189218.
IsChangePoint: Marked as a change because one of 2/15/2022 2:06:03 PM, 2/21/2022 10:29:02 AM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -11.387721713527105 (T) = (0 -11.539000714157897) / Math.Sqrt((0.18397043435650473 / (42)) + (0.6424268018298204 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.27295633754901966 = (9.06472623905967 - 11.539000714157897) / 9.06472623905967 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.12 |
| Baseline | dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053 |
| Compare | 285b8259af30e255980b1ebfb2ddd60ca291a83a |
| Diff | Diff |
Regressions in System.Memory.Span<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Fill - Duration of single invocation | 21.22 ns | 28.77 ns | 1.36 | 0.65 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span<Int32>*'Payloads
Histogram
System.Memory.Span<Int32>.Fill(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.774092619192036 > 23.95949360303189.
IsChangePoint: Marked as a change because one of 2/22/2022 2:43:53 PM, 4/8/2022 7:02:49 PM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsRegressionStdDev: Marked as regression because -11.485891512345141 (T) = (0 -28.970127171976586) / Math.Sqrt((7.816805707043072 / (42)) + (3.0514221177095733 / (15))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (15) - 2, .025) and -0.3288196960209214 = (21.801398081866232 - 28.970127171976586) / 21.801398081866232 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository