Open
Description
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in System.Memory.Span<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IndexOfAnyFourValues - Duration of single invocation | 439.97 ns | 713.01 ns | 1.62 | 0.22 | False | |||||
BinarySearch - Duration of single invocation | 15.06 ns | 17.88 ns | 1.19 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span<Int32>*'
Related Issues
Regressions
- [Perf] Linux/x64: 44 Regressions on 10/20/2022 5:12:08 PM
- [Perf] Windows/x64: 3 Regressions on 10/20/2022 5:12:08 PM
- [Perf] Alpine/x64: 33 Regressions on 10/20/2022 5:12:08 PM
- [Perf] Windows/x86: 20 Regressions on 10/20/2022 5:12:08 PM
Improvements
- [Perf] Linux/x64: 13 Improvements on 10/20/2022 5:12:08 PM
- [Perf] Alpine/x64: 10 Improvements on 10/20/2022 5:12:08 PM
- [Perf] Windows/x86: 7 Improvements on 10/20/2022 5:12:08 PM
Payloads
Histogram
Edge Detector Info
System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 713.0098712561947 > 462.0428735342041.
IsChangePoint: Marked as a change because one of 10/6/2022 2:13:23 PM, 10/10/2022 3:36:43 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -22.603671485593864 (T) = (0 -677.1893953197551) / Math.Sqrt((1062.4183182118222 / (28)) + (1205.8514882587024 / (17))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (17) - 2, .025) and -0.5344131432675984 = (441.3344595560824 - 677.1893953197551) / 441.3344595560824 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Int32>.BinarySearch(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 17.87614736606446 > 15.888871239483413.
IsChangePoint: Marked as a change because one of 9/1/2022 8:04:48 PM, 9/3/2022 11:44:47 AM, 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -108.03613797539897 (T) = (0 -17.866921908955383) / Math.Sqrt((0.0018277091568930724 / (17)) + (0.014880323094427219 / (28))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (28) - 2, .025) and -0.18042267257043704 = (15.136037560214895 - 17.866921908955383) / 15.136037560214895 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in System.Collections.ContainsKeyTrue<Int32, Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SortedList - Duration of single invocation | 19.18 μs | 22.44 μs | 1.17 | 0.08 | False | |||||
SortedDictionary - Duration of single invocation | 50.06 μs | 53.37 μs | 1.07 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyTrue<Int32, Int32>*'
Related Issues
Regressions
- [Perf] Linux/x64: 44 Regressions on 10/20/2022 5:12:08 PM
- [Perf] Windows/x64: 3 Regressions on 10/20/2022 5:12:08 PM
- [Perf] Alpine/x64: 33 Regressions on 10/20/2022 5:12:08 PM
- [Perf] Windows/x86: 20 Regressions on 10/20/2022 5:12:08 PM
Improvements
- [Perf] Linux/x64: 13 Improvements on 10/20/2022 5:12:08 PM
- [Perf] Alpine/x64: 10 Improvements on 10/20/2022 5:12:08 PM
- [Perf] Windows/x86: 7 Improvements on 10/20/2022 5:12:08 PM
Payloads
Histogram
Edge Detector Info
System.Collections.ContainsKeyTrue<Int32, Int32>.SortedList(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 22.438892165492963 > 20.023574709208834.
IsChangePoint: Marked as a change because one of 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -14.970512874974974 (T) = (0 -22209.074415824947) / Math.Sqrt((475391.173428142 / (28)) + (248041.08098720506 / (17))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (17) - 2, .025) and -0.13606242092498128 = (19549.16737562918 - 22209.074415824947) / 19549.16737562918 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsKeyTrue<Int32, Int32>.SortedDictionary(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 53.365818350456635 > 52.53282514299701.
IsChangePoint: Marked as a change because one of 9/8/2022 1:37:01 AM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -21.064415828408524 (T) = (0 -53409.32513040828) / Math.Sqrt((473412.54519836785 / (28)) + (59075.5091285743 / (17))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (17) - 2, .025) and -0.059666643236459684 = (50402.0065850938 - 53409.32513040828) / 50402.0065850938 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in System.Text.Json.Document.Tests.Perf_EnumerateArray
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
EnumerateUsingIndexer - Duration of single invocation | 1.43 μs | 1.51 μs | 1.06 | 0.16 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'
Related Issues
Regressions
- [Perf] Linux/x64: 44 Regressions on 10/20/2022 5:12:08 PM
- [Perf] Windows/x64: 3 Regressions on 10/20/2022 5:12:08 PM
- [Perf] Alpine/x64: 33 Regressions on 10/20/2022 5:12:08 PM
- [Perf] Windows/x86: 20 Regressions on 10/20/2022 5:12:08 PM
Improvements
- [Perf] Linux/x64: 13 Improvements on 10/20/2022 5:12:08 PM
- [Perf] Alpine/x64: 10 Improvements on 10/20/2022 5:12:08 PM
- [Perf] Windows/x86: 7 Improvements on 10/20/2022 5:12:08 PM
Payloads
Histogram
Edge Detector Info
System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfNumbers)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.5141732654600297 > 1.5047829101324972.
IsChangePoint: Marked as a change because one of 8/16/2022 10:54:40 PM, 8/27/2022 1:31:13 PM, 9/9/2022 3:12:19 AM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -366.7528251007236 (T) = (0 -1514.3883990953564) / Math.Sqrt((0.6712288996611178 / (28)) + (0.43955708282761813 / (17))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (17) - 2, .025) and -0.05714952865543299 = (1432.5205262319666 - 1514.3883990953564) / 1432.5205262319666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository