Closed
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 7d5efbb9e10b6d8beb91c90cbdefd7360869cece |
| Compare | 0e24ea7c2a0436a8f2bf83e8f5981ec035518b99 |
| Diff | Diff |
Improvements in System.Tests.Perf_Random
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| NextDouble - Duration of single invocation | 9.80 ns | 5.25 ns | 0.54 | 0.07 | False | |||||
| NextSingle - Duration of single invocation | 9.74 ns | 5.39 ns | 0.55 | 0.05 | False | |||||
| Next_int_int - Duration of single invocation | 10.93 ns | 6.86 ns | 0.63 | 0.04 | False | |||||
| Next_long - Duration of single invocation | 48.21 ns | 36.10 ns | 0.75 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Random*'Related Issues
Regressions
- [Perf] Linux/arm64: 2 Regressions on 11/1/2022 1:15:18 PM
- [Perf] Windows/arm64: 5 Regressions on 11/1/2022 1:15:18 PM
- [Perf] Windows/arm64: 1 Regression on 11/1/2022 1:15:18 PM
Improvements
- [Perf] Windows/arm64: 2 Improvements on 11/1/2022 1:15:18 PM
- [Perf] Windows/arm64: 1 Improvement on 11/1/2022 1:15:18 PM
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_Random.NextDouble
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.245453529456549 < 9.410982425047393.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/2/2022 9:17:08 PM falls between 10/25/2022 8:54:59 AM and 11/2/2022 9:17:08 PM.
IsImprovementStdDev: Marked as improvement because 122.33535339617472 (T) = (0 -5.383446127084485) / Math.Sqrt((0.0679117371993256 / (299)) + (0.006549023862243264 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.452111670086412 = (9.825809080353205 - 5.383446127084485) / 9.825809080353205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.NextSingle
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.388147609913367 < 9.352229106485277.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/2/2022 9:17:08 PM falls between 10/25/2022 8:54:59 AM and 11/2/2022 9:17:08 PM.
IsImprovementStdDev: Marked as improvement because 92.72633249337508 (T) = (0 -5.361938222079984) / Math.Sqrt((0.03189901407676259 / (299)) + (0.013308975235443693 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.454693789240343 = (9.83289409928113 - 5.361938222079984) / 9.83289409928113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.Next_int_int
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.864649436048075 < 10.549012945893656.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/2/2022 9:17:08 PM falls between 10/25/2022 8:54:59 AM and 11/2/2022 9:17:08 PM.
IsImprovementStdDev: Marked as improvement because 87.81021259436436 (T) = (0 -6.767399038784227) / Math.Sqrt((0.07655222310064919 / (299)) + (0.011971172026461214 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.38105330602628984 = (10.933734850955796 - 6.767399038784227) / 10.933734850955796 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.Next_long
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.09742712213731 < 45.362677040084805.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/2/2022 9:17:08 PM falls between 10/25/2022 8:54:59 AM and 11/2/2022 9:17:08 PM.
IsImprovementStdDev: Marked as improvement because 216.518290087698 (T) = (0 -36.09064050366681) / Math.Sqrt((0.3689284038129063 / (299)) + (0.01095752390732764 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.24917675356067348 = (48.06809149133513 - 36.09064050366681) / 48.06809149133513 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 7d5efbb9e10b6d8beb91c90cbdefd7360869cece |
| Compare | 0e24ea7c2a0436a8f2bf83e8f5981ec035518b99 |
| Diff | Diff |
Improvements in IfStatements.IfStatements
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| SingleArray - Duration of single invocation | 78.18 μs | 35.51 μs | 0.45 | 0.00 | False | |||||
| Single - Duration of single invocation | 65.27 μs | 32.21 μs | 0.49 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'IfStatements.IfStatements*'Related Issues
Regressions
- [Perf] Linux/arm64: 2 Regressions on 11/1/2022 1:15:18 PM
- [Perf] Windows/arm64: 5 Regressions on 11/1/2022 1:15:18 PM
- [Perf] Windows/arm64: 1 Regression on 11/1/2022 1:15:18 PM
Improvements
- [Perf] Windows/arm64: 2 Improvements on 11/1/2022 1:15:18 PM
- [Perf] Windows/arm64: 1 Improvement on 11/1/2022 1:15:18 PM
Payloads
Histogram
Edge Detector Info
IfStatements.IfStatements.SingleArray
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.514625765306135 < 74.25949538124999.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/2/2022 9:17:08 PM falls between 10/25/2022 8:54:59 AM and 11/2/2022 9:17:08 PM.
IsImprovementStdDev: Marked as improvement because 460.4660240388723 (T) = (0 -35516.83361443767) / Math.Sqrt((1916274.9811927783 / (248)) + (298.0841567584043 / (6))) is greater than 1.9694223653661387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (248) + (6) - 2, .975) and 0.5334285622642002 = (76123.03442061406 - 35516.83361443767) / 76123.03442061406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### IfStatements.IfStatements.Single
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.20711107256145 < 61.99857373437501.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/2/2022 9:17:08 PM falls between 10/25/2022 8:54:59 AM and 11/2/2022 9:17:08 PM.
IsImprovementStdDev: Marked as improvement because 294.9968672527027 (T) = (0 -32326.30029687551) / Math.Sqrt((2169224.6288032285 / (248)) + (9878.846721978549 / (5))) is greater than 1.9694602272243544 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (248) + (5) - 2, .975) and 0.4858499737441178 = (62873.28337271611 - 32326.30029687551) / 62873.28337271611 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository