Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 8006e6a89bc02e410331e6323e3f6321b224b327 |
| Compare | e4163ea55ebb3673c29e1c2a850a6a790029d278 |
| Diff | Diff |
Regressions in BenchmarksGame.FannkuchRedux_5
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| RunBench - Duration of single invocation | 23.95 ms | 26.24 ms | 1.10 | 0.01 | True |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.FannkuchRedux_5*'Payloads
Histogram
BenchmarksGame.FannkuchRedux_5.RunBench(n: 10, expectedSum: 38)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.23831273571429 > 25.0471650415.
IsChangePoint: Marked as a change because one of 4/26/2022 12:44:27 PM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsRegressionStdDev: Marked as regression because -26.310636692824072 (T) = (0 -25737862.459520478) / Math.Sqrt((86587725068.70877 / (27)) + (81681730737.5921 / (36))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (36) - 2, .025) and -0.08183666563172187 = (23790894.94484017 - 25737862.459520478) / 23790894.94484017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository