Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | c2469455eab98fdcea71997d3cc5004a992fc88f |
| Compare | 58614a0f6da787c9d6d914b99564f066d13f69ed |
| Diff | Diff |
Regressions in System.Numerics.Tests.Perf_Matrix4x4
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CreateFromScalars - Duration of single invocation | 11.07 ns | 13.67 ns | 1.24 | 0.31 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'Payloads
Histogram
System.Numerics.Tests.Perf_Matrix4x4.CreateFromScalars
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.674686313685328 > 11.226523444264867.
IsChangePoint: Marked as a change because one of 1/6/2023 6:58:01 PM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM.
IsRegressionStdDev: Marked as regression because -45.93808286476237 (T) = (0 -13.889578611786805) / Math.Sqrt((0.0970283309484572 / (34)) + (0.03245853242272149 / (14))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (14) - 2, .025) and -0.3120965503828176 = (10.585790053128619 - 13.889578611786805) / 10.585790053128619 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
