Closed
Description
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | f58a7902a0b4fc9914494de820a7aae8045abd48 |
Compare | b8554beea57afb5d75c55c8ea19ec9965b8917fe |
Diff | Diff |
Regressions in ByteMark
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
BenchBitOps - Duration of single invocation | 987.31 ms | 1.06 secs | 1.08 | 0.12 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'ByteMark*'
Payloads
Histogram
ByteMark.BenchBitOps
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0647829177333334 > 989.7809939675001.
IsChangePoint: Marked as a change because one of 8/9/2022 8:37:54 AM, 8/11/2022 10:02:44 AM falls between 8/2/2022 9:32:02 PM and 8/11/2022 10:02:44 AM.
IsRegressionStdDev: Marked as regression because -9.590419168084054 (T) = (0 -1070207212.7276349) / Math.Sqrt((662883780414606.9 / (42)) + (1173360342952634 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.12894561950448133 = (947970561.4140848 - 1070207212.7276349) / 947970561.4140848 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | f58a7902a0b4fc9914494de820a7aae8045abd48 |
Compare | b8554beea57afb5d75c55c8ea19ec9965b8917fe |
Diff | Diff |
Regressions in System.Tests.Perf_GC<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
AllocateUninitializedArray - Duration of single invocation | 575.53 ns | 707.11 ns | 1.23 | 0.28 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_GC<Byte>*'
Payloads
Histogram
System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 10000, pinned: True)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 707.1082382724502 > 583.7455950254075.
IsChangePoint: Marked as a change because one of 6/24/2022 3:13:27 AM, 8/9/2022 8:37:54 AM, 8/11/2022 10:02:44 AM falls between 8/2/2022 9:32:02 PM and 8/11/2022 10:02:44 AM.
IsRegressionStdDev: Marked as regression because -10.45194810411 (T) = (0 -684.6831923943578) / Math.Sqrt((881.919006930377 / (42)) + (671.827120731933 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.185404420925391 = (577.5946000436348 - 684.6831923943578) / 577.5946000436348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository