Skip to content

Regression from "Mark phase prefetching" #73782

Closed
@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline f58a7902a0b4fc9914494de820a7aae8045abd48
Compare b8554beea57afb5d75c55c8ea19ec9965b8917fe
Diff Diff

Regressions in ByteMark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BenchBitOps - Duration of single invocation 987.31 ms 1.06 secs 1.08 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

Histogram

ByteMark.BenchBitOps


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0647829177333334 > 989.7809939675001.
IsChangePoint: Marked as a change because one of 8/9/2022 8:37:54 AM, 8/11/2022 10:02:44 AM falls between 8/2/2022 9:32:02 PM and 8/11/2022 10:02:44 AM.
IsRegressionStdDev: Marked as regression because -9.590419168084054 (T) = (0 -1070207212.7276349) / Math.Sqrt((662883780414606.9 / (42)) + (1173360342952634 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.12894561950448133 = (947970561.4140848 - 1070207212.7276349) / 947970561.4140848 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline f58a7902a0b4fc9914494de820a7aae8045abd48
Compare b8554beea57afb5d75c55c8ea19ec9965b8917fe
Diff Diff

Regressions in System.Tests.Perf_GC<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AllocateUninitializedArray - Duration of single invocation 575.53 ns 707.11 ns 1.23 0.28 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_GC&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 10000, pinned: True)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 707.1082382724502 > 583.7455950254075.
IsChangePoint: Marked as a change because one of 6/24/2022 3:13:27 AM, 8/9/2022 8:37:54 AM, 8/11/2022 10:02:44 AM falls between 8/2/2022 9:32:02 PM and 8/11/2022 10:02:44 AM.
IsRegressionStdDev: Marked as regression because -10.45194810411 (T) = (0 -684.6831923943578) / Math.Sqrt((881.919006930377 / (42)) + (671.827120731933 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.185404420925391 = (577.5946000436348 - 684.6831923943578) / 577.5946000436348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions