Skip to content

Regressions in Microsoft.Extensions.DependencyInjection.ScopeValidation #89579

Closed
@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 2ee61bbe7bb04e97159f017c4f9eaf486dc46c1f
Compare a4a8e8e712eff30c9223ebdeeeb4c80fb798e972
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_RandomAccess

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.42 ms 5.89 ms 1.09 0.18 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_RandomAccess*'

Payloads

Baseline
Compare

System.IO.Tests.Perf_RandomAccess.Write(fileSize: 1048576, bufferSize: 4096, options: Asynchronous)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/23/2023 8:43:31 PM, 7/20/2023 8:15:02 AM, 7/26/2023 10:32:59 PM falls between 7/18/2023 9:59:03 AM and 7/26/2023 10:32:59 PM.
IsRegressionStdDev: Marked as regression because -5.966407543366567 (T) = (0 -5839819.7549007805) / Math.Sqrt((1717212977.202498 / (6)) + (75014545037.80371 / (15))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (15) - 2, .025) and -0.08025088202255358 = (5405984.713446275 - 5839819.7549007805) / 5405984.713446275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 2ee61bbe7bb04e97159f017c4f9eaf486dc46c1f
Compare a4a8e8e712eff30c9223ebdeeeb4c80fb798e972
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.ScopeValidation

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
24.44 ns 49.53 ns 2.03 0.29 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ScopeValidation*'

Payloads

Baseline
Compare

Microsoft.Extensions.DependencyInjection.ScopeValidation.TransientWithScopeValidation

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/20/2023 8:15:02 AM, 7/26/2023 10:32:59 PM falls between 7/18/2023 9:59:03 AM and 7/26/2023 10:32:59 PM.
IsRegressionStdDev: Marked as regression because -54.75286757222362 (T) = (0 -49.38343542733087) / Math.Sqrt((0.5241935187568165 / (6)) + (1.527196223135816 / (14))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (14) - 2, .025) and -0.9662585539660059 = (25.115433231160225 - 49.38343542733087) / 25.115433231160225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions