Closed
Description
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | 2ee61bbe7bb04e97159f017c4f9eaf486dc46c1f |
Compare | a4a8e8e712eff30c9223ebdeeeb4c80fb798e972 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.IO.Tests.Perf_RandomAccess
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
5.42 ms | 5.89 ms | 1.09 | 0.18 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_RandomAccess*'
Payloads
System.IO.Tests.Perf_RandomAccess.Write(fileSize: 1048576, bufferSize: 4096, options: Asynchronous)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/23/2023 8:43:31 PM, 7/20/2023 8:15:02 AM, 7/26/2023 10:32:59 PM falls between 7/18/2023 9:59:03 AM and 7/26/2023 10:32:59 PM.
IsRegressionStdDev: Marked as regression because -5.966407543366567 (T) = (0 -5839819.7549007805) / Math.Sqrt((1717212977.202498 / (6)) + (75014545037.80371 / (15))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (15) - 2, .025) and -0.08025088202255358 = (5405984.713446275 - 5839819.7549007805) / 5405984.713446275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | 2ee61bbe7bb04e97159f017c4f9eaf486dc46c1f |
Compare | a4a8e8e712eff30c9223ebdeeeb4c80fb798e972 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in Microsoft.Extensions.DependencyInjection.ScopeValidation
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
24.44 ns | 49.53 ns | 2.03 | 0.29 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ScopeValidation*'
Payloads
Microsoft.Extensions.DependencyInjection.ScopeValidation.TransientWithScopeValidation
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/20/2023 8:15:02 AM, 7/26/2023 10:32:59 PM falls between 7/18/2023 9:59:03 AM and 7/26/2023 10:32:59 PM.
IsRegressionStdDev: Marked as regression because -54.75286757222362 (T) = (0 -49.38343542733087) / Math.Sqrt((0.5241935187568165 / (6)) + (1.527196223135816 / (14))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (14) - 2, .025) and -0.9662585539660059 = (25.115433231160225 - 49.38343542733087) / 25.115433231160225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository