Closed
Description
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | e5efd8010e19593298dc2c3ee15106d5aec5a924 |
Compare | d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
751.29 ns | 937.49 ns | 1.25 | 0.28 | True | ||||
186.37 ns | 231.52 ns | 1.24 | 0.19 | True | ||||
841.88 ns | 1.10 μs | 1.30 | 0.26 | True | ||||
521.40 ns | 638.60 ns | 1.22 | 0.16 | True | ||||
803.59 ns | 1.05 μs | 1.31 | 0.19 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'
Payloads
Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_2
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 937.4885304137601 > 782.6306844345268.
IsChangePoint: Marked as a change because one of 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.57533020886312 (T) = (0 -913.3093549720485) / Math.Sqrt((794.3142520150316 / (17)) + (883.4239470863048 / (10))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (10) - 2, .025) and -0.2088178416281168 = (755.5392744219777 - 913.3093549720485) / 755.5392744219777 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_0
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 231.51951972196002 > 190.29670154175133.
IsChangePoint: Marked as a change because one of 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.84070934412893 (T) = (0 -228.9814988320599) / Math.Sqrt((68.78035798504725 / (17)) + (32.494263057636125 / (10))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (10) - 2, .025) and -0.24788981911731445 = (183.4949651196195 - 228.9814988320599) / 183.4949651196195 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_5
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0976442408928198 > 900.2410247684038.
IsChangePoint: Marked as a change because one of 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.754210690411856 (T) = (0 -1030.1455812063896) / Math.Sqrt((444.1185305935255 / (17)) + (1560.1326389021433 / (10))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (10) - 2, .025) and -0.20061249787407653 = (858.0167064981144 - 1030.1455812063896) / 858.0167064981144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_1
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 638.5950079089644 > 558.4607140027446.
IsChangePoint: Marked as a change because one of 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.006487530742078 (T) = (0 -649.7809004478281) / Math.Sqrt((183.92760854923986 / (17)) + (237.03435416261723 / (10))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (10) - 2, .025) and -0.22086317962599875 = (532.2307292835902 - 649.7809004478281) / 532.2307292835902 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_4
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0514375987647184 > 852.065324303373.
IsChangePoint: Marked as a change because one of 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.801726343492371 (T) = (0 -972.413101311331) / Math.Sqrt((768.8120970299191 / (17)) + (1080.7576179650252 / (10))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (10) - 2, .025) and -0.1768413106650319 = (826.2907602740605 - 972.413101311331) / 826.2907602740605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | e5efd8010e19593298dc2c3ee15106d5aec5a924 |
Compare | d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Memory.ReadOnlySequence
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
4.63 ns | 6.34 ns | 1.37 | 0.21 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySequence*'
Payloads
System.Memory.ReadOnlySequence.Slice_Start_And_EndPosition(Segment: Single)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.340009845624023 > 6.0938046941127215.
IsChangePoint: Marked as a change because one of 4/6/2023 5:23:20 AM, 4/12/2023 2:59:13 PM, 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -5.696217547152051 (T) = (0 -6.337058393698055) / Math.Sqrt((0.13692199921224932 / (16)) + (0.0006062989790194301 / (10))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (10) - 2, .025) and -0.0910438764198155 = (5.808252564958864 - 6.337058393698055) / 5.808252564958864 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | e5efd8010e19593298dc2c3ee15106d5aec5a924 |
Compare | d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_Decimal
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
5.27 ns | 6.38 ns | 1.21 | 0.15 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'
Payloads
System.Tests.Perf_Decimal.Floor
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.375184436757004 > 5.850781937424999.
IsChangePoint: Marked as a change because one of 4/7/2023 2:04:00 PM, 4/12/2023 8:39:35 PM, 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.6641298336938 (T) = (0 -6.371208046394829) / Math.Sqrt((0.03859310189152029 / (16)) + (0.005617433296458507 / (10))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (10) - 2, .025) and -0.13244448655792648 = (5.6260665507411876 - 6.371208046394829) / 5.6260665507411876 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository