Open
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | d1fc57ea18ee90aee8690697caed2b9f162409eb |
| Compare | 7ebf9676b4e37c183072f97efaa21e6b4a7d33f3 |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 514.09 ns | 466.97 ns | 0.91 | 0.33 | True | ||||
| 490.02 ns | 438.35 ns | 0.89 | 0.28 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'Payloads
Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory_1Injected_2Explicit_OutOfOrder
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 466.96986583368505 < 482.89613928437456.
IsChangePoint: Marked as a change because one of 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 12.289872813124159 (T) = (0 -444.2632633601629) / Math.Sqrt((162.37198274731338 / (21)) + (170.16772573394073 / (17))) is greater than 2.0280940009779607 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (17) - 2, .975) and 0.1043612992709398 = (496.0295518701096 - 444.2632633601629) / 496.0295518701096 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory_1Injected_2Explicit
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 438.3531035851229 < 463.71396971290244.
IsChangePoint: Marked as a change because one of 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 11.537281080338497 (T) = (0 -441.88097045005384) / Math.Sqrt((310.8492112052483 / (20)) + (147.55891104756222 / (18))) is greater than 2.0280940009779607 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .975) and 0.11285821880346121 = (498.0950957512831 - 441.88097045005384) / 498.0950957512831 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | d1fc57ea18ee90aee8690697caed2b9f162409eb |
| Compare | 7ebf9676b4e37c183072f97efaa21e6b4a7d33f3 |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Improvements in System.Reflection.Invoke
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 2.80 μs | 2.36 μs | 0.84 | 0.39 | False | ||||
| 244.71 ns | 190.34 ns | 0.78 | 0.21 | False | ||||
| 245.68 ns | 202.22 ns | 0.82 | 0.25 | False | ||||
| 286.13 ns | 240.70 ns | 0.84 | 0.22 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'Payloads
System.Reflection.Invoke.StaticMethod5_arrayNotCached_int_string_struct_class_bool
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.358166071428572 < 2.4981590875049604.
IsChangePoint: Marked as a change because one of 7/14/2023 8:09:31 AM, 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 8.690209465081626 (T) = (0 -2354.2608651838864) / Math.Sqrt((16280.990977913641 / (299)) + (5783.970113310081 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.07001213206951944 = (2531.496319863685 - 2354.2608651838864) / 2531.496319863685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Reflection.Invoke.StaticMethod4_int_string_struct_class
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 190.33908648053753 < 231.1120633944852.
IsChangePoint: Marked as a change because one of 4/24/2023 11:08:13 AM, 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 39.3016504995556 (T) = (0 -194.38267654572343) / Math.Sqrt((171.6494611674663 / (299)) + (48.83374678271449 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.26824111335693324 = (265.6376028960183 - 194.38267654572343) / 265.6376028960183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Reflection.Invoke.Ctor4_int_string_struct_class
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 202.22046286919831 < 239.53851486197271.
IsChangePoint: Marked as a change because one of 4/24/2023 11:08:13 AM, 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 27.762318510054477 (T) = (0 -205.6364630286128) / Math.Sqrt((1771.8144776687582 / (242)) + (47.27760342197175 / (17))) is greater than 1.9692374962334795 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (242) + (17) - 2, .975) and 0.30026534072834776 = (293.8777725297444 - 205.6364630286128) / 293.8777725297444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Reflection.Invoke.StaticMethod4_arrayNotCached_int_string_struct_class
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 240.70340499441969 < 282.20912324266976.
IsChangePoint: Marked as a change because one of 4/24/2023 11:08:13 AM, 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 24.152495407761766 (T) = (0 -250.4565060442411) / Math.Sqrt((173.09806975927867 / (299)) + (122.80681984104073 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.2078301638379438 = (316.1651638462596 - 250.4565060442411) / 316.1651638462596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository