Skip to content

[Perf] Windows/arm64: 16 Improvements on 5/13/2023 2:36:22 AM #17908

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 14.56 ns 13.10 ns 0.90 0.79 False
TryParse - Duration of single invocation 27.65 ns 17.96 ns 0.65 0.77 False
TryParse - Duration of single invocation 27.97 ns 20.83 ns 0.74 0.78 False
Parse - Duration of single invocation 22.97 ns 21.27 ns 0.93 0.75 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.Parse(value: "0")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.102373474666663 < 15.902560429651494.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 6.851225530786881 (T) = (0 -11.331555730728097) / Math.Sqrt((4.701337331212279 / (299)) + (1.3014047651553922 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.19490580674895147 = (14.074819848060478 - 11.331555730728097) / 14.074819848060478 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "-32768")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.95526920913403 < 23.651233562321007.
IsChangePoint: Marked as a change because one of 4/19/2023 6:20:25 PM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 4.873270916144853 (T) = (0 -19.372186837560108) / Math.Sqrt((11.072180458925335 / (299)) + (4.092870999769785 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.14995927084562513 = (22.789716037291136 - 19.372186837560108) / 22.789716037291136 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "32767")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.831728068850975 < 25.50267927704003.
IsChangePoint: Marked as a change because one of 4/18/2023 7:42:00 PM, 5/11/2023 7:02:02 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 2.3972300309005545 (T) = (0 -19.69650770321412) / Math.Sqrt((10.125383604171326 / (299)) + (6.930686305919963 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08395144169291632 = (21.501597840636883 - 19.69650770321412) / 21.501597840636883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "32767")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.27443552334166 < 22.864867556704564.
IsChangePoint: Marked as a change because one of 4/19/2023 6:20:25 PM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 3.1563741262735796 (T) = (0 -19.882315311963907) / Math.Sqrt((10.79213843816966 / (299)) + (3.0231122310299297 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.08454307329401338 = (21.718460729228198 - 19.882315311963907) / 21.718460729228198 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 14.83 ns 13.34 ns 0.90 0.82 False
ParseSpan - Duration of single invocation 27.74 ns 20.40 ns 0.74 0.72 False
TryParse - Duration of single invocation 23.36 ns 9.00 ns 0.39 0.78 False
TryParse - Duration of single invocation 23.80 ns 11.99 ns 0.50 0.78 False
Parse - Duration of single invocation 27.92 ns 19.37 ns 0.69 0.74 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.Parse(value: "0")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.343072951103554 < 16.80100509896595.
IsChangePoint: Marked as a change because one of 4/24/2023 7:54:29 PM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 5.5499329126782895 (T) = (0 -12.559418849981057) / Math.Sqrt((5.04666616378861 / (299)) + (1.8747953389207486 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.17334659226215224 = (15.193089065404251 - 12.559418849981057) / 15.193089065404251 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.ParseSpan(value: "12345")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.403711832671455 < 22.48752342223847.
IsChangePoint: Marked as a change because one of 4/25/2023 1:14:10 AM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 4.95165682142072 (T) = (0 -18.898784151316452) / Math.Sqrt((8.467585366181476 / (299)) + (4.157993617007115 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.15502443982636258 = (22.36607192216644 - 18.898784151316452) / 22.36607192216644 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "0")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.003994978168034 < 13.206589980991051.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 6.90728800609157 (T) = (0 -11.004323594537754) / Math.Sqrt((5.319592693710412 / (299)) + (1.7316563111341867 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.21525002587570466 = (14.022712911609215 - 11.004323594537754) / 14.022712911609215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "12345")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.989283457176489 < 21.087178863915337.
IsChangePoint: Marked as a change because one of 4/26/2023 2:24:59 PM, 5/10/2023 10:36:48 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 3.5962053172317185 (T) = (0 -17.690776492831436) / Math.Sqrt((6.740654430252058 / (299)) + (8.125801344300097 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.13247152235191628 = (20.39215650971145 - 17.690776492831436) / 20.39215650971145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.Parse(value: "12345")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.371243723731283 < 24.248340053266375.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 3.4061480098904617 (T) = (0 -18.950231417912452) / Math.Sqrt((8.931387888566698 / (299)) + (9.504254825540052 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.15107630231136013 = (22.322655698631277 - 18.950231417912452) / 22.322655698631277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseSpan - Duration of single invocation 22.58 ns 11.32 ns 0.50 0.80 False
ParseSpan - Duration of single invocation 33.51 ns 25.60 ns 0.76 0.72 False
ParseSpan - Duration of single invocation 19.27 ns 11.40 ns 0.59 0.79 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.TryParseSpan(value: "12345")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.316060283406772 < 20.42404206678519.
IsChangePoint: Marked as a change because one of 4/23/2023 2:41:50 AM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 5.689840704515408 (T) = (0 -15.197939144795379) / Math.Sqrt((6.176458620903995 / (299)) + (6.835173122192147 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.23905958566354313 = (19.97257453863591 - 15.197939144795379) / 19.97257453863591 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseSpan(value: "2147483647")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.596318172393158 < 31.65685138494683.
IsChangePoint: Marked as a change because one of 4/21/2023 12:49:36 AM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 4.257497698035128 (T) = (0 -28.360229930490057) / Math.Sqrt((12.805924186758778 / (299)) + (4.58238924914036 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.0960580156567357 = (31.373949237565782 - 28.360229930490057) / 31.373949237565782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseSpan(value: "4")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.399205774449495 < 17.70678293325335.
IsChangePoint: Marked as a change because one of 4/24/2023 1:33:55 PM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 7.721163940719957 (T) = (0 -12.091745095642903) / Math.Sqrt((6.176315942121327 / (299)) + (1.3813760673619526 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.21040304579445787 = (15.313819324200772 - 12.091745095642903) / 15.313819324200772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchmark.GetChildKeysTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AddChainedConfigurationNoDelimiter - Duration of single invocation 1.35 ms 1.15 ms 0.85 0.53 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'
Details

Payloads

Baseline
Compare

Histogram

Benchmark.GetChildKeysTests.AddChainedConfigurationNoDelimiter


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1475676018957346 < 1.2579251331077674.
IsChangePoint: Marked as a change because one of 4/23/2023 8:38:14 PM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 4.845273086684241 (T) = (0 -1119294.5185594517) / Math.Sqrt((8480409784.669578 / (299)) + (5451409352.320438 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09829033765176558 = (1241302.5669976545 - 1119294.5185594517) / 1241302.5669976545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tasks.Tests.Perf_AsyncMethods

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EmptyAsyncMethodInvocation - Duration of single invocation 30.18 ns 21.96 ns 0.73 0.64 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.Tests.Perf_AsyncMethods*'
Details

Payloads

Baseline
Compare

Histogram

System.Threading.Tasks.Tests.Perf_AsyncMethods.EmptyAsyncMethodInvocation


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.963439583333333 < 27.720772197916666.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 16.32594156904064 (T) = (0 -22.491839113756615) / Math.Sqrt((8.515473856037735 / (299)) + (2.802996994074635 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.29735734313166096 = (32.01035247988243 - 22.491839113756615) / 32.01035247988243 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_Lock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReaderWriterLockSlimPerf - Duration of single invocation 95.69 ns 54.68 ns 0.57 0.68 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Lock*'
Details

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Lock.ReaderWriterLockSlimPerf


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 54.68042273504567 < 85.88403327735898.
IsChangePoint: Marked as a change because one of 3/29/2023 4:47:17 AM, 4/19/2023 4:06:12 AM, 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 16.59388484226527 (T) = (0 -58.09307161123003) / Math.Sqrt((72.07896347638741 / (299)) + (16.52419386020584 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.2821539402823559 = (80.92692134310833 - 58.09307161123003) / 80.92692134310833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToUpper - Duration of single invocation 30.98 ns 21.29 ns 0.69 0.69 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.ToUpper(s: "TEST")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.291559711212944 < 28.899316999625494.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/17/2023 6:28:21 PM falls between 5/9/2023 1:09:22 AM and 5/17/2023 6:28:21 PM.
IsImprovementStdDev: Marked as improvement because 7.428422202060382 (T) = (0 -24.59638325728003) / Math.Sqrt((9.370195302461902 / (299)) + (6.325115992579236 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.19747747050741285 = (30.648838323369745 - 24.59638325728003) / 30.648838323369745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions