Skip to content

[Perf] Linux/x64: 15 Improvements on 5/23/2023 8:43:31 PM #18347

Description

@performanceautofiler

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline b4008aefaf8e3b262fbb764070ea1dd1abe7d97c
Compare 7a119156cbe958c5c34098434e84010526fac5f1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableHashSet - Duration of single invocation 252.35 μs 216.15 μs 0.86 0.21 False
HashSet - Duration of single invocation 140.95 μs 94.89 μs 0.67 0.26 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrueComparer<Int32>.ImmutableHashSet(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 216.15239726027394 < 236.4186887338954.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 28.97487886489423 (T) = (0 -200228.51656510602) / Math.Sqrt((120432671.41361667 / (299)) + (62904641.210256666 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.22667820725151724 = (258920.04912142555 - 200228.51656510602) / 258920.04912142555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrueComparer<Int32>.HashSet(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 94.88558113636363 < 138.2483214905981.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 41.31109384995639 (T) = (0 -94901.55506400333) / Math.Sqrt((80791477.2846416 / (299)) + (21161169.43838766 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.3545796145779181 = (147038.360125457 - 94901.55506400333) / 147038.360125457 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline b4008aefaf8e3b262fbb764070ea1dd1abe7d97c
Compare 7a119156cbe958c5c34098434e84010526fac5f1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 298.96 μs 163.59 μs 0.55 0.15 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.SortedList(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 163.58565871465774 < 285.2327028195679.
IsChangePoint: Marked as a change because one of 3/10/2023 8:23:08 AM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 64.41185533726686 (T) = (0 -165119.33721758984) / Math.Sqrt((1559805382.2905133 / (299)) + (12198394.316928022 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.48725355902971096 = (322029.22150981374 - 165119.33721758984) / 322029.22150981374 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline b4008aefaf8e3b262fbb764070ea1dd1abe7d97c
Compare 7a119156cbe958c5c34098434e84010526fac5f1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 296.65 μs 176.24 μs 0.59 0.16 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 176.23520979853478 < 285.9982395514494.
IsChangePoint: Marked as a change because one of 3/10/2023 8:23:08 AM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 54.42416788713075 (T) = (0 -167685.48120899615) / Math.Sqrt((1476836607.6453333 / (299)) + (59277762.74129165 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.48510191046967027 = (325667.3206186343 - 167685.48120899615) / 325667.3206186343 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline b4008aefaf8e3b262fbb764070ea1dd1abe7d97c
Compare 7a119156cbe958c5c34098434e84010526fac5f1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 760.93 μs 601.68 μs 0.79 0.07 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<Int32>.SortedList(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 601.6782046474359 < 724.468590549009.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 39.39538866954864 (T) = (0 -565406.152344928) / Math.Sqrt((2988255378.9774218 / (299)) + (375268615.4963207 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.287226462518656 = (793247.9569076691 - 565406.152344928) / 793247.9569076691 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline b4008aefaf8e3b262fbb764070ea1dd1abe7d97c
Compare 7a119156cbe958c5c34098434e84010526fac5f1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 105.94 μs 25.06 μs 0.24 0.10 False
Array - Duration of single invocation 106.21 μs 24.75 μs 0.23 0.07 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.List(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.056246138461542 < 99.74313021802197.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/22/2023 10:50:22 PM, 3/23/2023 11:51:53 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 29.382022611279545 (T) = (0 -24890.87585221288) / Math.Sqrt((2955451138.168948 / (299)) + (67710.23600250998 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.787774860510643 = (117285.23732911081 - 24890.87585221288) / 117285.23732911081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<Int32>.Array(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.750039986666668 < 100.88926611195053.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/22/2023 10:50:22 PM, 3/23/2023 11:51:53 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 29.377073078560475 (T) = (0 -24856.576422577033) / Math.Sqrt((2938734049.374298 / (299)) + (126844.51925506153 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.7875327195398929 = (116990.13781674545 - 24856.576422577033) / 116990.13781674545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline b4008aefaf8e3b262fbb764070ea1dd1abe7d97c
Compare 7a119156cbe958c5c34098434e84010526fac5f1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 380.47 μs 283.58 μs 0.75 0.10 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<Int32>.SortedList(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 283.5769479707792 < 364.689821853586.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 46.06278853583816 (T) = (0 -274513.9865947143) / Math.Sqrt((1129403928.0694246 / (299)) + (68174343.97633786 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.318920940703091 = (403057.44663196715 - 274513.9865947143) / 403057.44663196715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline b4008aefaf8e3b262fbb764070ea1dd1abe7d97c
Compare 7a119156cbe958c5c34098434e84010526fac5f1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 216.66 μs 145.82 μs 0.67 0.14 False
SortedList - Duration of single invocation 146.70 μs 76.90 μs 0.52 0.13 False
SortedSet - Duration of single invocation 122.47 μs 38.76 μs 0.32 0.09 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.ImmutableSortedSet(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 145.82228521273927 < 205.69236652355386.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/22/2023 10:50:22 PM, 3/23/2023 11:51:53 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 26.139814896894357 (T) = (0 -140241.51319313585) / Math.Sqrt((3151181944.255056 / (299)) + (36111751.93534518 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.398779590500952 = (233261.3979455365 - 140241.51319313585) / 233261.3979455365 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.SortedList(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 76.9017375 < 138.2353455817403.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 119.47612624845775 (T) = (0 -75863.00762827836) / Math.Sqrt((92853396.26220939 / (299)) + (1676307.631807264 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.501838516178786 = (152285.97571687208 - 75863.00762827836) / 152285.97571687208 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.SortedSet(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 38.75611649584417 < 111.0735108525149.
IsChangePoint: Marked as a change because one of 3/22/2023 10:50:22 PM, 3/23/2023 11:51:53 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 29.6096542966746 (T) = (0 -37060.807985769265) / Math.Sqrt((3062898737.9717245 / (299)) + (470880.1841029791 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.7191455758266673 = (131957.35867381864 - 37060.807985769265) / 131957.35867381864 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline b4008aefaf8e3b262fbb764070ea1dd1abe7d97c
Compare 7a119156cbe958c5c34098434e84010526fac5f1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 265.41 μs 155.39 μs 0.59 0.12 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.SortedList(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 155.39260633169934 < 255.39188580151625.
IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 57.02463087173388 (T) = (0 -153530.11179063364) / Math.Sqrt((1034816744.2891661 / (299)) + (30845033.158290207 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.46036038152128367 = (284504.89277167286 - 153530.11179063364) / 284504.89277167286 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline b4008aefaf8e3b262fbb764070ea1dd1abe7d97c
Compare 7a119156cbe958c5c34098434e84010526fac5f1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 280.13 μs 171.42 μs 0.61 0.13 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.SortedList(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 171.41617903428093 < 266.5964711708169.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 59.14486338326896 (T) = (0 -168306.35075624837) / Math.Sqrt((1093155548.842343 / (299)) + (21758334.197790243 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.4384329075821683 = (299708.35725363466 - 168306.35075624837) / 299708.35725363466 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline b4008aefaf8e3b262fbb764070ea1dd1abe7d97c
Compare 7a119156cbe958c5c34098434e84010526fac5f1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 383.22 μs 279.04 μs 0.73 0.06 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.SortedList(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 279.0416124043367 < 369.55555900297617.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 5/23/2023 1:41:12 PM, 5/29/2023 10:47:10 AM falls between 5/20/2023 9:58:12 PM and 5/29/2023 10:47:10 AM.
IsImprovementStdDev: Marked as improvement because 37.80486174569895 (T) = (0 -279193.290950107) / Math.Sqrt((998178557.7051817 / (299)) + (125784777.2247224 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.3118440008112583 = (405712.20955603727 - 279193.290950107) / 405712.20955603727 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions