Description
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 764f77412ef4ec1a7e5dc521608247487a66cd7e |
Compare | bc23bf4ee16cbd1999105b1916e1f2f76baef6a7 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
15.85 μs | 11.98 μs | 0.76 | 0.16 | False | ||||
15.99 μs | 11.85 μs | 0.74 | 0.16 | False | ||||
14.97 μs | 11.43 μs | 0.76 | 0.19 | False | ||||
15.72 μs | 11.70 μs | 0.74 | 0.17 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>*'
Payloads
System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.97979256407804 < 15.12848710513757.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 45.732200329083874 (T) = (0 -11897.277108272141) / Math.Sqrt((1679797.4594270252 / (299)) + (12090.492926850187 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.2341914774393741 = (15535.576789471264 - 11897.277108272141) / 15535.576789471264 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.854840285472678 < 15.450438957883096.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 43.77026997528059 (T) = (0 -12214.678742572398) / Math.Sqrt((1504901.320812375 / (299)) + (36049.6844460866 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.2381307610817587 = (16032.513348242946 - 12214.678742572398) / 16032.513348242946 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToStream(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.427941977566446 < 14.915361935448349.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 33.31534922128242 (T) = (0 -11693.18505864384) / Math.Sqrt((1595740.4418589645 / (299)) + (112327.98006081002 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.24053710068719159 = (15396.650803119268 - 11693.18505864384) / 15396.650803119268 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.697943014113987 < 14.569715272321321.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 40.11623118500445 (T) = (0 -11354.384268344129) / Math.Sqrt((1292272.4584998072 / (299)) + (76636.8992397586 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.2492490986744805 = (15124.036812072984 - 11354.384268344129) / 15124.036812072984 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 764f77412ef4ec1a7e5dc521608247487a66cd7e |
Compare | bc23bf4ee16cbd1999105b1916e1f2f76baef6a7 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsTrue<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
381.39 μs | 297.10 μs | 0.78 | 0.43 | False | ||||
361.36 μs | 299.86 μs | 0.83 | 0.31 | False | ||||
359.02 μs | 327.68 μs | 0.91 | 0.43 | False | ||||
557.53 μs | 265.95 μs | 0.48 | 0.14 | True | ||||
327.95 μs | 297.36 μs | 0.91 | 0.36 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<String>*'
Payloads
System.Collections.ContainsTrue<String>.ICollection(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 297.10244246967653 < 325.77371089628144.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 32.79323392776587 (T) = (0 -295203.9928763724) / Math.Sqrt((214145565.527984 / (299)) + (26204545.17182185 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.1456183874928932 = (345517.7271548747 - 295203.9928763724) / 345517.7271548747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<String>.ImmutableArray(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 299.85698837601086 < 344.31956773874566.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 66.83265233498841 (T) = (0 -300141.3094970354) / Math.Sqrt((202927466.05487454 / (299)) + (1694406.6425927738 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.16474198472579724 = (359339.63399142417 - 300141.3094970354) / 359339.63399142417 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<String>.Stack(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 327.6786335069445 < 366.8077895715375.
IsChangePoint: Marked as a change because one of 5/25/2023 10:38:38 PM, 7/25/2023 7:04:35 PM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 12.287073965072064 (T) = (0 -322613.81977146625) / Math.Sqrt((639393853.163297 / (299)) + (137451539.84903058 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.10855877580095216 = (361901.39182909334 - 322613.81977146625) / 361901.39182909334 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<String>.LinkedList(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 265.9548455019557 < 529.3170662295387.
IsChangePoint: Marked as a change because one of 5/8/2023 6:34:59 AM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 39.442986542331575 (T) = (0 -278852.7313833645) / Math.Sqrt((366359256.12469965 / (299)) + (837204968.3343611 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.501222018973128 = (559071.8555964906 - 278852.7313833645) / 559071.8555964906 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<String>.List(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 297.35749363207543 < 313.93433505502855.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/1/2023 6:56:14 PM, 7/25/2023 7:04:35 PM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 22.13029649836736 (T) = (0 -293334.93575273524) / Math.Sqrt((236991245.27249724 / (299)) + (105589986.64145012 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.16295501272230983 = (350441.0637554194 - 293334.93575273524) / 350441.0637554194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 764f77412ef4ec1a7e5dc521608247487a66cd7e |
Compare | bc23bf4ee16cbd1999105b1916e1f2f76baef6a7 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsFalse<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
609.67 μs | 542.41 μs | 0.89 | 0.30 | False | ||||
727.30 μs | 546.78 μs | 0.75 | 0.36 | False | ||||
664.67 μs | 538.03 μs | 0.81 | 0.29 | False | ||||
1.06 ms | 477.87 μs | 0.45 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<String>*'
Payloads
System.Collections.ContainsFalse<String>.ImmutableArray(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 542.412015625 < 580.4907561192413.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 46.685317903168055 (T) = (0 -542036.791244316) / Math.Sqrt((2297210389.5575714 / (299)) + (1145699.2941150994 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.19349563327100944 = (672081.6570934408 - 542036.791244316) / 672081.6570934408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsFalse<String>.Stack(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 546.7785336206897 < 660.3484920885754.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 29.332268320856727 (T) = (0 -560669.0567572012) / Math.Sqrt((1715994166.8938923 / (299)) + (278464799.74323976 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.20503232042769653 = (705272.76915062 - 560669.0567572012) / 705272.76915062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsFalse<String>.Queue(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 538.02679375 < 631.8500782412574.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 38.33212050884245 (T) = (0 -541753.08399953) / Math.Sqrt((2032120875.081644 / (299)) + (43959152.85377035 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.18078037472048542 = (661303.8399985668 - 541753.08399953) / 661303.8399985668 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsFalse<String>.LinkedList(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 477.8671241883117 < 1.002821549828869.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 53.47478444633236 (T) = (0 -493709.9354061453) / Math.Sqrt((2581136538.825138 / (299)) + (1770829542.9526162 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.5419318184945184 = (1077808.8401240269 - 493709.9354061453) / 1077808.8401240269 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 764f77412ef4ec1a7e5dc521608247487a66cd7e |
Compare | bc23bf4ee16cbd1999105b1916e1f2f76baef6a7 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsTrueComparer<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
40.60 μs | 34.85 μs | 0.86 | 0.13 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer<String>*'
Payloads
System.Collections.ContainsTrueComparer<String>.ImmutableHashSet(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.84707033929903 < 37.006128456550705.
IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 22.241459557454178 (T) = (0 -35281.73038278723) / Math.Sqrt((3495502.2441583113 / (299)) + (455749.15103244863 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.11707523820804192 = (39960.06444669246 - 35281.73038278723) / 39960.06444669246 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository