Description
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 195bf987887ea645a747ff0586440f7920c47b01 |
Compare | 63a7a13190448f1ba5b3b449f1b9628d042d209d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.Sort<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
List - Duration of single invocation | 6.45 μs | 5.80 μs | 0.90 | 0.17 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<Int32>*'
Payloads
Histogram
System.Collections.Sort<Int32>.List(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 6.60857280715939 (T) = (0 -5917.036122539682) / Math.Sqrt((31026.933693964485 / (299)) + (15982.93035389738 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07605147266278466 = (6404.0754949762795 - 5917.036122539682) / 6404.0754949762795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 195bf987887ea645a747ff0586440f7920c47b01 |
Compare | 63a7a13190448f1ba5b3b449f1b9628d042d209d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Globalization.Tests.StringSearch
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IndexOf_Word_NotFound - Duration of single invocation | 630.60 ns | 560.37 ns | 0.89 | 0.00 | False | |||||
IndexOf_Word_NotFound - Duration of single invocation | 630.53 ns | 560.20 ns | 0.89 | 0.00 | False | |||||
IndexOf_Word_NotFound - Duration of single invocation | 972.86 ns | 813.76 ns | 0.84 | 0.00 | False | |||||
IndexOf_Word_NotFound - Duration of single invocation | 630.21 ns | 565.25 ns | 0.90 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
Payloads
Histogram
System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 253.57583013706957 (T) = (0 -560.8285427113) / Math.Sqrt((3.067647567245063 / (299)) + (0.19521845260956933 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11039879301561933 = (630.4269129899537 - 560.8285427113) / 630.4269129899537 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 335.43410173328925 (T) = (0 -560.3666850425417) / Math.Sqrt((2.549714317952406 / (299)) + (0.10779984928880615 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11207290608992006 = (631.0953780843744 - 560.3666850425417) / 631.0953780843744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/21/2023 1:51:00 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 195.46215480786444 (T) = (0 -812.9808314600823) / Math.Sqrt((75.14136434811185 / (299)) + (1.5017150872871674 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17251174105812017 = (982.4681168282093 - 812.9808314600823) / 982.4681168282093 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 44.66639370096367 (T) = (0 -562.2130464328677) / Math.Sqrt((3.2880491914222154 / (299)) + (6.953843860861608 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10813331268853882 = (630.3778966424491 - 562.2130464328677) / 630.3778966424491 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 195bf987887ea645a747ff0586440f7920c47b01 |
Compare | 63a7a13190448f1ba5b3b449f1b9628d042d209d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.TryAddDefaultSize<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Dictionary - Duration of single invocation | 13.56 μs | 11.38 μs | 0.84 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddDefaultSize<Int32>*'
Payloads
Histogram
System.Collections.TryAddDefaultSize<Int32>.Dictionary(Count: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 125.60612064878433 (T) = (0 -11397.621717303518) / Math.Sqrt((44835.78730678877 / (299)) + (304.13367980100054 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.14872644055377252 = (13388.90605825691 - 11397.621717303518) / 13388.90605825691 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 195bf987887ea645a747ff0586440f7920c47b01 |
Compare | 63a7a13190448f1ba5b3b449f1b9628d042d209d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Single
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToStringWithFormat - Duration of single invocation | 216.76 ns | 194.40 ns | 0.90 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'
Payloads
Histogram
System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G17")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 26.260700155123168 (T) = (0 -195.24170803231854) / Math.Sqrt((6.913693463625097 / (299)) + (1.154983133257625 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07912719077439576 = (212.01810508066197 - 195.24170803231854) / 212.01810508066197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 195bf987887ea645a747ff0586440f7920c47b01 |
Compare | 63a7a13190448f1ba5b3b449f1b9628d042d209d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Memory.Span<Char>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SequenceEqual - Duration of single invocation | 62.03 ns | 53.87 ns | 0.87 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Char>*'
Payloads
Histogram
System.Memory.Span<Char>.SequenceEqual(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 31.92117568296261 (T) = (0 -54.334160097541435) / Math.Sqrt((0.4122321390792662 / (299)) + (0.1691065148301776 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12371392331171674 = (62.00504782968205 - 54.334160097541435) / 62.00504782968205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 195bf987887ea645a747ff0586440f7920c47b01 |
Compare | 63a7a13190448f1ba5b3b449f1b9628d042d209d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Linq.Tests.Perf_Enumerable
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SelectToList - Duration of single invocation | 656.79 ns | 562.60 ns | 0.86 | 0.46 | False | |||||
SelectToList - Duration of single invocation | 391.46 ns | 311.60 ns | 0.80 | 0.41 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Payloads
Histogram
System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 3.4432541226982694 (T) = (0 -508.1394543071963) / Math.Sqrt((1425.3683662752633 / (299)) + (2254.7849653006706 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.15708503252480843 = (602.8359608196798 - 508.1394543071963) / 602.8359608196798 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 30.035343482453023 (T) = (0 -309.09692971800587) / Math.Sqrt((773.7001823610248 / (299)) + (12.516483492195778 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.2242089313878641 = (398.4280590790635 - 309.09692971800587) / 398.4280590790635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 195bf987887ea645a747ff0586440f7920c47b01 |
Compare | 63a7a13190448f1ba5b3b449f1b9628d042d209d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Tests.Perf_Encoding
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetString - Duration of single invocation | 27.13 ns | 24.66 ns | 0.91 | 0.27 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'
Payloads
Histogram
System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "ascii")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/17/2023 11:08:46 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 7.964103968693639 (T) = (0 -24.463333504494102) / Math.Sqrt((0.8513902822252984 / (299)) + (0.6572996502546483 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1329770921599296 = (28.215325435215107 - 24.463333504494102) / 28.215325435215107 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 195bf987887ea645a747ff0586440f7920c47b01 |
Compare | 63a7a13190448f1ba5b3b449f1b9628d042d209d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.TryAddDefaultSize<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Dictionary - Duration of single invocation | 27.66 μs | 24.96 μs | 0.90 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddDefaultSize<String>*'
Payloads
Histogram
System.Collections.TryAddDefaultSize<String>.Dictionary(Count: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 63.52772782890929 (T) = (0 -24950.04424143476) / Math.Sqrt((266075.37093546434 / (299)) + (203.37658100858704 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07304025243642087 = (26915.995335302803 - 24950.04424143476) / 26915.995335302803 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 195bf987887ea645a747ff0586440f7920c47b01 |
Compare | 63a7a13190448f1ba5b3b449f1b9628d042d209d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
CertProp - Duration of single invocation | 62.54 ns | 42.39 ns | 0.68 | 0.04 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests*'
Payloads
Histogram
System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/18/2023 9:46:56 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 50.57319388571694 (T) = (0 -41.9206349759531) / Math.Sqrt((18.307644473564398 / (299)) + (0.1676030597830944 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.29219821667969825 = (59.22651788090048 - 41.9206349759531) / 59.22651788090048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 195bf987887ea645a747ff0586440f7920c47b01 |
Compare | 63a7a13190448f1ba5b3b449f1b9628d042d209d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
CtorInvoke - Duration of single invocation | 362.87 μs | 280.60 μs | 0.77 | 0.33 | False | |||||
CtorInvoke - Duration of single invocation | 361.08 μs | 307.90 μs | 0.85 | 0.24 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: Compiled)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/21/2023 1:51:00 AM, 4/12/2023 2:59:13 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 4.4710811569082205 (T) = (0 -263941.3752885983) / Math.Sqrt((529327065.434973 / (299)) + (208938282.09139478 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1252273796406848 = (301725.6932209237 - 263941.3752885983) / 301725.6932209237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: IgnoreCase, Compiled)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/21/2023 1:51:00 AM, 4/12/2023 2:59:13 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 7.664054320382402 (T) = (0 -302577.6700604377) / Math.Sqrt((480900598.0601342 / (299)) + (25139162.813897338 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0741171696904298 = (326799.0939623218 - 302577.6700604377) / 326799.0939623218 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository