Open
Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 2eca2d319bf8d1247edf38facfdd501e0e2f0030 |
Compare | 51a8dd5323721b363e61069575511f783e7ea6d3 |
Diff | Diff |
Regressions in System.Tests.Perf_Int32
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryFormat - Duration of single invocation | 359.97 ns | 492.86 ns | 1.37 | 0.17 | False | |||||
ToString - Duration of single invocation | 651.85 ns | 729.21 ns | 1.12 | 0.07 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
Payloads
Histogram
System.Tests.Perf_Int32.TryFormat(value: 12345)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 492.8608440749241 > 375.545925450121.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -25.84457571190944 (T) = (0 -502.44390338617205) / Math.Sqrt((983.1173759194976 / (47)) + (178.30350063196926 / (10))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (10) - 2, .025) and -0.4709948723134669 = (341.56740641520196 - 502.44390338617205) / 341.56740641520196 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.ToString(value: -2147483648)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 729.2063155758293 > 684.3506826283758.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 7:15:22 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -10.20566597903779 (T) = (0 -722.3251669528804) / Math.Sqrt((5026.290572474745 / (46)) + (141.95231641833035 / (10))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (10) - 2, .025) and -0.18622758522786528 = (608.9262936961015 - 722.3251669528804) / 608.9262936961015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 2eca2d319bf8d1247edf38facfdd501e0e2f0030 |
Compare | 51a8dd5323721b363e61069575511f783e7ea6d3 |
Diff | Diff |
Regressions in System.Text.Json.Tests.Perf_Basic
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
WriteBasicUtf8 - Duration of single invocation | 71.44 ms | 90.50 ms | 1.27 | 0.05 | False | |||||
WriteBasicUtf8 - Duration of single invocation | 52.14 ms | 75.38 ms | 1.45 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'
Payloads
Histogram
System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 100000)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.49589283928572 > 73.034914943125.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -43.82252103494706 (T) = (0 -91814303.24142857) / Math.Sqrt((1044661612834.847 / (46)) + (2194022181497.7998 / (10))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (10) - 2, .025) and -0.30693774754127945 = (70251474.038574 - 91814303.24142857) / 70251474.038574 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 100000)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.38498208928571 > 54.62824496634616.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/15/2023 4:08:29 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -65.4121511111436 (T) = (0 -74875062.99024725) / Math.Sqrt((1754074128119.2312 / (46)) + (757653307401.3975 / (10))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (10) - 2, .025) and -0.41810652770680706 = (52799321.86147276 - 74875062.99024725) / 52799321.86147276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 2eca2d319bf8d1247edf38facfdd501e0e2f0030 |
Compare | 51a8dd5323721b363e61069575511f783e7ea6d3 |
Diff | Diff |
Regressions in System.Tests.Perf_Version
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryFormat2 - Duration of single invocation | 760.17 ns | 1.12 μs | 1.47 | 0.12 | False | |||||
ToString2 - Duration of single invocation | 1.10 μs | 1.43 μs | 1.30 | 0.04 | False | |||||
ToString3 - Duration of single invocation | 1.47 μs | 1.90 μs | 1.30 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'
Payloads
Histogram
System.Tests.Perf_Version.TryFormat2
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1184515228533525 > 724.2589986636451.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -86.24907913695705 (T) = (0 -1087.2573937011782) / Math.Sqrt((226.79804288115042 / (46)) + (159.77626131237326 / (10))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (10) - 2, .025) and -0.5691834273522646 = (692.881007248301 - 1087.2573937011782) / 692.881007248301 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Version.ToString2
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4267705436460205 > 1.1659301484824813.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -28.963582231678842 (T) = (0 -1402.8259962134505) / Math.Sqrt((4483.3668800814485 / (46)) + (341.5761509150203 / (10))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (10) - 2, .025) and -0.3103953822855898 = (1070.5364313529885 - 1402.8259962134505) / 1070.5364313529885 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Version.ToString3
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9020946846902582 > 1.5187384945824778.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -46.43041809205259 (T) = (0 -1902.88170383561) / Math.Sqrt((5319.790771938327 / (47)) + (79.24576453853257 / (10))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (10) - 2, .025) and -0.3670995365588348 = (1391.9116003984668 - 1902.88170383561) / 1391.9116003984668 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 2eca2d319bf8d1247edf38facfdd501e0e2f0030 |
Compare | 51a8dd5323721b363e61069575511f783e7ea6d3 |
Diff | Diff |
Regressions in System.Buffers.Text.Tests.Utf8FormatterTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
FormatterInt64 - Duration of single invocation | 777.56 ns | 981.08 ns | 1.26 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
Payloads
Histogram
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 981.0816089963674 > 798.1077434198894.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -32.89379017734937 (T) = (0 -998.0796408524504) / Math.Sqrt((133.1485354529759 / (47)) + (450.97475587610467 / (10))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (10) - 2, .025) and -0.2956183978865858 = (770.3500062059315 - 998.0796408524504) / 770.3500062059315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 2eca2d319bf8d1247edf38facfdd501e0e2f0030 |
Compare | 51a8dd5323721b363e61069575511f783e7ea6d3 |
Diff | Diff |
Regressions in System.Text.Json.Tests.Perf_Deep
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
WriteDeepUtf16 - Duration of single invocation | 175.98 ms | 200.27 ms | 1.14 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Deep*'
Payloads
Histogram
System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 200.27485742857144 > 188.79953076923078.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/9/2022 1:52:32 PM, 1/15/2023 4:08:29 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -21.08812700573864 (T) = (0 -202543721.80666667) / Math.Sqrt((12700212976706.117 / (47)) + (10186352047534.66 / (10))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (10) - 2, .025) and -0.1340451753872421 = (178602868.9179019 - 202543721.80666667) / 178602868.9179019 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 2eca2d319bf8d1247edf38facfdd501e0e2f0030 |
Compare | 51a8dd5323721b363e61069575511f783e7ea6d3 |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_BigInteger
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToStringD - Duration of single invocation | 659.78 ns | 1.03 μs | 1.56 | 0.13 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'
Payloads
Histogram
System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0287410134526986 > 712.3003201155551.
IsChangePoint: Marked as a change because one of 12/1/2022 11:48:13 PM, 12/9/2022 1:52:32 PM, 1/15/2023 4:08:29 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -24.12521454824905 (T) = (0 -1013.3107592956) / Math.Sqrt((3672.7741414237175 / (47)) + (792.2529223974778 / (10))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (10) - 2, .025) and -0.425858241708403 = (710.6672526446205 - 1013.3107592956) / 710.6672526446205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 2eca2d319bf8d1247edf38facfdd501e0e2f0030 |
Compare | 51a8dd5323721b363e61069575511f783e7ea6d3 |
Diff | Diff |
Regressions in System.Tests.Perf_Int64
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryFormat - Duration of single invocation | 388.84 ns | 566.44 ns | 1.46 | 0.16 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'
Payloads
Histogram
System.Tests.Perf_Int64.TryFormat(value: 12345)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 566.4432835672932 > 407.215650687202.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/15/2023 4:08:29 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -31.97460420023956 (T) = (0 -551.4386234994943) / Math.Sqrt((664.1440875497849 / (47)) + (155.45488516080795 / (10))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (10) - 2, .025) and -0.4617164935130571 = (377.254156976897 - 551.4386234994943) / 377.254156976897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository