Description
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | c50c745fa88f0850c8e8ebed37749118929d7e87 |
Compare | e07f4527bdedff6278accf9db8a8c7f9f2a48beb |
Diff | Diff |
Regressions in System.Collections.CreateAddAndClear<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ConcurrentQueue - Duration of single invocation | 8.27 μs | 10.07 μs | 1.22 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<String>*'
Payloads
Histogram
System.Collections.CreateAddAndClear<String>.ConcurrentQueue(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.071544886250992 > 8.690614821057371.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -115.96901487670786 (T) = (0 -10138.163344797133) / Math.Sqrt((1792.8044408339288 / (24)) + (2886.4230344746566 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.22354139893389938 = (8285.917708735278 - 10138.163344797133) / 8285.917708735278 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | c50c745fa88f0850c8e8ebed37749118929d7e87 |
Compare | e07f4527bdedff6278accf9db8a8c7f9f2a48beb |
Diff | Diff |
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Cache
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IsMatch_Multithreading - Duration of single invocation | 44.91 ms | 52.38 ms | 1.17 | 0.70 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Cache*'
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch_Multithreading(total: 400000, unique: 1, cacheSize: 15)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.38371396428572 > 43.91111331048077.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -7.513511692363451 (T) = (0 -58167555.76737351) / Math.Sqrt((24969324819449.668 / (24)) + (49564850370344.016 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.356419523829264 = (42883160.22108158 - 58167555.76737351) / 42883160.22108158 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | c50c745fa88f0850c8e8ebed37749118929d7e87 |
Compare | e07f4527bdedff6278accf9db8a8c7f9f2a48beb |
Diff | Diff |
Regressions in System.Collections.CtorFromCollection<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ImmutableArray - Duration of single invocation | 388.22 ns | 427.13 ns | 1.10 | 0.08 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<String>*'
Payloads
Histogram
System.Collections.CtorFromCollection<String>.ImmutableArray(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 427.1302381530856 > 408.0231409748177.
IsChangePoint: Marked as a change because one of 10/14/2022 9:06:25 AM, 11/21/2022 10:59:14 PM, 12/14/2022 12:26:48 AM, 1/5/2023 10:03:03 PM, 1/21/2023 9:40:41 PM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -14.413592522064945 (T) = (0 -421.3813120784167) / Math.Sqrt((82.058613008072 / (24)) + (15.272155129017252 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.07704630140098774 = (391.23788042380096 - 421.3813120784167) / 391.23788042380096 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | c50c745fa88f0850c8e8ebed37749118929d7e87 |
Compare | e07f4527bdedff6278accf9db8a8c7f9f2a48beb |
Diff | Diff |
Regressions in System.MathBenchmarks.Single
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Exp2 - Duration of single invocation | 50.50 μs | 69.02 μs | 1.37 | 0.20 | False | |||||
Pow - Duration of single invocation | 51.45 μs | 70.20 μs | 1.36 | 0.16 | False | |||||
Exp - Duration of single invocation | 30.01 μs | 36.97 μs | 1.23 | 0.20 | False | |||||
ExpM1 - Duration of single invocation | 32.45 μs | 39.58 μs | 1.22 | 0.17 | False | |||||
Exp10M1 - Duration of single invocation | 58.47 μs | 69.76 μs | 1.19 | 0.24 | False | |||||
Exp2M1 - Duration of single invocation | 52.65 μs | 69.77 μs | 1.33 | 0.20 | False | |||||
Exp10 - Duration of single invocation | 50.70 μs | 68.95 μs | 1.36 | 0.18 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'
Payloads
Histogram
System.MathBenchmarks.Single.Exp2
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.01845049559472 > 53.05348502326052.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -34.245477860947055 (T) = (0 -69741.96585675362) / Math.Sqrt((837127.121138001 / (24)) + (4368036.929151213 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.3744935333590274 = (50740.119297845056 - 69741.96585675362) / 50740.119297845056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.MathBenchmarks.Single.Pow
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.19791825485798 > 54.0616930265459.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -47.28875514768354 (T) = (0 -70505.77769355456) / Math.Sqrt((1022034.7805501956 / (24)) + (1760714.395377746 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.35507007642491106 = (52031.09338785662 - 70505.77769355456) / 52031.09338785662 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.MathBenchmarks.Single.Exp
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.97051248522459 > 31.51393453035029.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -5.6168754184609115 (T) = (0 -40237.68471020789) / Math.Sqrt((8347150.054725872 / (24)) + (41024205.08871314 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.3126820269740823 = (30653.032404931637 - 40237.68471020789) / 30653.032404931637 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.MathBenchmarks.Single.ExpM1
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.58192440025252 > 34.0543378092583.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -8.341714256826993 (T) = (0 -41344.43641041453) / Math.Sqrt((1633576.3321810076 / (24)) + (16106393.203310406 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.26448313274624735 = (32696.708512529764 - 41344.43641041453) / 32696.708512529764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.MathBenchmarks.Single.Exp10M1
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.75811953703706 > 55.29261521359428.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -33.38901808404263 (T) = (0 -70294.01947976642) / Math.Sqrt((2452201.92971196 / (24)) + (2501915.2900683894 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.31842638619186886 = (53316.60547450286 - 70294.01947976642) / 53316.60547450286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.MathBenchmarks.Single.Exp2M1
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.77374728422619 > 55.27473974431819.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -27.019015458421595 (T) = (0 -70859.34337773589) / Math.Sqrt((3600744.37885578 / (24)) + (4334937.568735229 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.3287225498742918 = (53328.92362249724 - 70859.34337773589) / 53328.92362249724 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.MathBenchmarks.Single.Exp10
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.94804320851689 > 53.21810607312452.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -30.56350158178773 (T) = (0 -69028.52617952159) / Math.Sqrt((7620123.501661613 / (24)) + (73040.3255655771 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.3356043500506718 = (51683.364296397136 - 69028.52617952159) / 51683.364296397136 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | c50c745fa88f0850c8e8ebed37749118929d7e87 |
Compare | e07f4527bdedff6278accf9db8a8c7f9f2a48beb |
Diff | Diff |
Regressions in System.Linq.Tests.Perf_OrderBy
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
OrderByValueType - Duration of single invocation | 43.90 μs | 50.68 μs | 1.15 | 0.45 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_OrderBy*'
Payloads
Histogram
System.Linq.Tests.Perf_OrderBy.OrderByValueType(NumberOfPeople: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.67796747870075 > 46.19685214061421.
IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 1/5/2023 10:03:03 PM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -7.135680451957544 (T) = (0 -50690.210505066556) / Math.Sqrt((2833638.52332989 / (24)) + (13500456.952154541 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.16017201680390003 = (43691.97823328862 - 50690.210505066556) / 43691.97823328862 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | c50c745fa88f0850c8e8ebed37749118929d7e87 |
Compare | e07f4527bdedff6278accf9db8a8c7f9f2a48beb |
Diff | Diff |
Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SingleParallel - Duration of single invocation | 55.07 μs | 71.53 μs | 1.30 | 0.35 | False | |||||
SingleParallel - Duration of single invocation | 67.01 μs | 83.28 μs | 1.24 | 0.31 | False | |||||
SingleParallel - Duration of single invocation | 57.63 μs | 76.16 μs | 1.32 | 0.28 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Byte>*'
Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.53111953722221 > 62.327370546699996.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -9.934707576395398 (T) = (0 -66740.98591333599) / Math.Sqrt((6672800.292791727 / (24)) + (4880891.868175631 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.12824263552769538 = (59154.816359266784 - 66740.98591333599) / 59154.816359266784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.27730550266666 > 68.40260402082059.
IsChangePoint: Marked as a change because one of 10/26/2022 4:56:52 PM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -31.694639522864268 (T) = (0 -83175.24421248147) / Math.Sqrt((6679767.415053553 / (24)) + (651460.5285907873 / (15))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (15) - 2, .025) and -0.2757532381393107 = (65196.96891680461 - 83175.24421248147) / 65196.96891680461 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 76.15692898133334 > 62.118965069350004.
IsChangePoint: Marked as a change because one of 10/26/2022 10:09:37 PM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -21.728974389110206 (T) = (0 -74143.22368359951) / Math.Sqrt((3875769.4388656598 / (24)) + (4132585.9298055647 / (15))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (15) - 2, .025) and -0.2402859508107063 = (59779.13692817063 - 74143.22368359951) / 59779.13692817063 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | c50c745fa88f0850c8e8ebed37749118929d7e87 |
Compare | e07f4527bdedff6278accf9db8a8c7f9f2a48beb |
Diff | Diff |
Regressions in System.Collections.CreateAddAndClear<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ConcurrentQueue - Duration of single invocation | 7.89 μs | 8.68 μs | 1.10 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'
Payloads
Histogram
System.Collections.CreateAddAndClear<Int32>.ConcurrentQueue(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.683263982126823 > 8.275400607283967.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -69.55931491590971 (T) = (0 -8663.537660373604) / Math.Sqrt((954.6137982490305 / (24)) + (1563.086420817273 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.10392001356575693 = (7847.975898534197 - 8663.537660373604) / 7847.975898534197 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | c50c745fa88f0850c8e8ebed37749118929d7e87 |
Compare | e07f4527bdedff6278accf9db8a8c7f9f2a48beb |
Diff | Diff |
Regressions in System.Buffers.Text.Tests.Base64Tests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Base64Decode - Duration of single invocation | 332.43 ns | 352.08 ns | 1.06 | 0.16 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'
Payloads
Histogram
System.Buffers.Text.Tests.Base64Tests.Base64Decode(NumberOfBytes: 1000)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 352.08118756989154 > 348.6693292218774.
IsChangePoint: Marked as a change because one of 10/26/2022 2:40:44 AM, 12/14/2022 12:26:48 AM, 1/5/2023 10:03:03 PM, 1/21/2023 9:40:41 PM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -21.606153075615218 (T) = (0 -351.89829338836716) / Math.Sqrt((14.429385268369273 / (24)) + (2.6282020847723397 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.056768726951027366 = (332.99461311999517 - 351.89829338836716) / 332.99461311999517 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository