Skip to content

[Perf] Linux/arm64: 29 Regressions on 1/27/2023 11:43:56 PM #81549

Closed as not planned
@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline c50c745fa88f0850c8e8ebed37749118929d7e87
Compare e07f4527bdedff6278accf9db8a8c7f9f2a48beb
Diff Diff

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentQueue - Duration of single invocation 8.27 μs 10.07 μs 1.22 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.ConcurrentQueue(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.071544886250992 > 8.690614821057371.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -115.96901487670786 (T) = (0 -10138.163344797133) / Math.Sqrt((1792.8044408339288 / (24)) + (2886.4230344746566 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.22354139893389938 = (8285.917708735278 - 10138.163344797133) / 8285.917708735278 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline c50c745fa88f0850c8e8ebed37749118929d7e87
Compare e07f4527bdedff6278accf9db8a8c7f9f2a48beb
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Cache

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch_Multithreading - Duration of single invocation 44.91 ms 52.38 ms 1.17 0.70 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Cache*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch_Multithreading(total: 400000, unique: 1, cacheSize: 15)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.38371396428572 > 43.91111331048077.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -7.513511692363451 (T) = (0 -58167555.76737351) / Math.Sqrt((24969324819449.668 / (24)) + (49564850370344.016 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.356419523829264 = (42883160.22108158 - 58167555.76737351) / 42883160.22108158 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline c50c745fa88f0850c8e8ebed37749118929d7e87
Compare e07f4527bdedff6278accf9db8a8c7f9f2a48beb
Diff Diff

Regressions in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableArray - Duration of single invocation 388.22 ns 427.13 ns 1.10 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.ImmutableArray(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 427.1302381530856 > 408.0231409748177.
IsChangePoint: Marked as a change because one of 10/14/2022 9:06:25 AM, 11/21/2022 10:59:14 PM, 12/14/2022 12:26:48 AM, 1/5/2023 10:03:03 PM, 1/21/2023 9:40:41 PM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -14.413592522064945 (T) = (0 -421.3813120784167) / Math.Sqrt((82.058613008072 / (24)) + (15.272155129017252 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.07704630140098774 = (391.23788042380096 - 421.3813120784167) / 391.23788042380096 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline c50c745fa88f0850c8e8ebed37749118929d7e87
Compare e07f4527bdedff6278accf9db8a8c7f9f2a48beb
Diff Diff

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Exp2 - Duration of single invocation 50.50 μs 69.02 μs 1.37 0.20 False
Pow - Duration of single invocation 51.45 μs 70.20 μs 1.36 0.16 False
Exp - Duration of single invocation 30.01 μs 36.97 μs 1.23 0.20 False
ExpM1 - Duration of single invocation 32.45 μs 39.58 μs 1.22 0.17 False
Exp10M1 - Duration of single invocation 58.47 μs 69.76 μs 1.19 0.24 False
Exp2M1 - Duration of single invocation 52.65 μs 69.77 μs 1.33 0.20 False
Exp10 - Duration of single invocation 50.70 μs 68.95 μs 1.36 0.18 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.Exp2


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.01845049559472 > 53.05348502326052.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -34.245477860947055 (T) = (0 -69741.96585675362) / Math.Sqrt((837127.121138001 / (24)) + (4368036.929151213 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.3744935333590274 = (50740.119297845056 - 69741.96585675362) / 50740.119297845056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Single.Pow

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.19791825485798 > 54.0616930265459.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -47.28875514768354 (T) = (0 -70505.77769355456) / Math.Sqrt((1022034.7805501956 / (24)) + (1760714.395377746 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.35507007642491106 = (52031.09338785662 - 70505.77769355456) / 52031.09338785662 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Single.Exp

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.97051248522459 > 31.51393453035029.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -5.6168754184609115 (T) = (0 -40237.68471020789) / Math.Sqrt((8347150.054725872 / (24)) + (41024205.08871314 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.3126820269740823 = (30653.032404931637 - 40237.68471020789) / 30653.032404931637 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Single.ExpM1

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.58192440025252 > 34.0543378092583.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -8.341714256826993 (T) = (0 -41344.43641041453) / Math.Sqrt((1633576.3321810076 / (24)) + (16106393.203310406 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.26448313274624735 = (32696.708512529764 - 41344.43641041453) / 32696.708512529764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Single.Exp10M1

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.75811953703706 > 55.29261521359428.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -33.38901808404263 (T) = (0 -70294.01947976642) / Math.Sqrt((2452201.92971196 / (24)) + (2501915.2900683894 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.31842638619186886 = (53316.60547450286 - 70294.01947976642) / 53316.60547450286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Single.Exp2M1

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.77374728422619 > 55.27473974431819.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -27.019015458421595 (T) = (0 -70859.34337773589) / Math.Sqrt((3600744.37885578 / (24)) + (4334937.568735229 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.3287225498742918 = (53328.92362249724 - 70859.34337773589) / 53328.92362249724 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Single.Exp10

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.94804320851689 > 53.21810607312452.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -30.56350158178773 (T) = (0 -69028.52617952159) / Math.Sqrt((7620123.501661613 / (24)) + (73040.3255655771 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.3356043500506718 = (51683.364296397136 - 69028.52617952159) / 51683.364296397136 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline c50c745fa88f0850c8e8ebed37749118929d7e87
Compare e07f4527bdedff6278accf9db8a8c7f9f2a48beb
Diff Diff

Regressions in System.Linq.Tests.Perf_OrderBy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OrderByValueType - Duration of single invocation 43.90 μs 50.68 μs 1.15 0.45 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_OrderBy*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_OrderBy.OrderByValueType(NumberOfPeople: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.67796747870075 > 46.19685214061421.
IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 1/5/2023 10:03:03 PM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -7.135680451957544 (T) = (0 -50690.210505066556) / Math.Sqrt((2833638.52332989 / (24)) + (13500456.952154541 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.16017201680390003 = (43691.97823328862 - 50690.210505066556) / 43691.97823328862 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline c50c745fa88f0850c8e8ebed37749118929d7e87
Compare e07f4527bdedff6278accf9db8a8c7f9f2a48beb
Diff Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleParallel - Duration of single invocation 55.07 μs 71.53 μs 1.30 0.35 False
SingleParallel - Duration of single invocation 67.01 μs 83.28 μs 1.24 0.31 False
SingleParallel - Duration of single invocation 57.63 μs 76.16 μs 1.32 0.28 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.53111953722221 > 62.327370546699996.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -9.934707576395398 (T) = (0 -66740.98591333599) / Math.Sqrt((6672800.292791727 / (24)) + (4880891.868175631 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.12824263552769538 = (59154.816359266784 - 66740.98591333599) / 59154.816359266784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.27730550266666 > 68.40260402082059.
IsChangePoint: Marked as a change because one of 10/26/2022 4:56:52 PM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -31.694639522864268 (T) = (0 -83175.24421248147) / Math.Sqrt((6679767.415053553 / (24)) + (651460.5285907873 / (15))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (15) - 2, .025) and -0.2757532381393107 = (65196.96891680461 - 83175.24421248147) / 65196.96891680461 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 76.15692898133334 > 62.118965069350004.
IsChangePoint: Marked as a change because one of 10/26/2022 10:09:37 PM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -21.728974389110206 (T) = (0 -74143.22368359951) / Math.Sqrt((3875769.4388656598 / (24)) + (4132585.9298055647 / (15))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (15) - 2, .025) and -0.2402859508107063 = (59779.13692817063 - 74143.22368359951) / 59779.13692817063 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline c50c745fa88f0850c8e8ebed37749118929d7e87
Compare e07f4527bdedff6278accf9db8a8c7f9f2a48beb
Diff Diff

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentQueue - Duration of single invocation 7.89 μs 8.68 μs 1.10 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.ConcurrentQueue(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.683263982126823 > 8.275400607283967.
IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -69.55931491590971 (T) = (0 -8663.537660373604) / Math.Sqrt((954.6137982490305 / (24)) + (1563.086420817273 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.10392001356575693 = (7847.975898534197 - 8663.537660373604) / 7847.975898534197 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline c50c745fa88f0850c8e8ebed37749118929d7e87
Compare e07f4527bdedff6278accf9db8a8c7f9f2a48beb
Diff Diff

Regressions in System.Buffers.Text.Tests.Base64Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Base64Decode - Duration of single invocation 332.43 ns 352.08 ns 1.06 0.16 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Base64Tests.Base64Decode(NumberOfBytes: 1000)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 352.08118756989154 > 348.6693292218774.
IsChangePoint: Marked as a change because one of 10/26/2022 2:40:44 AM, 12/14/2022 12:26:48 AM, 1/5/2023 10:03:03 PM, 1/21/2023 9:40:41 PM, 1/27/2023 7:12:54 PM, 2/2/2023 2:17:49 AM falls between 1/24/2023 11:54:43 AM and 2/2/2023 2:17:49 AM.
IsRegressionStdDev: Marked as regression because -21.606153075615218 (T) = (0 -351.89829338836716) / Math.Sqrt((14.429385268369273 / (24)) + (2.6282020847723397 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.056768726951027366 = (332.99461311999517 - 351.89829338836716) / 332.99461311999517 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions