[Perf] Changes at 6/3/2022 1:48:25 AM

[performanceautofiler]

Run Information

Architecture	arm64
OS	ubuntu 18.04
Baseline	b09ae1ad36858f526b68a005c6f6b2cf8f7ab4b6
Compare	625a9cdd4437a414f1fda220f5bacba101f209c8
Diff	Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BitArrayOr - Duration of single invocation	17.81 ns	10.65 ns	0.60	0.80	False
BitArrayXor - Duration of single invocation	19.64 ns	8.09 ns	0.41	0.74	False
BitArrayBoolArrayCtor - Duration of single invocation	206.05 ns	186.63 ns	0.91	0.23	False
BitArrayNot - Duration of single invocation	11.82 ns	6.16 ns	0.52	0.77	False
BitArrayAnd - Duration of single invocation	19.91 ns	11.55 ns	0.58	0.74	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 4)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.65282861822423 < 17.240126985292058.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 6/2/2022 10:27:31 PM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 3.9177599353272106 (T) = (0 -10.553501216006573) / Math.Sqrt((10.556584896870353 / (299)) + (1.3773654891893679 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10154726608732029 = (11.746306530836673 - 10.553501216006573) / 11.746306530836673 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.087192122695013 < 18.65576840846393.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 6/2/2022 10:27:31 PM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 4.797580659460228 (T) = (0 -9.789998882110291) / Math.Sqrt((12.72572692564954 / (250)) + (2.140529827857361 / (24))) is greater than 1.9687238473654567 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (250) + (24) - 2, .975) and 0.1549911879043717 = (11.58567667221247 - 9.789998882110291) / 11.58567667221247 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayBoolArrayCtor(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 186.62645566046677 < 195.11632115013106.
IsChangePoint: Marked as a change because one of 6/6/2022 8:15:49 PM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 24.370246364708514 (T) = (0 -187.4430501535247) / Math.Sqrt((38.00169604555384 / (299)) + (3.8784963741131544 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.09231433130304312 = (206.50656567334775 - 187.4430501535247) / 206.50656567334775 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.160090312455986 < 13.368117333194617.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 6/2/2022 10:27:31 PM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 4.443662974861718 (T) = (0 -6.217083632802981) / Math.Sqrt((10.999836628758265 / (299)) + (1.0538594477771053 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.16877515213611005 = (7.479424669245458 - 6.217083632802981) / 7.479424669245458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.553999801056401 < 16.27177116714739.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 6/2/2022 10:27:31 PM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 4.482333490349224 (T) = (0 -10.374085814688454) / Math.Sqrt((10.749912417678319 / (299)) + (1.12088477387492 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1104946341144484 = (11.662757991752507 - 10.374085814688454) / 11.662757991752507 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[AndyAyersMS]

Likely from dotnet/runtime#70171