Skip to content

[Perf] Changes at 6/7/2022 6:41:39 PM #5980

Closed
@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline bab1a1f239325746873d2ae795ea343e7901dbce
Compare 31cd7eb907133929846e5d229d18cc8bc77bde63
Diff Diff

Improvements in System.Threading.Channels.Tests.UnboundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
PingPong - Duration of single invocation 55.35 ms 51.35 ms 0.93 0.52 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Channels.Tests.UnboundedChannelPerfTests*'

Payloads

Baseline
Compare

Histogram

System.Threading.Channels.Tests.UnboundedChannelPerfTests.PingPong


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.34978317857143 < 55.93948785865384.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 10.659402079034132 (T) = (0 -50257771.63543956) / Math.Sqrt((16834023031200.523 / (299)) + (6231554032261.666 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1948910114477687 = (62423562.95861811 - 50257771.63543956) / 62423562.95861811 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 18.04
Baseline bab1a1f239325746873d2ae795ea343e7901dbce
Compare 31cd7eb907133929846e5d229d18cc8bc77bde63
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleParallel - Duration of single invocation 52.89 μs 14.13 μs 0.27 0.04 False
SingleSerial - Duration of single invocation 1.48 μs 1.36 μs 0.92 0.31 False
SingleParallel - Duration of single invocation 57.84 μs 9.11 μs 0.16 0.03 False
SingleSerial - Duration of single invocation 1.62 μs 1.04 μs 0.64 0.62 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.130569566499998 < 50.18482271276667.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 89.17703727092004 (T) = (0 -13564.354656289472) / Math.Sqrt((631593.5427050352 / (299)) + (950881.2049405216 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.7424610106922827 = (52669.130576117415 - 13564.354656289472) / 52669.130576117415 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3603381203333336 < 1.4476956044624998.
IsChangePoint: Marked as a change because one of 3/4/2022 4:48:09 PM, 4/19/2022 4:53:57 AM, 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 7.75136539293671 (T) = (0 -1306.6759323552633) / Math.Sqrt((5427.221642237763 / (299)) + (2123.517431842601 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1109817656628146 = (1469.7965484695214 - 1306.6759323552633) / 1469.7965484695214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.114894403529412 < 54.70560359812381.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 115.9224806266954 (T) = (0 -9789.273519676643) / Math.Sqrt((566337.7800032693 / (299)) + (832650.0357085539 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8293460181574484 = (57363.28806384604 - 9789.273519676643) / 57363.28806384604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0410420100000002 < 1.30779533661875.
IsChangePoint: Marked as a change because one of 5/29/2022 5:15:01 PM, 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 9.58536911478917 (T) = (0 -1085.2608078519231) / Math.Sqrt((10189.146495006533 / (299)) + (2576.86609849191 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.17152206840239936 = (1309.945342489876 - 1085.2608078519231) / 1309.945342489876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline bab1a1f239325746873d2ae795ea343e7901dbce
Compare 31cd7eb907133929846e5d229d18cc8bc77bde63
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleParallel - Duration of single invocation 52.85 μs 13.14 μs 0.25 0.05 False
SingleParallel - Duration of single invocation 50.24 μs 6.89 μs 0.14 0.13 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.141911763333331 < 50.25820595876236.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 125.45379070722589 (T) = (0 -13189.190156333332) / Math.Sqrt((733408.5774327392 / (299)) + (486062.24365584325 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.7501797472985337 = (52794.719458131076 - 13189.190156333332) / 52794.719458131076 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.8876041836842115 < 47.51520831210416.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 76.3185501223648 (T) = (0 -7662.260730092397) / Math.Sqrt((1381913.07345562 / (299)) + (1523850.8852830054 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8471001493123216 = (50112.937950107946 - 7662.260730092397) / 50112.937950107946 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline bab1a1f239325746873d2ae795ea343e7901dbce
Compare 31cd7eb907133929846e5d229d18cc8bc77bde63
Diff Diff

Improvements in System.Threading.Tasks.Tests.Perf_AsyncMethods

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Yield - Duration of single invocation 1.59 μs 857.32 ns 0.54 0.57 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tasks.Tests.Perf_AsyncMethods*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tasks.Tests.Perf_AsyncMethods.Yield


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 857.3235512857143 < 1.50769502470375.
IsChangePoint: Marked as a change because one of 4/19/2022 6:32:34 PM, 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 46.68779610722028 (T) = (0 -841.103379947619) / Math.Sqrt((14738.04378794372 / (293)) + (738.7624122829625 / (5))) is greater than 1.9680107275502179 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (293) + (5) - 2, .975) and 0.4385707785820341 = (1498.1467794342766 - 841.103379947619) / 1498.1467794342766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline bab1a1f239325746873d2ae795ea343e7901dbce
Compare 31cd7eb907133929846e5d229d18cc8bc77bde63
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 29.06 ms 10.44 ms 0.36 0.59 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:?[^\s#])?(?:#[^\s])?", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.4391839625 < 27.717872212083336.
IsChangePoint: Marked as a change because one of 5/27/2022 6:33:44 PM, 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 2.942289278587551 (T) = (0 -12487041.667146962) / Math.Sqrt((201692538688928.72 / (299)) + (20216386412992.637 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.3385352281217428 = (18877863.490280032 - 12487041.667146962) / 18877863.490280032 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions