Closed
Description
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bab1a1f239325746873d2ae795ea343e7901dbce |
Compare | 31cd7eb907133929846e5d229d18cc8bc77bde63 |
Diff | Diff |
Improvements in System.Threading.Channels.Tests.UnboundedChannelPerfTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
PingPong - Duration of single invocation | 55.35 ms | 51.35 ms | 0.93 | 0.52 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Channels.Tests.UnboundedChannelPerfTests*'
Payloads
Histogram
System.Threading.Channels.Tests.UnboundedChannelPerfTests.PingPong
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.34978317857143 < 55.93948785865384.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 10.659402079034132 (T) = (0 -50257771.63543956) / Math.Sqrt((16834023031200.523 / (299)) + (6231554032261.666 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1948910114477687 = (62423562.95861811 - 50257771.63543956) / 62423562.95861811 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | arm64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bab1a1f239325746873d2ae795ea343e7901dbce |
Compare | 31cd7eb907133929846e5d229d18cc8bc77bde63 |
Diff | Diff |
Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SingleParallel - Duration of single invocation | 52.89 μs | 14.13 μs | 0.27 | 0.04 | False | |||||
SingleSerial - Duration of single invocation | 1.48 μs | 1.36 μs | 0.92 | 0.31 | False | |||||
SingleParallel - Duration of single invocation | 57.84 μs | 9.11 μs | 0.16 | 0.03 | False | |||||
SingleSerial - Duration of single invocation | 1.62 μs | 1.04 μs | 0.64 | 0.62 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Byte>*'
Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.130569566499998 < 50.18482271276667.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 89.17703727092004 (T) = (0 -13564.354656289472) / Math.Sqrt((631593.5427050352 / (299)) + (950881.2049405216 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.7424610106922827 = (52669.130576117415 - 13564.354656289472) / 52669.130576117415 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3603381203333336 < 1.4476956044624998.
IsChangePoint: Marked as a change because one of 3/4/2022 4:48:09 PM, 4/19/2022 4:53:57 AM, 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 7.75136539293671 (T) = (0 -1306.6759323552633) / Math.Sqrt((5427.221642237763 / (299)) + (2123.517431842601 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1109817656628146 = (1469.7965484695214 - 1306.6759323552633) / 1469.7965484695214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.114894403529412 < 54.70560359812381.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 115.9224806266954 (T) = (0 -9789.273519676643) / Math.Sqrt((566337.7800032693 / (299)) + (832650.0357085539 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8293460181574484 = (57363.28806384604 - 9789.273519676643) / 57363.28806384604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0410420100000002 < 1.30779533661875.
IsChangePoint: Marked as a change because one of 5/29/2022 5:15:01 PM, 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 9.58536911478917 (T) = (0 -1085.2608078519231) / Math.Sqrt((10189.146495006533 / (299)) + (2576.86609849191 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.17152206840239936 = (1309.945342489876 - 1085.2608078519231) / 1309.945342489876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bab1a1f239325746873d2ae795ea343e7901dbce |
Compare | 31cd7eb907133929846e5d229d18cc8bc77bde63 |
Diff | Diff |
Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SingleParallel - Duration of single invocation | 52.85 μs | 13.14 μs | 0.25 | 0.05 | False | |||||
SingleParallel - Duration of single invocation | 50.24 μs | 6.89 μs | 0.14 | 0.13 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Object>*'
Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.141911763333331 < 50.25820595876236.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 125.45379070722589 (T) = (0 -13189.190156333332) / Math.Sqrt((733408.5774327392 / (299)) + (486062.24365584325 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.7501797472985337 = (52794.719458131076 - 13189.190156333332) / 52794.719458131076 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.8876041836842115 < 47.51520831210416.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 76.3185501223648 (T) = (0 -7662.260730092397) / Math.Sqrt((1381913.07345562 / (299)) + (1523850.8852830054 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8471001493123216 = (50112.937950107946 - 7662.260730092397) / 50112.937950107946 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bab1a1f239325746873d2ae795ea343e7901dbce |
Compare | 31cd7eb907133929846e5d229d18cc8bc77bde63 |
Diff | Diff |
Improvements in System.Threading.Tasks.Tests.Perf_AsyncMethods
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Yield - Duration of single invocation | 1.59 μs | 857.32 ns | 0.54 | 0.57 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tasks.Tests.Perf_AsyncMethods*'
Payloads
Histogram
System.Threading.Tasks.Tests.Perf_AsyncMethods.Yield
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 857.3235512857143 < 1.50769502470375.
IsChangePoint: Marked as a change because one of 4/19/2022 6:32:34 PM, 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 46.68779610722028 (T) = (0 -841.103379947619) / Math.Sqrt((14738.04378794372 / (293)) + (738.7624122829625 / (5))) is greater than 1.9680107275502179 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (293) + (5) - 2, .975) and 0.4385707785820341 = (1498.1467794342766 - 841.103379947619) / 1498.1467794342766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 18.04 |
Baseline | bab1a1f239325746873d2ae795ea343e7901dbce |
Compare | 31cd7eb907133929846e5d229d18cc8bc77bde63 |
Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Count - Duration of single invocation | 29.06 ms | 10.44 ms | 0.36 | 0.59 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:?[^\s#])?(?:#[^\s])?", Options: NonBacktracking)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.4391839625 < 27.717872212083336.
IsChangePoint: Marked as a change because one of 5/27/2022 6:33:44 PM, 6/7/2022 9:37:06 AM, 6/9/2022 12:28:12 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 12:28:12 AM.
IsImprovementStdDev: Marked as improvement because 2.942289278587551 (T) = (0 -12487041.667146962) / Math.Sqrt((201692538688928.72 / (299)) + (20216386412992.637 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.3385352281217428 = (18877863.490280032 - 12487041.667146962) / 18877863.490280032 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository