-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | f1e10a53132cc648cea914e75f953ff5a5087399 |
| Compare | de0b85f2b1690355f05bf13d3a88b8828b9853b1 |
| Diff | Diff |
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Count - Duration of single invocation | 4.56 ms | 8.34 ms | 1.83 | 0.08 | True | |||||
| Count - Duration of single invocation | 3.31 ms | 6.74 ms | 2.04 | 0.07 | True | |||||
| Count - Duration of single invocation | 4.51 ms | 8.29 ms | 1.84 | 0.06 | True |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: NonBacktracking)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.340815666666668 > 4.726036502497475.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -57.2571338950626 (T) = (0 -8390487.941977082) / Math.Sqrt((7818886183.785275 / (23)) + (148372360025.30594 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.8579772658824698 = (4515926.053590281 - 8390487.941977082) / 4515926.053590281 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: NonBacktracking)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.738108863523573 > 3.4824430852604173.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -230.11277573533923 (T) = (0 -6760860.29769661) / Math.Sqrt((4163092571.808942 / (23)) + (1425926982.3002477 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -1.027713587578754 = (3334228.433005471 - 6760860.29769661) / 3334228.433005471 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.285871557692309 > 4.731868686555944.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -72.49287773610047 (T) = (0 -8355328.40275375) / Math.Sqrt((1522638542.8598952 / (23)) + (96277890522.02982 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.8535516993334773 = (4507739.603787831 - 8355328.40275375) / 4507739.603787831 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | f1e10a53132cc648cea914e75f953ff5a5087399 |
| Compare | de0b85f2b1690355f05bf13d3a88b8828b9853b1 |
| Diff | Diff |
Regressions in System.Threading.Channels.Tests.BoundedChannelPerfTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| PingPong - Duration of single invocation | 10.68 ms | 11.48 ms | 1.08 | 0.12 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'Payloads
Histogram
System.Threading.Channels.Tests.BoundedChannelPerfTests.PingPong
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.483906828054298 > 11.177547596358696.
IsChangePoint: Marked as a change because one of 3/4/2022 11:33:07 AM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -13.454632532900792 (T) = (0 -11457107.121401245) / Math.Sqrt((51819395511.96745 / (23)) + (78626422250.46277 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.0855089872305328 = (10554594.44019146 - 11457107.121401245) / 10554594.44019146 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | f1e10a53132cc648cea914e75f953ff5a5087399 |
| Compare | de0b85f2b1690355f05bf13d3a88b8828b9853b1 |
| Diff | Diff |
Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| SingleSerial - Duration of single invocation | 320.73 ns | 377.65 ns | 1.18 | 0.11 | False | |||||
| SingleSerial - Duration of single invocation | 304.77 ns | 363.47 ns | 1.19 | 0.18 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Byte>*'Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 377.64991607142855 > 337.06842618571426.
IsChangePoint: Marked as a change because one of 2/23/2022 3:28:40 AM, 4/8/2022 10:19:59 AM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -30.00621472037777 (T) = (0 -375.3227486811438) / Math.Sqrt((39.06110740213733 / (23)) + (36.12035619091378 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.15221060159970629 = (325.74144705842247 - 375.3227486811438) / 325.74144705842247 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 363.46922040816327 > 322.73015333571436.
IsChangePoint: Marked as a change because one of 3/4/2022 3:52:45 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -25.614525912904604 (T) = (0 -364.7889477103773) / Math.Sqrt((65.04840047003277 / (23)) + (54.09452180995144 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.17212572413881427 = (311.2199828038033 - 364.7889477103773) / 311.2199828038033 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | f1e10a53132cc648cea914e75f953ff5a5087399 |
| Compare | de0b85f2b1690355f05bf13d3a88b8828b9853b1 |
| Diff | Diff |
Regressions in System.Threading.Channels.Tests.UnboundedChannelPerfTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| PingPong - Duration of single invocation | 11.24 ms | 12.22 ms | 1.09 | 0.08 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Channels.Tests.UnboundedChannelPerfTests*'Payloads
Histogram
System.Threading.Channels.Tests.UnboundedChannelPerfTests.PingPong
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.221611837301584 > 11.713151503260871.
IsChangePoint: Marked as a change because one of 3/4/2022 3:52:45 PM, 4/8/2022 7:02:49 PM, 4/13/2022 1:59:07 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -16.9173345628046 (T) = (0 -12121900.066502059) / Math.Sqrt((26176971242.8094 / (23)) + (84054772863.84938 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.09055028050787453 = (11115397.687905625 - 12121900.066502059) / 11115397.687905625 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | f1e10a53132cc648cea914e75f953ff5a5087399 |
| Compare | de0b85f2b1690355f05bf13d3a88b8828b9853b1 |
| Diff | Diff |
Regressions in System.Threading.Tasks.Tests.Perf_AsyncMethods
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Yield - Duration of single invocation | 286.02 ns | 335.84 ns | 1.17 | 0.07 | True |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tasks.Tests.Perf_AsyncMethods*'Payloads
Histogram
System.Threading.Tasks.Tests.Perf_AsyncMethods.Yield
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 335.8411737333334 > 298.34728274749995.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -36.33611859898587 (T) = (0 -335.5713782329692) / Math.Sqrt((24.539556620704968 / (23)) + (30.74939443819776 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.1779003623284728 = (284.88944308465267 - 335.5713782329692) / 284.88944308465267 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | f1e10a53132cc648cea914e75f953ff5a5087399 |
| Compare | de0b85f2b1690355f05bf13d3a88b8828b9853b1 |
| Diff | Diff |
Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Object>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| SingleSerial - Duration of single invocation | 345.52 ns | 385.09 ns | 1.11 | 0.12 | False | |||||
| SingleSerial - Duration of single invocation | 341.15 ns | 384.71 ns | 1.13 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Object>*'Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 385.0914007857143 > 357.5385098666667.
IsChangePoint: Marked as a change because one of 3/4/2022 3:52:45 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -21.419046122139918 (T) = (0 -381.2374003492809) / Math.Sqrt((76.94792665571985 / (23)) + (34.07846515728208 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.13219986600310113 = (336.72270400024644 - 381.2374003492809) / 336.72270400024644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 384.7131822222222 > 359.7747919428571.
IsChangePoint: Marked as a change because one of 3/4/2022 11:33:07 AM, 4/8/2022 10:19:59 AM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -20.754947919742296 (T) = (0 -381.1052267487689) / Math.Sqrt((45.19552534822438 / (23)) + (39.32008869080374 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.1058372434970825 = (344.630486077289 - 381.1052267487689) / 344.630486077289 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository




_1.png)
_2.png)


_1.png)
_2.png)