Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 0e1b3bba7fb779c84e6d5d1027b37bd10126ba9f |
| Compare | 2c110b6229e928c3f38062e9d8bc875804efc539 |
| Diff | Diff |
Improvements in System.IO.Tests.StreamReaderReadLineTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ReadLineAsync - Duration of single invocation | 28.01 μs | 16.68 μs | 0.60 | 0.03 | False | |||||
| ReadLineAsync - Duration of single invocation | 33.13 μs | 23.42 μs | 0.71 | 0.03 | False | |||||
| ReadLineAsync - Duration of single invocation | 26.26 μs | 15.69 μs | 0.60 | 0.02 | False | |||||
| ReadLineAsync - Duration of single invocation | 26.67 μs | 16.13 μs | 0.60 | 0.02 | False | |||||
| ReadLineAsync - Duration of single invocation | 58.69 μs | 52.16 μs | 0.89 | 0.07 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'Payloads
Histogram
System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048])
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.675306514808725 < 26.6192188817723.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 6/8/2022 3:33:18 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 95.78145479416212 (T) = (0 -16656.97803004773) / Math.Sqrt((3532798.3203992774 / (299)) + (8011.615508344169 / (38))) is greater than 1.967070609662329 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (38) - 2, .975) and 0.3867264409324027 = (27160.76338815666 - 16656.97803004773) / 27160.76338815666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 33, 128])
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.42084917664671 < 31.515455175799435.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/20/2022 10:18:01 PM, 5/27/2022 3:44:45 PM, 6/8/2022 3:33:18 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 72.76077874700333 (T) = (0 -23746.56405787241) / Math.Sqrt((2028238.5669226078 / (299)) + (318925.56200608757 / (38))) is greater than 1.967070609662329 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (38) - 2, .975) and 0.2740292771229695 = (32710.085006960744 - 23746.56405787241) / 32710.085006960744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024])
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.690589296736363 < 25.00336884282742.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 6/8/2022 3:33:18 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 137.61562195956637 (T) = (0 -15784.293468503518) / Math.Sqrt((1553141.3086999303 / (299)) + (8455.597990133645 / (38))) is greater than 1.967070609662329 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (38) - 2, .975) and 0.39086911564064825 = (25912.810980031845 - 15784.293468503518) / 25912.810980031845 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 1024])
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.13263356695816 < 25.32692240017361.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 6/8/2022 3:33:18 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 97.18606939774574 (T) = (0 -16038.43687863541) / Math.Sqrt((2972877.6614809292 / (299)) + (6361.955660164876 / (38))) is greater than 1.967070609662329 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (38) - 2, .975) and 0.3786062377733955 = (25810.42465113554 - 16038.43687863541) / 25810.42465113554 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 9, 32])
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.159778118008944 < 55.902286652040885.
IsChangePoint: Marked as a change because one of 4/14/2022 8:50:18 PM, 5/27/2022 3:44:45 PM, 6/8/2022 3:33:18 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 28.260720516258715 (T) = (0 -51621.1856149446) / Math.Sqrt((2506880.1868202006 / (299)) + (1114749.9519489673 / (38))) is greater than 1.967070609662329 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (38) - 2, .975) and 0.09610732024388564 = (57109.86134866462 - 51621.1856149446) / 57109.86134866462 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 0e1b3bba7fb779c84e6d5d1027b37bd10126ba9f |
| Compare | 2c110b6229e928c3f38062e9d8bc875804efc539 |
| Diff | Diff |
Improvements in System.IO.Tests.Perf_File
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ReadAllLinesAsync - Duration of single invocation | 30.83 μs | 27.74 μs | 0.90 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'Payloads
Histogram
System.IO.Tests.Perf_File.ReadAllLinesAsync
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.74493009309595 < 29.31967355873113.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 6/8/2022 3:33:18 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 70.15658300362523 (T) = (0 -27762.00036367639) / Math.Sqrt((220998.1390770169 / (299)) + (48427.392281225366 / (38))) is greater than 1.967070609662329 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (38) - 2, .975) and 0.1018467325991194 = (30910.092265227082 - 27762.00036367639) / 30910.092265227082 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository