Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 9053aa1e66ab7c1af7a14e649fb0bf0b0c501cde |
| Compare | 136b312bc4b471c92eeaf25ded3d0b4dec3afd13 |
| Diff | Diff |
Regressions in System.Net.Http.Tests.SocketsHttpHandlerPerfTest
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Get - Duration of single invocation | 121.45 μs | 202.73 μs | 1.67 | 0.54 | False | |||||
| Get_EnumerateHeaders_Unvalidated - Duration of single invocation | 134.39 μs | 183.62 μs | 1.37 | 0.47 | False | |||||
| Get_EnumerateHeaders_Validated - Duration of single invocation | 48.81 μs | 54.92 μs | 1.13 | 0.00 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Http.Tests.SocketsHttpHandlerPerfTest*'Payloads
Histogram
System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get(ssl: False, chunkedResponse: True, responseLength: 1)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 202.73157894736843 > 125.80243421052631.
IsChangePoint: Marked as a change because one of 3/4/2022 3:52:45 PM, 3/8/2022 2:08:40 PM falls between 2/27/2022 3:52:50 PM and 3/8/2022 2:08:40 PM.
IsRegressionStdDev: Marked as regression because -18.274347728716307 (T) = (0 -184167.39422084624) / Math.Sqrt((181388575.9425565 / (46)) + (145445285.78867027 / (17))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (17) - 2, .025) and -0.5403656486044143 = (119560.8292016273 - 184167.39422084624) / 119560.8292016273 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Unvalidated(ssl: False, chunkedResponse: True, responseLength: 1)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 183.62 > 127.86513157894737.
IsChangePoint: Marked as a change because one of 3/4/2022 3:52:45 PM, 3/8/2022 2:08:40 PM falls between 2/27/2022 3:52:50 PM and 3/8/2022 2:08:40 PM.
IsRegressionStdDev: Marked as regression because -12.829534994295162 (T) = (0 -179896.05263157896) / Math.Sqrt((234842204.29634947 / (46)) + (252142676.98554683 / (17))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (17) - 2, .025) and -0.46721217324052555 = (122610.79611563987 - 179896.05263157896) / 122610.79611563987 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Validated(ssl: True, chunkedResponse: False, responseLength: 1)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.91753515430563 > 52.055332915890716.
IsChangePoint: Marked as a change because one of 3/4/2022 3:52:45 PM, 3/8/2022 2:08:40 PM falls between 2/27/2022 3:52:50 PM and 3/8/2022 2:08:40 PM.
IsRegressionStdDev: Marked as regression because -35.43027244872814 (T) = (0 -55389.85383918084) / Math.Sqrt((435095.83502289344 / (46)) + (291375.36021432554 / (17))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (17) - 2, .025) and -0.11647133480658584 = (49611.532434709945 - 55389.85383918084) / 49611.532434709945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository