Open
Description
Run Information
Architecture | x86 |
---|---|
OS | Windows 10.0.18362 |
Baseline | e013f9dfc31c49c34428ea3400eb0a0a7a8f1e24 |
Compare | 011b9496d03aff0120161f0609f4c2a850f39a40 |
Diff | Diff |
Regressions in System.Net.Http.Tests.SocketsHttpHandlerPerfTest
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Get_EnumerateHeaders_Unvalidated - Duration of single invocation | 36.75 μs | 51.52 μs | 1.40 | 0.44 | False | |||||
Get - Duration of single invocation | 55.05 μs | 73.33 μs | 1.33 | 0.27 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Http.Tests.SocketsHttpHandlerPerfTest*'
Payloads
Histogram
Edge Detector Info
System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Unvalidated(ssl: False, chunkedResponse: False, responseLength: 1)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.52245506535948 > 38.363666913216655.
IsChangePoint: Marked as a change because one of 9/2/2022 11:09:40 AM, 9/13/2022 12:23:36 PM, 9/22/2022 3:11:11 PM, 9/27/2022 3:32:36 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 3:32:36 AM.
IsRegressionStdDev: Marked as regression because -8.96045248747586 (T) = (0 -48887.48441540962) / Math.Sqrt((2253962.57402785 / (35)) + (40030642.70745127 / (23))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (23) - 2, .025) and -0.32667784985179654 = (36849.55200003592 - 48887.48441540962) / 36849.55200003592 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get(ssl: True, chunkedResponse: True, responseLength: 1)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 73.33093996840442 > 57.7645757995595.
IsChangePoint: Marked as a change because one of 8/13/2022 7:54:18 PM, 9/2/2022 11:09:40 AM, 9/22/2022 3:11:11 PM, 9/27/2022 3:32:36 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 3:32:36 AM.
IsRegressionStdDev: Marked as regression because -39.89770487921154 (T) = (0 -73622.40186663314) / Math.Sqrt((3112491.070022479 / (35)) + (2405533.7651674664 / (23))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (23) - 2, .025) and -0.3130166466317612 = (56071.18695371783 - 73622.40186663314) / 56071.18695371783 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x86 |
---|---|
OS | Windows 10.0.18362 |
Baseline | e013f9dfc31c49c34428ea3400eb0a0a7a8f1e24 |
Compare | 011b9496d03aff0120161f0609f4c2a850f39a40 |
Diff | Diff |
Regressions in System.Collections.ContainsFalse<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Span - Duration of single invocation | 21.88 μs | 26.03 μs | 1.19 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse<Int32>*'
Payloads
Histogram
Edge Detector Info
System.Collections.ContainsFalse<Int32>.Span(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.030177777777777 > 22.890186888415403.
IsChangePoint: Marked as a change because one of 8/17/2022 2:11:12 PM, 9/13/2022 6:36:50 PM, 9/22/2022 3:11:11 PM, 9/27/2022 3:32:36 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 3:32:36 AM.
IsRegressionStdDev: Marked as regression because -18.47173386739818 (T) = (0 -26053.181093988405) / Math.Sqrt((1520485.061950293 / (35)) + (1969.3783705020223 / (23))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (23) - 2, .025) and -0.17360089970626924 = (22199.353375162747 - 26053.181093988405) / 22199.353375162747 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x86 |
---|---|
OS | Windows 10.0.18362 |
Baseline | e013f9dfc31c49c34428ea3400eb0a0a7a8f1e24 |
Compare | 011b9496d03aff0120161f0609f4c2a850f39a40 |
Diff | Diff |
Regressions in System.Collections.Tests.Perf_BitArray
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
BitArraySetAll - Duration of single invocation | 9.86 ns | 12.22 ns | 1.24 | 0.29 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'
Payloads
Histogram
Edge Detector Info
System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.223696591978399 > 9.779698143501854.
IsChangePoint: Marked as a change because one of 9/22/2022 3:11:11 PM, 9/27/2022 3:32:36 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 3:32:36 AM.
IsRegressionStdDev: Marked as regression because -8.083244116459822 (T) = (0 -11.95869735018357) / Math.Sqrt((1.3963728310277783 / (35)) + (0.6438112191327675 / (23))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (23) - 2, .025) and -0.21376311557685285 = (9.852579302098896 - 11.95869735018357) / 9.852579302098896 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x86 |
---|---|
OS | Windows 10.0.18362 |
Baseline | e013f9dfc31c49c34428ea3400eb0a0a7a8f1e24 |
Compare | 011b9496d03aff0120161f0609f4c2a850f39a40 |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
DotBenchmark - Duration of single invocation | 32.73 ns | 35.43 ns | 1.08 | 0.09 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int64>*'
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<Int64>.DotBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.43270041553871 > 34.31409092453629.
IsChangePoint: Marked as a change because one of 8/26/2022 6:25:06 PM, 9/13/2022 12:23:36 PM, 9/22/2022 3:11:11 PM, 9/27/2022 3:32:36 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 3:32:36 AM.
IsRegressionStdDev: Marked as regression because -12.45357883145386 (T) = (0 -35.64003329631755) / Math.Sqrt((0.5275154685890452 / (35)) + (0.7395035494308355 / (23))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (23) - 2, .025) and -0.08217421293960321 = (32.93372995786459 - 35.64003329631755) / 32.93372995786459 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x86 |
---|---|
OS | Windows 10.0.18362 |
Baseline | e013f9dfc31c49c34428ea3400eb0a0a7a8f1e24 |
Compare | 011b9496d03aff0120161f0609f4c2a850f39a40 |
Diff | Diff |
Regressions in System.Tests.Perf_Int64
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ParseSpan - Duration of single invocation | 0.41 ns | 90.97 ns | 219.53 | 0.50 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int64*'
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.97053517886009 > 0.34195371954454246.
IsChangePoint: Marked as a change because one of 8/13/2022 2:59:44 AM, 9/13/2022 12:23:36 PM, 9/22/2022 3:11:11 PM, 9/27/2022 3:32:36 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 3:32:36 AM.
IsRegressionStdDev: Marked as regression because -397.67415731456595 (T) = (0 -90.5828921460029) / Math.Sqrt((1.622910537503068 / (35)) + (0.10782617365513492 / (23))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (23) - 2, .025) and -123.89375901500135 = (0.7252795724974755 - 90.5828921460029) / 0.7252795724974755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository