Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 19fc478d67c2fc55e62b50676935f903a77f5a4d |
Compare | 8300e381e41bdf013c5dc697e3e9c15f68e8f249 |
Diff | Diff |
Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IterateForEachArray - Duration of single invocation | 122.34 ns | 168.58 ns | 1.38 | 0.48 | False | |||||
IterateGetPositionTenSegments - Duration of single invocation | 567.79 ns | 649.16 ns | 1.14 | 0.30 | False | |||||
SliceTenSegments - Duration of single invocation | 116.74 ns | 123.93 ns | 1.06 | 0.34 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Char>*'
Payloads
Histogram
System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachArray
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 168.57730147382318 > 129.4312299220948.
IsChangePoint: Marked as a change because one of 11/11/2022 3:15:52 PM, 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -7.991065657844798 (T) = (0 -150.12832464926575) / Math.Sqrt((39.48952697299523 / (32)) + (125.02324060651331 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.16277782883613565 = (129.11178810446907 - 150.12832464926575) / 129.11178810446907 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionTenSegments
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 649.1550815292176 > 590.0567236408356.
IsChangePoint: Marked as a change because one of 1/18/2023 3:58:17 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -8.399963335329797 (T) = (0 -611.4948611668126) / Math.Sqrt((221.78149977044868 / (31)) + (367.3784059836182 / (23))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (23) - 2, .025) and -0.07073393605606147 = (571.0987954852642 - 611.4948611668126) / 571.0987954852642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.SliceTenSegments
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.93399282784121 > 123.78641176051869.
IsChangePoint: Marked as a change because one of 1/17/2023 2:19:41 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -5.511648651675112 (T) = (0 -131.6193788119163) / Math.Sqrt((34.47185217413856 / (27)) + (68.24448449170157 / (27))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (27) - 2, .025) and -0.08894142571645364 = (120.86910801957892 - 131.6193788119163) / 120.86910801957892 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 19fc478d67c2fc55e62b50676935f903a77f5a4d |
Compare | 8300e381e41bdf013c5dc697e3e9c15f68e8f249 |
Diff | Diff |
Regressions in PerfLabTests.LowLevelPerf
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
EmptyStaticFunction - Duration of single invocation | 1.86 ms | 2.08 ms | 1.12 | 0.20 | False | |||||
GenericClassGenericInstanceMethod - Duration of single invocation | 288.07 μs | 399.86 μs | 1.39 | 0.06 | False | |||||
IntegerFormatting - Duration of single invocation | 6.43 ms | 16.00 ms | 2.49 | 0.22 | False | |||||
GenericClassGenericStaticMethod - Duration of single invocation | 525.26 μs | 654.43 μs | 1.25 | 0.13 | False | |||||
GenericGenericMethod - Duration of single invocation | 282.59 μs | 407.45 μs | 1.44 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
Payloads
Histogram
PerfLabTests.LowLevelPerf.EmptyStaticFunction
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0777625026041666 > 1.9564735608333337.
IsChangePoint: Marked as a change because one of 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -8.33904228229637 (T) = (0 -2092664.665958521) / Math.Sqrt((13224642035.058285 / (32)) + (4610432699.730079 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.11043203818290226 = (1884549.9715434476 - 2092664.665958521) / 1884549.9715434476 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 399.85781274038453 > 302.60327825.
IsChangePoint: Marked as a change because one of 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -32.49655378639152 (T) = (0 -385545.59940948687) / Math.Sqrt((3722473.27127428 / (32)) + (193594183.73892084 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.33632609820108267 = (288511.61399039906 - 385545.59940948687) / 288511.61399039906 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.LowLevelPerf.IntegerFormatting
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.999866671428572 > 6.750741233177083.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -85.51900747711608 (T) = (0 -15665665.741158683) / Math.Sqrt((88809789056.23221 / (14)) + (190736111824.69205 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -1.3554463402301093 = (6650826.840584391 - 15665665.741158683) / 6650826.840584391 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 654.4288209635417 > 551.8575060757212.
IsChangePoint: Marked as a change because one of 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -12.904246876630882 (T) = (0 -678746.1346516752) / Math.Sqrt((14108803.420922305 / (32)) + (3050603211.164123 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.28904395828650026 = (526550.0298018685 - 678746.1346516752) / 526550.0298018685 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.LowLevelPerf.GenericGenericMethod
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 407.4453983516483 > 296.680255409727.
IsChangePoint: Marked as a change because one of 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -27.281880825488344 (T) = (0 -421779.54325366556) / Math.Sqrt((26778.137048615015 / (32)) + (572677012.4046113 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.49258026122844734 = (282584.1626141604 - 421779.54325366556) / 282584.1626141604 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 19fc478d67c2fc55e62b50676935f903a77f5a4d |
Compare | 8300e381e41bdf013c5dc697e3e9c15f68e8f249 |
Diff | Diff |
Regressions in System.Memory.Span<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Reverse - Duration of single invocation | 24.07 ns | 29.50 ns | 1.23 | 0.33 | False | |||||
Reverse - Duration of single invocation | 13.92 ns | 16.34 ns | 1.17 | 0.49 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Byte>*'
Payloads
Histogram
System.Memory.Span<Byte>.Reverse(Size: 33)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.496475509301526 > 25.27195062677713.
IsChangePoint: Marked as a change because one of 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -6.369546646014871 (T) = (0 -27.27771003379253) / Math.Sqrt((2.5862794141959067 / (32)) + (2.5748570077647295 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.11590645135749313 = (24.44444155745257 - 27.27771003379253) / 24.44444155745257 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Byte>.Reverse(Size: 4)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.335283971111732 > 14.595210687065808.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/11/2023 4:46:06 PM, 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -7.246150159413201 (T) = (0 -16.257344985231487) / Math.Sqrt((0.6651465984984736 / (32)) + (1.7153993588341885 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.16288530149559166 = (13.98017926989261 - 16.257344985231487) / 13.98017926989261 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 19fc478d67c2fc55e62b50676935f903a77f5a4d |
Compare | 8300e381e41bdf013c5dc697e3e9c15f68e8f249 |
Diff | Diff |
Regressions in System.Collections.CtorDefaultSizeNonGeneric
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Hashtable - Duration of single invocation | 33.20 ns | 35.76 ns | 1.08 | 0.55 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSizeNonGeneric*'
Payloads
Histogram
System.Collections.CtorDefaultSizeNonGeneric.Hashtable
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.75823674828893 > 34.60016184970855.
IsChangePoint: Marked as a change because one of 12/5/2022 8:20:45 AM, 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -7.092898541732629 (T) = (0 -37.7738977748661) / Math.Sqrt((2.704561306230093 / (32)) + (5.5736301002327835 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.1225169092093533 = (33.65107239361963 - 37.7738977748661) / 33.65107239361963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 19fc478d67c2fc55e62b50676935f903a77f5a4d |
Compare | 8300e381e41bdf013c5dc697e3e9c15f68e8f249 |
Diff | Diff |
Regressions in System.Linq.Tests.Perf_Enumerable
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Max - Duration of single invocation | 2.20 μs | 2.55 μs | 1.16 | 0.24 | False | |||||
Min - Duration of single invocation | 2.16 μs | 2.53 μs | 1.17 | 0.22 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Payloads
Histogram
System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5542139839806897 > 2.2965649845619986.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -15.545053296030265 (T) = (0 -2599.3393038626728) / Math.Sqrt((5964.80182199081 / (32)) + (8670.259667415621 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.1683460289006842 = (2224.802618029552 - 2599.3393038626728) / 2224.802618029552 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.52573214617615 > 2.2845308787465597.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -13.94699586344723 (T) = (0 -2592.7625435435334) / Math.Sqrt((4919.536191229149 / (32)) + (14450.998808596494 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.18084862529589368 = (2195.6773188382604 - 2592.7625435435334) / 2195.6773188382604 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 19fc478d67c2fc55e62b50676935f903a77f5a4d |
Compare | 8300e381e41bdf013c5dc697e3e9c15f68e8f249 |
Diff | Diff |
Regressions in System.Collections.Concurrent.IsEmpty<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Bag - Duration of single invocation | 46.02 ns | 55.31 ns | 1.20 | 0.65 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty<Int32>*'
Payloads
Histogram
System.Collections.Concurrent.IsEmpty<Int32>.Bag(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.31339062338936 > 50.430543768541376.
IsChangePoint: Marked as a change because one of 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -4.010250998350441 (T) = (0 -56.00902489524304) / Math.Sqrt((20.5846307653969 / (32)) + (29.086145886795244 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.1115770780520157 = (50.38699160061496 - 56.00902489524304) / 50.38699160061496 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 19fc478d67c2fc55e62b50676935f903a77f5a4d |
Compare | 8300e381e41bdf013c5dc697e3e9c15f68e8f249 |
Diff | Diff |
Regressions in System.Tests.Perf_Random
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Next_int_unseeded - Duration of single invocation | 31.54 ns | 36.32 ns | 1.15 | 0.68 | False | |||||
Next_int - Duration of single invocation | 31.08 ns | 39.53 ns | 1.27 | 0.67 | False | |||||
Next_long - Duration of single invocation | 92.72 ns | 101.65 ns | 1.10 | 0.34 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'
Payloads
Histogram
System.Tests.Perf_Random.Next_int_unseeded
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.31676164278097 > 32.450065407653106.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -7.761320066892196 (T) = (0 -36.77698501590825) / Math.Sqrt((4.1244260360284954 / (32)) + (8.120823073221315 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.17499110542221658 = (31.299798650554855 - 36.77698501590825) / 31.299798650554855 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.Next_int
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.527610167892746 > 34.15863407003011.
IsChangePoint: Marked as a change because one of 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -8.133106537691837 (T) = (0 -35.36393237750524) / Math.Sqrt((5.645423626696244 / (32)) + (6.777199158960745 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.1905864814717677 = (29.702951383917444 - 35.36393237750524) / 29.702951383917444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.Next_long
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 101.64717876401252 > 97.65172483793113.
IsChangePoint: Marked as a change because one of 1/18/2023 3:58:17 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -7.849730294836891 (T) = (0 -102.9104190573666) / Math.Sqrt((9.193992187457193 / (31)) + (15.929321628870577 / (23))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (23) - 2, .025) and -0.08209034779825782 = (95.10335182894816 - 102.9104190573666) / 95.10335182894816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 19fc478d67c2fc55e62b50676935f903a77f5a4d |
Compare | 8300e381e41bdf013c5dc697e3e9c15f68e8f249 |
Diff | Diff |
Regressions in PerfLabTests.DictionaryExpansion
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ExpandDictionaries - Duration of single invocation | 1.54 μs | 1.78 μs | 1.16 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.DictionaryExpansion*'
Payloads
Histogram
PerfLabTests.DictionaryExpansion.ExpandDictionaries
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7846032902529425 > 1.619365508920234.
IsChangePoint: Marked as a change because one of 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -19.62660129151795 (T) = (0 -1764.8116436097232) / Math.Sqrt((671.913963459265 / (32)) + (2087.395903005932 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.1359958826298745 = (1553.5370071272757 - 1764.8116436097232) / 1553.5370071272757 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 19fc478d67c2fc55e62b50676935f903a77f5a4d |
Compare | 8300e381e41bdf013c5dc697e3e9c15f68e8f249 |
Diff | Diff |
Regressions in System.Tests.Perf_HashCode
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Add - Duration of single invocation | 7.47 μs | 10.35 μs | 1.38 | 0.20 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_HashCode*'
Payloads
Histogram
System.Tests.Perf_HashCode.Add
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.349680475694125 > 7.89002947445516.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/18/2023 9:06:44 AM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -31.23939466955221 (T) = (0 -9753.146522303574) / Math.Sqrt((121458.9151308833 / (32)) + (33038.97876432847 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.30399245155123705 = (7479.450138458374 - 9753.146522303574) / 7479.450138458374 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository