-
Notifications
You must be signed in to change notification settings - Fork 5.1k
Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Regressions in System.Collections.ContainsTrue<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ImmutableHashSet - Duration of single invocation | 32.29 μs | 36.41 μs | 1.13 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<String>*'
Payloads
Histogram
System.Collections.ContainsTrue<String>.ImmutableHashSet(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.41235108981795 > 33.81515342706716.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -18.599642750176685 (T) = (0 -36102.01133372504) / Math.Sqrt((672170.3616818737 / (26)) + (321371.36263329774 / (27))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (27) - 2, .025) and -0.11124265742351962 = (32487.96389568921 - 36102.01133372504) / 32487.96389568921 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Regressions in System.Hashing
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetStringHashCode - Duration of single invocation | 29.31 ns | 35.29 ns | 1.20 | 0.00 | False | |||||
GetStringHashCode - Duration of single invocation | 300.60 ns | 367.64 ns | 1.22 | 0.00 | False | |||||
GetStringHashCode - Duration of single invocation | 2.99 μs | 3.67 μs | 1.23 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Hashing*'
Payloads
Histogram
System.Hashing.GetStringHashCode(BytesCount: 100)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.29313893613284 > 30.78401297711811.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -135.42838161529687 (T) = (0 -35.40427065306296) / Math.Sqrt((0.009162700930984043 / (28)) + (0.04616867251241285 / (27))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (27) - 2, .025) and -0.20868019077850827 = (29.291677751630186 - 35.40427065306296) / 29.291677751630186 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Hashing.GetStringHashCode(BytesCount: 1000)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 367.6354811368269 > 315.7770026743071.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -530.7547613004574 (T) = (0 -367.9386961273943) / Math.Sqrt((0.1649982330629939 / (28)) + (0.2721170652369591 / (27))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (27) - 2, .025) and -0.22294277091957193 = (300.8633804268116 - 367.9386961273943) / 300.8633804268116 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Hashing.GetStringHashCode(BytesCount: 10000)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6740150402414486 > 3.1479971791020867.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -351.7087743229655 (T) = (0 -3670.848951389847) / Math.Sqrt((10.379123507797555 / (28)) + (88.93952000952916 / (27))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (27) - 2, .025) and -0.224614397896344 = (2997.5549509263255 - 3670.848951389847) / 2997.5549509263255 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Regressions in Microsoft.Extensions.Logging.LoggingOverhead
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TwoArguments_FilteredByLevel - Duration of single invocation | 52.71 ns | 56.03 ns | 1.06 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.LoggingOverhead*'
Payloads
Histogram
Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_FilteredByLevel
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.02761713687869 > 55.54323764013431.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -8.505790559807027 (T) = (0 -57.91890805119079) / Math.Sqrt((2.6196181816810222 / (28)) + (2.0835384659087244 / (27))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (27) - 2, .025) and -0.06459967498995105 = (54.40440140256242 - 57.91890805119079) / 54.40440140256242 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Regressions in System.Globalization.Tests.StringHash
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetHashCode - Duration of single invocation | 78.46 μs | 96.20 μs | 1.23 | 0.00 | False | |||||
GetHashCode - Duration of single invocation | 79.85 ns | 97.32 ns | 1.22 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringHash*'
Payloads
Histogram
System.Globalization.Tests.StringHash.GetHashCode(Count: 131072, Options: (en-US, Ordinal))
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.20395003834356 > 82.53206515389446.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -927.8163766760983 (T) = (0 -96218.72445464086) / Math.Sqrt((4799.708115284992 / (28)) + (5107.2703618762525 / (27))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (27) - 2, .025) and -0.22414805808620092 = (78600.56127938196 - 96218.72445464086) / 78600.56127938196 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringHash.GetHashCode(Count: 128, Options: (en-US, Ordinal))
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 97.32120498547347 > 83.84882269177078.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -240.165019247504 (T) = (0 -97.35952570185772) / Math.Sqrt((0.022850832525283242 / (27)) + (0.12000637324947573 / (27))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (27) - 2, .025) and -0.2186684111980267 = (79.89008725199274 - 97.35952570185772) / 79.89008725199274 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Regressions in System.Text.Json.Document.Tests.Perf_EnumerateArray
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
EnumerateArray - Duration of single invocation | 555.94 ns | 666.03 ns | 1.20 | 0.03 | False | |||||
EnumerateArray - Duration of single invocation | 555.50 ns | 666.11 ns | 1.20 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'
Payloads
Histogram
System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfNumbers)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 666.0286184872873 > 583.6842368983396.
IsChangePoint: Marked as a change because one of 10/31/2022 4:38:04 PM, 11/14/2022 1:59:26 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -64.2180625971499 (T) = (0 -666.0556611937677) / Math.Sqrt((92.8182726497298 / (28)) + (0.2808685969007546 / (27))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (27) - 2, .025) and -0.21332508980405204 = (548.9507031469476 - 666.0556611937677) / 548.9507031469476 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 666.1087253910298 > 583.492956795757.
IsChangePoint: Marked as a change because one of 10/31/2022 4:38:04 PM, 11/14/2022 1:59:26 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -65.4371743574446 (T) = (0 -666.1879416527432) / Math.Sqrt((91.97880423420412 / (27)) + (0.2674594162412254 / (27))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (27) - 2, .025) and -0.22183673129443549 = (545.234829326969 - 666.1879416527432) / 545.234829326969 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetSegmentHashCode - Duration of single invocation | 8.42 ns | 9.65 ns | 1.15 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'
Payloads
Histogram
Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetSegmentHashCode
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.645745549271561 > 8.843247772400257.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -71.17016400876543 (T) = (0 -9.714728087433029) / Math.Sqrt((0.0002696614693663075 / (27)) + (0.00862587104426406 / (27))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (27) - 2, .025) and -0.15336985510447312 = (8.422907920159801 - 9.714728087433029) / 8.422907920159801 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IterateTryGetTenSegments - Duration of single invocation | 34.61 ns | 38.80 ns | 1.12 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Byte>*'
Payloads
Histogram
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetTenSegments
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.80313055040986 > 36.41515371063089.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -21.071054527202303 (T) = (0 -38.640534493705175) / Math.Sqrt((0.36738886862822207 / (27)) + (0.3706076692052171 / (27))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (27) - 2, .025) and -0.0990879572400807 = (35.15690827032206 - 38.640534493705175) / 35.15690827032206 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Regressions in System.Collections.TryGetValueFalse<String, String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ConcurrentDictionary - Duration of single invocation | 11.44 μs | 13.06 μs | 1.14 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<String, String>*'
Payloads
Histogram
System.Collections.TryGetValueFalse<String, String>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.062030934553519 > 11.960065061368638.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -53.312222633056756 (T) = (0 -13146.895876043533) / Math.Sqrt((22317.200968174126 / (28)) + (5205.8544723019895 / (27))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (27) - 2, .025) and -0.1462392855878558 = (11469.59107172903 - 13146.895876043533) / 11469.59107172903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Regressions in System.Collections.ContainsKeyFalse<String, String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ConcurrentDictionary - Duration of single invocation | 11.68 μs | 13.31 μs | 1.14 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse<String, String>*'
Payloads
Histogram
System.Collections.ContainsKeyFalse<String, String>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.31282633298023 > 12.255327886504563.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -67.26612019697792 (T) = (0 -13316.463397055422) / Math.Sqrt((13296.717586853552 / (27)) + (2520.04918235662 / (27))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (27) - 2, .025) and -0.13928956137100498 = (11688.392353064815 - 13316.463397055422) / 11688.392353064815 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository