Open
Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | f70d8a9663d3d674c453e26d405dc13baa79fd16 |
Compare | 4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80 |
Diff | Diff |
Regressions in System.Collections.ContainsKeyTrue<Int32, Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SortedList - Duration of single invocation | 382.69 μs | 419.54 μs | 1.10 | 0.07 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue<Int32, Int32>*'
Payloads
Histogram
System.Collections.ContainsKeyTrue<Int32, Int32>.SortedList(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 419.5394734649123 > 401.911585527185.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -20.784450124744097 (T) = (0 -420305.82792123576) / Math.Sqrt((47191785.91851873 / (29)) + (48828580.69189607 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.09626396774899235 = (383398.3787538584 - 420305.82792123576) / 383398.3787538584 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | f70d8a9663d3d674c453e26d405dc13baa79fd16 |
Compare | 4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80 |
Diff | Diff |
Regressions in System.Collections.TryGetValueFalse<Int32, Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SortedList - Duration of single invocation | 420.67 μs | 467.08 μs | 1.11 | 0.07 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<Int32, Int32>*'
Payloads
Histogram
System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 467.0768386029412 > 438.34759868421054.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -24.948402219515575 (T) = (0 -461357.0615562561) / Math.Sqrt((23674853.905058924 / (29)) + (59688559.079077035 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.09715715355211081 = (420502.25900873495 - 461357.0615562561) / 420502.25900873495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | f70d8a9663d3d674c453e26d405dc13baa79fd16 |
Compare | 4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80 |
Diff | Diff |
Regressions in Microsoft.Extensions.DependencyInjection.TimeToFirstService
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Transient - Duration of single invocation | 31.07 μs | 33.03 μs | 1.06 | 0.09 | False | |||||
Transient - Duration of single invocation | 31.46 μs | 34.01 μs | 1.08 | 0.09 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'
Payloads
Histogram
Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.025174589545195 > 32.77686144684542.
IsChangePoint: Marked as a change because one of 11/22/2022 11:24:47 AM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -13.188129307379384 (T) = (0 -33389.7712626652) / Math.Sqrt((505263.59120631067 / (29)) + (201936.7952573102 / (31))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (31) - 2, .025) and -0.06508623282288062 = (31349.359548259017 - 33389.7712626652) / 31349.359548259017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.00875412661883 > 32.708026615930976.
IsChangePoint: Marked as a change because one of 11/22/2022 11:24:47 AM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -13.798450638946733 (T) = (0 -33354.07493353759) / Math.Sqrt((227161.60738955028 / (29)) + (516626.59633636766 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.06844432491263841 = (31217.41971558957 - 33354.07493353759) / 31217.41971558957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | f70d8a9663d3d674c453e26d405dc13baa79fd16 |
Compare | 4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80 |
Diff | Diff |
Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SortedList - Duration of single invocation | 425.92 μs | 460.68 μs | 1.08 | 0.09 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse<Int32, Int32>*'
Payloads
Histogram
System.Collections.ContainsKeyFalse<Int32, Int32>.SortedList(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 460.68048367346944 > 441.6207135089828.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -16.508334613944093 (T) = (0 -462851.33250016085) / Math.Sqrt((72344415.58201288 / (29)) + (107628844.58383995 / (31))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (31) - 2, .025) and -0.09543541206284899 = (422527.26852106315 - 462851.33250016085) / 422527.26852106315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | f70d8a9663d3d674c453e26d405dc13baa79fd16 |
Compare | 4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80 |
Diff | Diff |
Regressions in System.Collections.CreateAddAndClear<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SortedList - Duration of single invocation | 490.22 μs | 549.12 μs | 1.12 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'
Payloads
Histogram
System.Collections.CreateAddAndClear<Int32>.SortedList(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 549.1182464080458 > 517.1473475841346.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -15.967012940390267 (T) = (0 -537055.8063849128) / Math.Sqrt((51687651.82215776 / (29)) + (128556569.97395279 / (31))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (31) - 2, .025) and -0.07804470396039438 = (498175.821848519 - 537055.8063849128) / 498175.821848519 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | f70d8a9663d3d674c453e26d405dc13baa79fd16 |
Compare | 4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80 |
Diff | Diff |
Regressions in System.Collections.IndexerSet<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SortedList - Duration of single invocation | 366.08 μs | 408.34 μs | 1.12 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet<Int32>*'
Payloads
Histogram
System.Collections.IndexerSet<Int32>.SortedList(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 408.33596588827845 > 386.67392951699463.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -17.196476665148516 (T) = (0 -404765.7339687397) / Math.Sqrt((41497177.8183259 / (29)) + (73410751.39775394 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.08932156514910145 = (371575.9853825506 - 404765.7339687397) / 371575.9853825506 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | f70d8a9663d3d674c453e26d405dc13baa79fd16 |
Compare | 4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80 |
Diff | Diff |
Regressions in System.Collections.AddGivenSize<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SortedList - Duration of single invocation | 493.48 μs | 537.47 μs | 1.09 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize<Int32>*'
Payloads
Histogram
System.Collections.AddGivenSize<Int32>.SortedList(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 537.4745830555556 > 517.7054447753906.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -17.652056186774477 (T) = (0 -531116.001781808) / Math.Sqrt((22757925.704401605 / (29)) + (128967548.24963994 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.07866670713562972 = (492381.936207313 - 531116.001781808) / 492381.936207313 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository