[Perf] Linux/x64: 32 Improvements on 2/15/2023 2:23:18 PM #13307
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | c53d17cfd65507a037e62b163e685e9485825bba |
| Compare | 3b7dfff0e6e31262c43585c9eda8830c2d21675c |
| Diff | Diff |
Improvements in Microsoft.Extensions.DependencyInjection.GetService
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Singleton - Duration of single invocation | 153.13 ns | 125.98 ns | 0.82 | 0.39 | False | |||||
| ServiceScopeProvider - Duration of single invocation | 163.76 ns | 142.99 ns | 0.87 | 0.34 | False | |||||
| EmptyEnumerable - Duration of single invocation | 156.06 ns | 136.81 ns | 0.88 | 0.37 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'Details
Payloads
Histogram
Microsoft.Extensions.DependencyInjection.GetService.Singleton
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.97949097848921 < 145.219544778227.
IsChangePoint: Marked as a change because one of 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 22.98507657328535 (T) = (0 -129.96537821374147) / Math.Sqrt((50.850389268516714 / (299)) + (29.07777754953231 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.1518799488039057 = (153.2393651470128 - 129.96537821374147) / 153.2393651470128 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.GetService.ServiceScopeProvider
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 142.98753665499902 < 156.1096518002227.
IsChangePoint: Marked as a change because one of 1/11/2023 11:00:48 PM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 23.192725858667284 (T) = (0 -138.2874851559984) / Math.Sqrt((57.829604052194654 / (299)) + (20.9505477416115 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.13111887998465768 = (159.15581771825885 - 138.2874851559984) / 159.15581771825885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.GetService.EmptyEnumerable
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 136.81438616777157 < 148.04528485776302.
IsChangePoint: Marked as a change because one of 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 19.522542933894787 (T) = (0 -132.14772225375484) / Math.Sqrt((54.51723941951473 / (299)) + (37.80480261099215 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.14388509659990895 = (154.35746034664905 - 132.14772225375484) / 154.35746034664905 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | c53d17cfd65507a037e62b163e685e9485825bba |
| Compare | 3b7dfff0e6e31262c43585c9eda8830c2d21675c |
| Diff | Diff |
Improvements in System.Threading.Tests.Perf_Timer
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| LongScheduleAndDispose - Duration of single invocation | 532.06 ns | 489.17 ns | 0.92 | 0.05 | False | |||||
| ShortScheduleAndDispose - Duration of single invocation | 546.73 ns | 476.04 ns | 0.87 | 0.04 | False | |||||
| ShortScheduleAndDisposeWithFiringTimers - Duration of single invocation | 547.76 ns | 503.84 ns | 0.92 | 0.07 | False | |||||
| ScheduleManyThenDisposeMany - Duration of single invocation | 979.47 ms | 930.20 ms | 0.95 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Timer*'Details
Payloads
Histogram
System.Threading.Tests.Perf_Timer.LongScheduleAndDispose
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 489.1691913836177 < 505.2941655320029.
IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 30.571853319567357 (T) = (0 -486.36469738148685) / Math.Sqrt((202.5787504489693 / (299)) + (85.09139881398994 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.1007965203928182 = (540.8839138322239 - 486.36469738148685) / 540.8839138322239 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Threading.Tests.Perf_Timer.ShortScheduleAndDispose
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 476.04130729389095 < 505.94775839446237.
IsChangePoint: Marked as a change because one of 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 37.0218866356899 (T) = (0 -481.50302639492315) / Math.Sqrt((95.786737636838 / (299)) + (67.01593284275246 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.10425473992995844 = (537.5445987370033 - 481.50302639492315) / 537.5445987370033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Threading.Tests.Perf_Timer.ShortScheduleAndDisposeWithFiringTimers
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 503.8412173243148 < 524.4692855433735.
IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 47.3789101178497 (T) = (0 -501.29101757153524) / Math.Sqrt((149.63188033785255 / (299)) + (35.787347073267696 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.10537190174005642 = (560.3345329154639 - 501.29101757153524) / 560.3345329154639 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Threading.Tests.Perf_Timer.ScheduleManyThenDisposeMany
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 930.1983571428572 < 930.2731867616667.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 22.99144003867254 (T) = (0 -934658703.2110157) / Math.Sqrt((305695677460325.7 / (299)) + (124393835456709.94 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.05053177295788106 = (984402296.5600024 - 934658703.2110157) / 984402296.5600024 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | c53d17cfd65507a037e62b163e685e9485825bba |
| Compare | 3b7dfff0e6e31262c43585c9eda8830c2d21675c |
| Diff | Diff |
Improvements in System.ComponentModel.Tests.Perf_TypeDescriptorTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| GetConverter - Duration of single invocation | 1.23 μs | 954.89 ns | 0.78 | 0.26 | False | |||||
| GetConverter - Duration of single invocation | 876.03 ns | 719.72 ns | 0.82 | 0.24 | False | |||||
| GetConverter - Duration of single invocation | 1.45 μs | 1.26 μs | 0.87 | 0.29 | False | |||||
| GetConverter - Duration of single invocation | 1.20 μs | 990.18 ns | 0.82 | 0.25 | False | |||||
| GetConverter - Duration of single invocation | 761.32 ns | 586.03 ns | 0.77 | 0.24 | False | |||||
| GetConverter - Duration of single invocation | 888.66 ns | 728.37 ns | 0.82 | 0.23 | False | |||||
| GetConverter - Duration of single invocation | 1.24 μs | 1.02 μs | 0.83 | 0.25 | False | |||||
| GetConverter - Duration of single invocation | 1.23 μs | 970.53 ns | 0.79 | 0.29 | False | |||||
| GetConverter - Duration of single invocation | 1.44 μs | 1.13 μs | 0.79 | 0.27 | False | |||||
| GetConverter - Duration of single invocation | 899.21 ns | 713.68 ns | 0.79 | 0.28 | False | |||||
| GetConverter - Duration of single invocation | 1.32 μs | 1.06 μs | 0.80 | 0.26 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'Details
Payloads
Histogram
System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(int?))
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 954.8868210991494 < 1.179262874907403.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 35.52241243995188 (T) = (0 -1015.4669966504922) / Math.Sqrt((8358.170136497321 / (299)) + (1439.2769421965877 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22835650323186785 = (1315.979465781237 - 1015.4669966504922) / 1315.979465781237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassIDerived))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 719.7158207902163 < 832.812588357352.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 51.047131556146766 (T) = (0 -706.3673286837819) / Math.Sqrt((2234.2526072997093 / (299)) + (379.0970773723956 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.23772273778996897 = (926.6540715590121 - 706.3673286837819) / 926.6540715590121 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeEnum))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2562928480211197 < 1.418580237461358.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 47.943745081748226 (T) = (0 -1231.7762979202414) / Math.Sqrt((8347.791688813142 / (299)) + (1158.5238157429233 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.2345722248796478 = (1609.2652213026406 - 1231.7762979202414) / 1609.2652213026406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Enum))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 990.1832806004194 < 1.17178890462503.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 40.88599219213449 (T) = (0 -1005.8148173500676) / Math.Sqrt((4341.982989921652 / (299)) + (1001.133455701898 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.2123114413703461 = (1276.9194199035837 - 1005.8148173500676) / 1276.9194199035837 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.IDerived))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 586.0274988611603 < 719.2997670890054.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 55.01277671488137 (T) = (0 -589.0128988675995) / Math.Sqrt((2835.5047136275616 / (299)) + (241.2134376999199 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.27550876764025556 = (813.0021076295461 - 589.0128988675995) / 813.0021076295461 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassWithNoConverter))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 728.3664879979955 < 848.7599552422546.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 50.050085969613555 (T) = (0 -713.4851621300629) / Math.Sqrt((3409.694431708473 / (299)) + (286.8950196467499 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.23807846245797198 = (936.4286569871464 - 713.4851621300629) / 936.4286569871464 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeValueType?))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.024830865980533 < 1.1812005344818952.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 52.24724093273906 (T) = (0 -1000.0987597469357) / Math.Sqrt((7362.572040293224 / (299)) + (435.9318248341802 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.24224260865415553 = (1319.8139287967504 - 1000.0987597469357) / 1319.8139287967504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Guid))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 970.5255525396428 < 1.190158838270905.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 38.65778057040936 (T) = (0 -1006.4293557721685) / Math.Sqrt((4812.512818837336 / (299)) + (1378.5782919558487 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.22425456776530614 = (1297.370650153804 - 1006.4293557721685) / 1297.370650153804 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.DerivedClass))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1316285293381567 < 1.336150544253458.
IsChangePoint: Marked as a change because one of 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 52.808137640532706 (T) = (0 -1121.0404769846298) / Math.Sqrt((3245.2721068598817 / (299)) + (793.9248449451649 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.2159935227595605 = (1429.8867541636757 - 1121.0404769846298) / 1429.8867541636757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(string))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 713.6771407241075 < 853.2999635986953.
IsChangePoint: Marked as a change because one of 1/15/2023 10:25:39 PM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 45.24496395385011 (T) = (0 -710.8028981109862) / Math.Sqrt((2809.6998286028297 / (299)) + (591.6957419564235 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.24785103578469664 = (945.029418278263 - 710.8028981109862) / 945.029418278263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(int))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.057068413729128 < 1.1940301094256072.
IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 40.71888307827545 (T) = (0 -1020.7427950619572) / Math.Sqrt((4865.831446215083 / (299)) + (1162.8956727559532 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.22082932281210477 = (1310.0374859407182 - 1020.7427950619572) / 1310.0374859407182 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | c53d17cfd65507a037e62b163e685e9485825bba |
| Compare | 3b7dfff0e6e31262c43585c9eda8830c2d21675c |
| Diff | Diff |
Improvements in System.Collections.Concurrent.IsEmpty<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Dictionary - Duration of single invocation | 287.06 ns | 257.22 ns | 0.90 | 0.26 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty<Int32>*'Details
Payloads
Histogram
System.Collections.Concurrent.IsEmpty<Int32>.Dictionary(Size: 0)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 257.21855025747374 < 267.5820506755632.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 23.28228331619579 (T) = (0 -258.37551087551697) / Math.Sqrt((756.5834243432002 / (299)) + (39.43217389615091 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.14755749648909547 = (303.10022061471716 - 258.37551087551697) / 303.10022061471716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | c53d17cfd65507a037e62b163e685e9485825bba |
| Compare | 3b7dfff0e6e31262c43585c9eda8830c2d21675c |
| Diff | Diff |
Improvements in System.Threading.Tests.Perf_Monitor
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| EnterExit - Duration of single invocation | 58.43 ns | 33.67 ns | 0.58 | 0.30 | False | |||||
| TryEnterExit - Duration of single invocation | 96.27 ns | 38.48 ns | 0.40 | 0.32 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Monitor*'Details
Payloads
Histogram
System.Threading.Tests.Perf_Monitor.EnterExit
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.67290064574656 < 49.88320668132738.
IsChangePoint: Marked as a change because one of 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 51.83545384548628 (T) = (0 -31.94367247758765) / Math.Sqrt((4.405255241737448 / (299)) + (5.535694533825773 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.40608894519125444 = (53.785280167708486 - 31.94367247758765) / 53.785280167708486 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Threading.Tests.Perf_Monitor.TryEnterExit
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.47891169731602 < 91.49356061902296.
IsChangePoint: Marked as a change because one of 2/15/2023 12:21:17 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 115.04688276386418 (T) = (0 -38.74592544769721) / Math.Sqrt((12.779424068791789 / (299)) + (7.990296427813941 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.6101150562320407 = (99.37784484121272 - 38.74592544769721) / 99.37784484121272 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository