Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
DivideBenchmark - Duration of single invocation | 26.28 ns | 14.08 ns | 0.54 | 0.07 | False | |||||
DivisionOperatorBenchmark - Duration of single invocation | 16.22 ns | 14.04 ns | 0.87 | 0.10 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int32>*'
Payloads
Histogram
System.Numerics.Tests.Perf_VectorOf<Int32>.DivideBenchmark
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.079247741581415 < 15.690872131483493.
IsChangePoint: Marked as a change because one of 12/1/2022 8:17:56 AM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 7.268165698674812 (T) = (0 -14.57252238758615) / Math.Sqrt((3.2134236963539435 / (299)) + (5.095470706309408 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.18217455954592526 = (17.818622002630736 - 14.57252238758615) / 17.818622002630736 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Int32>.DivisionOperatorBenchmark
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.04375342940623 < 15.692813143439926.
IsChangePoint: Marked as a change because one of 12/1/2022 1:10:54 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 8.247946057151967 (T) = (0 -14.41700491876742) / Math.Sqrt((4.394261591948251 / (299)) + (4.610011788176779 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.19766344706313033 = (17.96877490623538 - 14.41700491876742) / 17.96877490623538 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in System.Globalization.Tests.StringSearch
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IsPrefix_FirstHalf - Duration of single invocation | 182.58 ns | 156.66 ns | 0.86 | 0.00 | False | |||||
IsPrefix_FirstHalf - Duration of single invocation | 181.55 ns | 156.57 ns | 0.86 | 0.00 | False | |||||
IsPrefix_FirstHalf - Duration of single invocation | 182.14 ns | 156.62 ns | 0.86 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
Payloads
Histogram
System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False))
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 156.65615165828865 < 172.68216279545794.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 523.296405161365 (T) = (0 -156.6447204502548) / Math.Sqrt((0.0568845061788563 / (299)) + (0.05532558402024849 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13650601781309987 = (181.40800478253897 - 156.6447204502548) / 181.40800478253897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, None, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 156.56564393247865 < 172.53243101694258.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 701.8023954882603 (T) = (0 -156.608017356354) / Math.Sqrt((0.03648856544870295 / (299)) + (0.03035841651326636 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13659853398855412 = (181.38493333792636 - 156.608017356354) / 181.38493333792636 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 156.62026422310404 < 172.4766087498163.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 585.9224882561207 (T) = (0 -156.67838114423287) / Math.Sqrt((0.05501791174174816 / (299)) + (0.043097442761697705 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13628198233429314 = (181.39992212698476 - 156.67838114423287) / 181.39992212698476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in PerfLabTests.GetMember
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetMethod3 - Duration of single invocation | 196.10 μs | 168.29 μs | 0.86 | 0.11 | False | |||||
GetMethod5 - Duration of single invocation | 327.91 μs | 283.90 μs | 0.87 | 0.06 | False | |||||
GetMethod15 - Duration of single invocation | 1.01 ms | 852.23 μs | 0.84 | 0.08 | False | |||||
GetMethod2 - Duration of single invocation | 131.72 μs | 112.06 μs | 0.85 | 0.12 | False | |||||
GetMethod12 - Duration of single invocation | 792.58 μs | 714.22 μs | 0.90 | 0.09 | False | |||||
GetMethod10 - Duration of single invocation | 670.09 μs | 564.52 μs | 0.84 | 0.12 | False | |||||
GetField - Duration of single invocation | 36.62 μs | 32.59 μs | 0.89 | 0.21 | False | |||||
GetMethod20 - Duration of single invocation | 1.34 ms | 1.14 ms | 0.85 | 0.10 | False | |||||
GetMethod4 - Duration of single invocation | 269.20 μs | 226.05 μs | 0.84 | 0.09 | False | |||||
GetMethod1 - Duration of single invocation | 65.48 μs | 55.75 μs | 0.85 | 0.24 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.GetMember*'
Payloads
Histogram
PerfLabTests.GetMember.GetMethod3
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 168.293898754529 < 186.2980804716991.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 26.085658657460026 (T) = (0 -173633.80250955911) / Math.Sqrt((17620418.200615227 / (299)) + (26287988.11505673 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13244148205179818 = (200140.73854085084 - 173633.80250955911) / 200140.73854085084 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod5
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 283.9046101934524 < 316.1027222098215.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 39.330706585988445 (T) = (0 -287446.24627466634) / Math.Sqrt((19834551.0170928 / (299)) + (35411302.896444544 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.138385778700344 = (333613.6279657541 - 287446.24627466634) / 333613.6279657541 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod15
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 852.2301832264959 < 949.061611390904.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 34.735962786631895 (T) = (0 -863461.9829353065) / Math.Sqrt((456581601.9849591 / (299)) + (394503291.2020023 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1391248390772022 = (1003004.874725083 - 863461.9829353065) / 1003004.874725083 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod2
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 112.0593858877184 < 125.03906003432623.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 25.587591418754737 (T) = (0 -115139.86744704431) / Math.Sqrt((8653241.678607965 / (299)) + (12739136.433926718 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1358900638628669 = (133246.78103084763 - 115139.86744704431) / 133246.78103084763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod12
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 714.2214692725752 < 760.4039378533654.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 31.339120869030822 (T) = (0 -693165.7028529083) / Math.Sqrt((199821450.23200053 / (299)) + (311713072.5317457 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1364442807881573 = (802687.8722840868 - 693165.7028529083) / 802687.8722840868 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod10
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 564.5222294642858 < 627.1083685028187.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 34.57278362812976 (T) = (0 -574188.9340147264) / Math.Sqrt((143384770.184701 / (299)) + (185053432.59993914 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1401947289797769 = (667812.7633870032 - 574188.9340147264) / 667812.7633870032 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetField
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.58684607337914 < 33.641075322122546.
IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 47.84761772603205 (T) = (0 -32658.16993103324) / Math.Sqrt((1347577.570335686 / (299)) + (20096.964760417995 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09736405281401252 = (36180.887801828314 - 32658.16993103324) / 36180.887801828314 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod20
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.135422182440476 < 1.2709009588014637.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 34.08205670004014 (T) = (0 -1154824.4229595356) / Math.Sqrt((579387731.9970685 / (299)) + (784923742.2215774 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1411472538585562 = (1344612.8316498955 - 1154824.4229595356) / 1344612.8316498955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod4
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 226.0455370710784 < 254.97632559746168.
IsChangePoint: Marked as a change because one of 1/11/2023 8:19:46 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 34.96805264127789 (T) = (0 -229422.8132181878) / Math.Sqrt((29068588.188159544 / (299)) + (27499893.88990161 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14074431938933055 = (267001.7998078732 - 229422.8132181878) / 267001.7998078732 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod1
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 55.74837267314536 < 62.23383211718144.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 18.972122603022623 (T) = (0 -56808.50350368158) / Math.Sqrt((3678499.9754763856 / (299)) + (6885013.154299531 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14724162899746263 = (66617.35074719372 - 56808.50350368158) / 66617.35074719372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
DivideBenchmark - Duration of single invocation | 38.44 ns | 27.66 ns | 0.72 | 0.05 | False | |||||
DivisionOperatorBenchmark - Duration of single invocation | 38.31 ns | 27.99 ns | 0.73 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int16>*'
Payloads
Histogram
System.Numerics.Tests.Perf_VectorOf<Int16>.DivideBenchmark
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.657101352321924 < 36.52354260156496.
IsChangePoint: Marked as a change because one of 11/2/2022 4:25:40 PM, 12/14/2022 5:20:21 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 40.711202788627936 (T) = (0 -27.715245604505387) / Math.Sqrt((10.526782676072207 / (215)) + (0.005099819870262427 / (27))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (215) + (27) - 2, .975) and 0.24565683317759013 = (36.74089833842189 - 27.715245604505387) / 36.74089833842189 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Int16>.DivisionOperatorBenchmark
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.9864767971997 < 36.36905982762844.
IsChangePoint: Marked as a change because one of 12/1/2022 8:17:56 AM, 12/14/2022 5:20:21 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 45.273013241325664 (T) = (0 -27.85675200110198) / Math.Sqrt((12.261213250013755 / (299)) + (0.025790217657861193 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.2497675176295404 = (37.1308263181099 - 27.85675200110198) / 37.1308263181099 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in LinqBenchmarks
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Count00ForX - Duration of single invocation | 101.95 ms | 90.17 ms | 0.88 | 0.46 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'
Payloads
Histogram
LinqBenchmarks.Count00ForX
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 90.17260977500001 < 97.84106525.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 54.474374341288474 (T) = (0 -90417877.39351852) / Math.Sqrt((20749970817787.105 / (299)) + (334032201541.02997 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1469610847150882 = (105995020.59448166 - 90417877.39351852) / 105995020.59448166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
DivideBenchmark - Duration of single invocation | 69.61 ns | 55.27 ns | 0.79 | 0.05 | False | |||||
DivisionOperatorBenchmark - Duration of single invocation | 69.94 ns | 55.24 ns | 0.79 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<SByte>*'
Payloads
Histogram
System.Numerics.Tests.Perf_VectorOf<SByte>.DivideBenchmark
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 55.273204011181186 < 66.08339175431158.
IsChangePoint: Marked as a change because one of 12/1/2022 8:17:56 AM, 12/14/2022 5:20:21 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 36.239082024787 (T) = (0 -55.75544112213575) / Math.Sqrt((30.259522194546186 / (299)) + (1.9420786209084173 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.21287373198650075 = (70.83417665992509 - 55.75544112213575) / 70.83417665992509 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<SByte>.DivisionOperatorBenchmark
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 55.244898701987985 < 66.30199093389471.
IsChangePoint: Marked as a change because one of 12/1/2022 8:17:56 AM, 12/14/2022 5:20:21 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 31.791015498379597 (T) = (0 -56.03683892958797) / Math.Sqrt((28.01456309471338 / (299)) + (5.333319767957 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.23439548903917673 = (73.1929320260437 - 56.03683892958797) / 73.1929320260437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Base64DecodeInPlace - Duration of single invocation | 147.78 ms | 128.19 ms | 0.87 | 0.37 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests*'
Payloads
Histogram
System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests.Base64DecodeInPlace(NumberOfBytes: 200000000)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 128.19097976923078 < 140.45979924836598.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 35.2963159078725 (T) = (0 -128134705.84564508) / Math.Sqrt((42740187188323.59 / (299)) + (11250090984.810453 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09444758490838331 = (141498938.89099884 - 128134705.84564508) / 141498938.89099884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ForEach_Array - Duration of single invocation | 6.04 ns | 4.76 ns | 0.79 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark*'
Payloads
Histogram
Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.76103949948052 < 5.7393568109296975.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 103.061222218321 (T) = (0 -4.758857156379724) / Math.Sqrt((0.06363210747877772 / (299)) + (7.6478039572481E-05 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.24129116387407973 = (6.272310179856549 - 4.758857156379724) / 6.272310179856549 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository