Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
Diff | Diff |
Regressions in Benchstone.BenchI.Fib
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 2.09 ms | 2.27 ms | 1.09 | 0.17 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Fib*'
Payloads
Histogram
Benchstone.BenchI.Fib.Test
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.266422619047619 > 2.1899359705528845.
IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -11.319119613617927 (T) = (0 -2263626.8315476193) / Math.Sqrt((1801730785.6133575 / (48)) + (2088520345.331355 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07844057075249286 = (2098981.522892959 - 2263626.8315476193) / 2098981.522892959 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
Diff | Diff |
Regressions in System.Memory.Span<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetPinnableReference - Duration of single invocation | 48.11 ns | 52.67 ns | 1.09 | 0.07 | False | |||||
GetPinnableReference - Duration of single invocation | 48.33 ns | 51.30 ns | 1.06 | 0.07 | False | |||||
GetPinnableReference - Duration of single invocation | 48.03 ns | 51.28 ns | 1.07 | 0.05 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Byte>*'
Payloads
Histogram
System.Memory.Span<Byte>.GetPinnableReference(Size: 4)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.667982595065624 > 50.50280039352934.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/18/2023 11:30:09 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -15.469259435235243 (T) = (0 -51.67797230036326) / Math.Sqrt((0.2850509205390661 / (48)) + (0.43533599711819626 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.0655349836577399 = (48.49955477103577 - 51.67797230036326) / 48.49955477103577 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Byte>.GetPinnableReference(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.29778339413507 > 50.80444936254753.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/18/2023 11:30:09 AM, 2/16/2023 3:28:45 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -14.421087047021448 (T) = (0 -51.784031725067656) / Math.Sqrt((0.37385255360164504 / (48)) + (0.5202599524821231 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06721226891908169 = (48.52271027348359 - 51.784031725067656) / 48.52271027348359 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Byte>.GetPinnableReference(Size: 33)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.277651934074235 > 50.93682447101759.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/18/2023 11:30:09 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -17.48319395514449 (T) = (0 -51.773782619936064) / Math.Sqrt((0.36335687710642867 / (48)) + (0.3101796556676777 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06579232794581882 = (48.57774001781711 - 51.773782619936064) / 48.57774001781711 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
Diff | Diff |
Regressions in System.Memory.Span<Char>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetPinnableReference - Duration of single invocation | 48.33 ns | 51.33 ns | 1.06 | 0.08 | False | |||||
GetPinnableReference - Duration of single invocation | 48.39 ns | 51.37 ns | 1.06 | 0.05 | False | |||||
GetPinnableReference - Duration of single invocation | 47.78 ns | 52.22 ns | 1.09 | 0.04 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Char>*'
Payloads
Histogram
System.Memory.Span<Char>.GetPinnableReference(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.327035258926706 > 50.469823882118156.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/3/2023 5:57:54 PM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -23.946223110244137 (T) = (0 -51.54681124233891) / Math.Sqrt((0.24531855542387265 / (48)) + (0.1430827279279217 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06454493669214545 = (48.421451707346456 - 51.54681124233891) / 48.421451707346456 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Char>.GetPinnableReference(Size: 4)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.366849349746786 > 50.438681861900456.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/18/2023 11:30:09 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -12.373137970557304 (T) = (0 -51.9742173180475) / Math.Sqrt((0.3315274809315786 / (48)) + (0.9203196484644269 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07392097562545819 = (48.39668699810747 - 51.9742173180475) / 48.39668699810747 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Char>.GetPinnableReference(Size: 33)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.21522142463035 > 50.465143874616594.
IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 1/18/2023 11:30:09 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -19.668367478669516 (T) = (0 -51.668983975887905) / Math.Sqrt((0.35983712494547004 / (48)) + (0.23894002964235447 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06725878031896139 = (48.41279821604849 - 51.668983975887905) / 48.41279821604849 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
Diff | Diff |
Regressions in System.Memory.Slice<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ReadOnlySpanStart - Duration of single invocation | 48.84 ns | 51.98 ns | 1.06 | 0.07 | False | |||||
ReadOnlySpanStartLength - Duration of single invocation | 46.96 ns | 54.29 ns | 1.16 | 0.14 | False | |||||
SpanStartLength - Duration of single invocation | 46.15 ns | 55.02 ns | 1.19 | 0.17 | False | |||||
SpanStart - Duration of single invocation | 48.73 ns | 52.85 ns | 1.08 | 0.09 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Slice<Byte>*'
Payloads
Histogram
System.Memory.Slice<Byte>.ReadOnlySpanStart
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.98042808521323 > 51.25801455652943.
IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 2/3/2023 8:13:11 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -15.050749833749904 (T) = (0 -52.36406818332557) / Math.Sqrt((0.2748014218059079 / (48)) + (0.5316887530776225 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06870838726943823 = (48.99752711505928 - 52.36406818332557) / 48.99752711505928 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<Byte>.ReadOnlySpanStartLength
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.294216092384666 > 49.15458624912584.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -9.805574649533408 (T) = (0 -54.289660924924014) / Math.Sqrt((1.571181016186533 / (48)) + (6.786331141339011 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.16238725361305847 = (46.70531335936022 - 54.289660924924014) / 46.70531335936022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<Byte>.SpanStartLength
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.020717755570246 > 48.37298017090156.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -10.76849416831492 (T) = (0 -53.931641041983085) / Math.Sqrt((1.545632083673282 / (48)) + (5.268983109563734 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.15884662945557304 = (46.539067095807326 - 53.931641041983085) / 46.539067095807326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<Byte>.SpanStart
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.851834228397635 > 51.199674661072336.
IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -5.757533354547897 (T) = (0 -53.080804443078954) / Math.Sqrt((0.2578186000808916 / (48)) + (6.3704257658731045 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.08628220796400243 = (48.8646541883138 - 53.080804443078954) / 48.8646541883138 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
Diff | Diff |
Regressions in PerfLabTests.LowLevelPerf
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
EmptyStaticFunction5Arg - Duration of single invocation | 32.89 ms | 37.49 ms | 1.14 | 0.06 | False | |||||
GenericClassGenericInstanceMethod - Duration of single invocation | 3.09 ms | 3.40 ms | 1.10 | 0.10 | False | |||||
GenericGenericMethod - Duration of single invocation | 2.84 ms | 3.03 ms | 1.07 | 0.09 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
Payloads
Histogram
PerfLabTests.LowLevelPerf.EmptyStaticFunction5Arg
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.48988778571428 > 34.50121421576923.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -19.400994182591752 (T) = (0 -37800968.98614164) / Math.Sqrt((365412563329.6146 / (47)) + (708020582596.5198 / (12))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (12) - 2, .025) and -0.15290684850543365 = (32787530.957201596 - 37800968.98614164) / 32787530.957201596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.402079466071428 > 3.2426906616927087.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -11.61781902155298 (T) = (0 -3349310.266535981) / Math.Sqrt((4505166019.332722 / (48)) + (3781405586.4303846 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07544037062553252 = (3114361.668037296 - 3349310.266535981) / 3114361.668037296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.LowLevelPerf.GenericGenericMethod
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0257708215277774 > 2.985019088854167.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/17/2023 6:13:48 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -16.62875330060637 (T) = (0 -3035684.726689242) / Math.Sqrt((3655384951.1262836 / (48)) + (316299130.86804336 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.058717951603747286 = (2867321.4826392457 - 3035684.726689242) / 2867321.4826392457 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
AbsBenchmark - Duration of single invocation | 13.32 ns | 14.45 ns | 1.09 | 0.32 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Byte>*'
Payloads
Histogram
System.Numerics.Tests.Perf_VectorOf<Byte>.AbsBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.452039906060195 > 14.215019766143751.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -8.474550521106528 (T) = (0 -14.575941273995385) / Math.Sqrt((0.2721421688418943 / (48)) + (0.1149210604076723 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07734232164285737 = (13.52953558138168 - 14.575941273995385) / 13.52953558138168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_Vector3
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ZeroBenchmark - Duration of single invocation | 11.47 ns | 13.09 ns | 1.14 | 0.50 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'
Payloads
Histogram
System.Numerics.Tests.Perf_Vector3.ZeroBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.089494764527748 > 11.772108646550445.
IsChangePoint: Marked as a change because one of 2/15/2023 8:48:29 AM, 2/21/2023 10:35:27 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -7.642733411448667 (T) = (0 -12.854107108745835) / Math.Sqrt((0.4714487008874269 / (48)) + (0.5791284154014491 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.16726231160592445 = (11.012183791885732 - 12.854107108745835) / 11.012183791885732 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
Diff | Diff |
Regressions in System.Memory.Span<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetPinnableReference - Duration of single invocation | 48.01 ns | 51.42 ns | 1.07 | 0.09 | False | |||||
GetPinnableReference - Duration of single invocation | 48.08 ns | 51.28 ns | 1.07 | 0.05 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*'
Payloads
Histogram
System.Memory.Span<Int32>.GetPinnableReference(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.4190578266497 > 50.40756758556486.
IsChangePoint: Marked as a change because one of 1/10/2023 11:45:30 PM, 1/11/2023 4:46:06 PM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -10.686263034368055 (T) = (0 -51.96178243286712) / Math.Sqrt((0.33164942634582284 / (48)) + (1.2632591639701611 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07397693802904472 = (48.3825868069635 - 51.96178243286712) / 48.3825868069635 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Int32>.GetPinnableReference(Size: 4)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.282881743900475 > 50.313979244103265.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/18/2023 11:30:09 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -11.433810740502958 (T) = (0 -52.025383735705255) / Math.Sqrt((0.24724885664087243 / (48)) + (1.1776162935537047 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07599896504352456 = (48.35077488536507 - 52.025383735705255) / 48.35077488536507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository