Skip to content

[Perf] Linux/x64: 18 Regressions on 3/11/2023 1:35:17 AM #14045

Open
@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff Diff

Regressions in Benchstone.BenchI.Fib

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 2.09 ms 2.27 ms 1.09 0.17 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Fib*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Fib.Test


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.266422619047619 > 2.1899359705528845.
IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -11.319119613617927 (T) = (0 -2263626.8315476193) / Math.Sqrt((1801730785.6133575 / (48)) + (2088520345.331355 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07844057075249286 = (2098981.522892959 - 2263626.8315476193) / 2098981.522892959 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetPinnableReference - Duration of single invocation 48.11 ns 52.67 ns 1.09 0.07 False
GetPinnableReference - Duration of single invocation 48.33 ns 51.30 ns 1.06 0.07 False
GetPinnableReference - Duration of single invocation 48.03 ns 51.28 ns 1.07 0.05 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.GetPinnableReference(Size: 4)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.667982595065624 > 50.50280039352934.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/18/2023 11:30:09 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -15.469259435235243 (T) = (0 -51.67797230036326) / Math.Sqrt((0.2850509205390661 / (48)) + (0.43533599711819626 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.0655349836577399 = (48.49955477103577 - 51.67797230036326) / 48.49955477103577 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.GetPinnableReference(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.29778339413507 > 50.80444936254753.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/18/2023 11:30:09 AM, 2/16/2023 3:28:45 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -14.421087047021448 (T) = (0 -51.784031725067656) / Math.Sqrt((0.37385255360164504 / (48)) + (0.5202599524821231 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06721226891908169 = (48.52271027348359 - 51.784031725067656) / 48.52271027348359 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.GetPinnableReference(Size: 33)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.277651934074235 > 50.93682447101759.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/18/2023 11:30:09 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -17.48319395514449 (T) = (0 -51.773782619936064) / Math.Sqrt((0.36335687710642867 / (48)) + (0.3101796556676777 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06579232794581882 = (48.57774001781711 - 51.773782619936064) / 48.57774001781711 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetPinnableReference - Duration of single invocation 48.33 ns 51.33 ns 1.06 0.08 False
GetPinnableReference - Duration of single invocation 48.39 ns 51.37 ns 1.06 0.05 False
GetPinnableReference - Duration of single invocation 47.78 ns 52.22 ns 1.09 0.04 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.GetPinnableReference(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.327035258926706 > 50.469823882118156.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/3/2023 5:57:54 PM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -23.946223110244137 (T) = (0 -51.54681124233891) / Math.Sqrt((0.24531855542387265 / (48)) + (0.1430827279279217 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06454493669214545 = (48.421451707346456 - 51.54681124233891) / 48.421451707346456 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.GetPinnableReference(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.366849349746786 > 50.438681861900456.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/18/2023 11:30:09 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -12.373137970557304 (T) = (0 -51.9742173180475) / Math.Sqrt((0.3315274809315786 / (48)) + (0.9203196484644269 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07392097562545819 = (48.39668699810747 - 51.9742173180475) / 48.39668699810747 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.GetPinnableReference(Size: 33)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.21522142463035 > 50.465143874616594.
IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 1/18/2023 11:30:09 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -19.668367478669516 (T) = (0 -51.668983975887905) / Math.Sqrt((0.35983712494547004 / (48)) + (0.23894002964235447 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06725878031896139 = (48.41279821604849 - 51.668983975887905) / 48.41279821604849 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff Diff

Regressions in System.Memory.Slice<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadOnlySpanStart - Duration of single invocation 48.84 ns 51.98 ns 1.06 0.07 False
ReadOnlySpanStartLength - Duration of single invocation 46.96 ns 54.29 ns 1.16 0.14 False
SpanStartLength - Duration of single invocation 46.15 ns 55.02 ns 1.19 0.17 False
SpanStart - Duration of single invocation 48.73 ns 52.85 ns 1.08 0.09 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Slice&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Slice<Byte>.ReadOnlySpanStart


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.98042808521323 > 51.25801455652943.
IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 2/3/2023 8:13:11 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -15.050749833749904 (T) = (0 -52.36406818332557) / Math.Sqrt((0.2748014218059079 / (48)) + (0.5316887530776225 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06870838726943823 = (48.99752711505928 - 52.36406818332557) / 48.99752711505928 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;Byte&gt;.ReadOnlySpanStartLength

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.294216092384666 > 49.15458624912584.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -9.805574649533408 (T) = (0 -54.289660924924014) / Math.Sqrt((1.571181016186533 / (48)) + (6.786331141339011 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.16238725361305847 = (46.70531335936022 - 54.289660924924014) / 46.70531335936022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;Byte&gt;.SpanStartLength

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.020717755570246 > 48.37298017090156.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -10.76849416831492 (T) = (0 -53.931641041983085) / Math.Sqrt((1.545632083673282 / (48)) + (5.268983109563734 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.15884662945557304 = (46.539067095807326 - 53.931641041983085) / 46.539067095807326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;Byte&gt;.SpanStart

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.851834228397635 > 51.199674661072336.
IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -5.757533354547897 (T) = (0 -53.080804443078954) / Math.Sqrt((0.2578186000808916 / (48)) + (6.3704257658731045 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.08628220796400243 = (48.8646541883138 - 53.080804443078954) / 48.8646541883138 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EmptyStaticFunction5Arg - Duration of single invocation 32.89 ms 37.49 ms 1.14 0.06 False
GenericClassGenericInstanceMethod - Duration of single invocation 3.09 ms 3.40 ms 1.10 0.10 False
GenericGenericMethod - Duration of single invocation 2.84 ms 3.03 ms 1.07 0.09 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.EmptyStaticFunction5Arg


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.48988778571428 > 34.50121421576923.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -19.400994182591752 (T) = (0 -37800968.98614164) / Math.Sqrt((365412563329.6146 / (47)) + (708020582596.5198 / (12))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (12) - 2, .025) and -0.15290684850543365 = (32787530.957201596 - 37800968.98614164) / 32787530.957201596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.402079466071428 > 3.2426906616927087.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -11.61781902155298 (T) = (0 -3349310.266535981) / Math.Sqrt((4505166019.332722 / (48)) + (3781405586.4303846 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07544037062553252 = (3114361.668037296 - 3349310.266535981) / 3114361.668037296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.GenericGenericMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0257708215277774 > 2.985019088854167.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/17/2023 6:13:48 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -16.62875330060637 (T) = (0 -3035684.726689242) / Math.Sqrt((3655384951.1262836 / (48)) + (316299130.86804336 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.058717951603747286 = (2867321.4826392457 - 3035684.726689242) / 2867321.4826392457 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AbsBenchmark - Duration of single invocation 13.32 ns 14.45 ns 1.09 0.32 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.AbsBenchmark


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.452039906060195 > 14.215019766143751.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -8.474550521106528 (T) = (0 -14.575941273995385) / Math.Sqrt((0.2721421688418943 / (48)) + (0.1149210604076723 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07734232164285737 = (13.52953558138168 - 14.575941273995385) / 13.52953558138168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ZeroBenchmark - Duration of single invocation 11.47 ns 13.09 ns 1.14 0.50 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector3.ZeroBenchmark


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.089494764527748 > 11.772108646550445.
IsChangePoint: Marked as a change because one of 2/15/2023 8:48:29 AM, 2/21/2023 10:35:27 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -7.642733411448667 (T) = (0 -12.854107108745835) / Math.Sqrt((0.4714487008874269 / (48)) + (0.5791284154014491 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.16726231160592445 = (11.012183791885732 - 12.854107108745835) / 11.012183791885732 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetPinnableReference - Duration of single invocation 48.01 ns 51.42 ns 1.07 0.09 False
GetPinnableReference - Duration of single invocation 48.08 ns 51.28 ns 1.07 0.05 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.GetPinnableReference(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.4190578266497 > 50.40756758556486.
IsChangePoint: Marked as a change because one of 1/10/2023 11:45:30 PM, 1/11/2023 4:46:06 PM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -10.686263034368055 (T) = (0 -51.96178243286712) / Math.Sqrt((0.33164942634582284 / (48)) + (1.2632591639701611 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07397693802904472 = (48.3825868069635 - 51.96178243286712) / 48.3825868069635 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.GetPinnableReference(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.282881743900475 > 50.313979244103265.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/18/2023 11:30:09 AM, 2/17/2023 12:34:41 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -11.433810740502958 (T) = (0 -52.025383735705255) / Math.Sqrt((0.24724885664087243 / (48)) + (1.1776162935537047 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07599896504352456 = (48.35077488536507 - 52.025383735705255) / 48.35077488536507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions