[Perf] Linux/x64: 1 Regression on 8/20/2023 5:37:20 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	a429b0ca4fbb2ed7014815792f1cf1b5939eb567
Compare	d74b52492d85a3a250654d670942e35f9f0be2e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
IndexOfAnyTwoValues - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	1.56 ns	2.65 ns	1.70	0.33	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*'

Payloads

Baseline
Compare

System.Memory.Span<Int32>.IndexOfAnyTwoValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.654301425878747 > 1.8619193806192755.
IsChangePoint: Marked as a change because one of 8/19/2023 11:06:43 PM, 8/28/2023 11:06:53 PM falls between 8/19/2023 11:06:43 PM and 8/28/2023 11:06:53 PM.
IsRegressionStdDev: Marked as regression because -5.451648309262017 (T) = (0 -2.1929101539938283) / Math.Sqrt((0.05550951201037635 / (8)) + (0.025818483466123296 / (27))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (27) - 2, .025) and -0.28352004260322977 = (1.7085125913158141 - 2.1929101539938283) / 1.7085125913158141 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[cincuranet]

The diff points to #88394 cc @MihaZupan. But I'm not 100% convinced.

[MihaZupan]

That PR is adding new APIs, but they're not consumed anywhere yet, so I doubt it's the cause.

[cincuranet]

The first spike points to d267844...374b111. That seems to be hardly related as well.

[cincuranet]

Also seems to be only Linux/Ubuntu 22.04 x64. Others are unaffected.

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	a429b0ca4fbb2ed7014815792f1cf1b5939eb567
Compare	d74b52492d85a3a250654d670942e35f9f0be2e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
IndexOfAnyTwoValues - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	1.56 ns	2.65 ns	1.70	0.33	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*'

Payloads

Baseline
Compare

System.Memory.Span<Int32>.IndexOfAnyTwoValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.654301425878747 > 1.8619193806192755.
IsChangePoint: Marked as a change because one of 8/19/2023 11:06:43 PM, 8/28/2023 11:06:53 PM falls between 8/19/2023 11:06:43 PM and 8/28/2023 11:06:53 PM.
IsRegressionStdDev: Marked as regression because -5.451648309262017 (T) = (0 -2.1929101539938283) / Math.Sqrt((0.05550951201037635 / (8)) + (0.025818483466123296 / (27))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (27) - 2, .025) and -0.28352004260322977 = (1.7085125913158141 - 2.1929101539938283) / 1.7085125913158141 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	area-System.Memory, os-linux, arch-x64, untriaged, runtime-coreclr, needs-area-label
Milestone:	-