[Perf] Linux/x64: 1 Regression on 10/15/2023 11:56:34 AM

### Run Information

Name | Value
-- | --
Architecture | x64
OS | ubuntu 22.04
Queue | TigerUbuntu
Baseline | [765798c49d5d908e512af0931b18fed782b4cf00](https://github.com/dotnet/runtime/commit/765798c49d5d908e512af0931b18fed782b4cf00)
Compare | [b9159ab91a5dbd124dd15d78ed073c52eba8c31b](https://github.com/dotnet/runtime/commit/b9159ab91a5dbd124dd15d78ed073c52eba8c31b)
Diff | [Diff](https://github.com/dotnet/runtime/compare/765798c49d5d908e512af0931b18fed782b4cf00...b9159ab91a5dbd124dd15d78ed073c52eba8c31b)
Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono


### Regressions in System.Globalization.Tests.StringSearch

Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio
-- | -- | -- | -- | -- | -- | -- | -- | --
|<ul><li>[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 22.04_LLVM=false_MonoAOT=false_MonoInterpreter=true_RunKind=micro_mono/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>)</li><li>📝 - [Benchmark Source](<https://github.com/dotnet/performance/blob/main/src/benchmarks/micro/libraries/System.Globalization/StringSearch.cs#L90-#L91>)</li><li>[📈 - ADX Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpVUXW/aMBR9R9p/8FuSLaQJIMqYMolSda1WRtfQ7aGqKpNcwJ0/ItsZUE377bsJtCV07TRFiuLrc47viY/NwZJkobT9QgUYEpOMWnymHNzLQn4toIC+sZrJuU+qUYnbVjxy3XC+M5mppQmiMBiIrNsJWuFpKxgveXABeub4xFky2Vx1O00qMsf/CyF6fxoFEzYH/YzCpAVekq6mhbQFaoedYKAFsh6gHKG0rKB+Dhp20a0SXa1Rl+cvyEe9lwlRr07ZtfHYUJAUTwY2XRms0BR2KQnonyhe97Hc99G4+dAYAUU+CJDWNH6R5QInyFHBeHYJuSJxTJxMWQn2QKMBJpBVQx1pKtNFhdMwMwcLoJk5EJTJPeBApwtmIbW4WgVHrw5R+oXZqtEniQmYKj/VXLI2FkTwiasp5eyeWqZkUCLQeBWbBCjqBWfmAntiq9tjNpuhirQnTBs7XFDtjvOSZfrEBdm8Snwy1hmTlJ/NpdIwpAZ8ckK5Ac976gLzOlRyxuaFrhY1185QiZzxajRSGTg3JP5NHMsQnjlvXmOen38bbeGzcqHX0SMl1WA8+T/CGUZJ5xrw/dCYLv7Bw9JnTNEWL1iq1a1ArfpeDBWmAfRE5duv+mZPMCiJxZiRj+VxhzI4bitstZvhYTM6nIS9frvbb7WDTqcbRb3uuzDsh6GHIqYQgmp2D4Tq+a2gK7cSLPfeJ289Ml3vreA/ZsMv7VQ3COrkWt1hnJ6BL8EU3MYau87cHSeb+gBPDZ2jUturqd0pJskP/C0xkxL07n2m5C4QQ5bhfOk3xZhZsmR2QVwDmoGJH683n6xSxQsh4/3+1pu6iTcNIdAyyyF2jvE34ileb4clhbjSeGXN5JzZmK7AeH8Ao8bYZW4FAAA=>)</li></ul> | 151.96 ns | 259.77 ns | 1.71 | 0.13 | True | | |

![graph](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/autofilereports/10_17_2023/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dfalse_MonoAOT%3Dfalse_MonoInterpreter%3Dtrue_RunKind%3Dmicro_mono_Regression/System.Globalization.Tests.StringSearch_1.png>)
[Test Report](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/autofilereports/10_17_2023/refs/heads/main_x64_ubuntu%2022.04_LLVM%3Dfalse_MonoAOT%3Dfalse_MonoInterpreter%3Dtrue_RunKind%3Dmicro_mono_Regression/System.Globalization.Tests.StringSearch.html>)

### Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

<details>
   <summary>Repro Steps</summary>


#### Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0`
- CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0`
- Mono Runtime build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release`
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder

Linux
```cmd
# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Globalization.Tests.StringSearch*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Globalization.Tests.StringSearch*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages
```

Windows
```cmd
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Globalization.Tests.StringSearch*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Globalization.Tests.StringSearch*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages
```

</details>

<details>


### Payloads

[Baseline](<https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-0f286e95-69d9-445c-ae2d-4186bf609d2c38ad26fffee425c99/dd053d61-2694-4f6b-8609-7685af6c98ff.zip?sv=2021-08-06&se=2023-11-13T07%3A18%3A36Z&sr=c&sp=rl&sig=MXKCtsevxzQD41Zg77oyu2BNG8avOscqUG6c5DR7ewM%3D>)
[Compare](<https://helixdi107v0xdeko0k025g8.blob.core.windows.net/helix-job-66ae9fc7-c4f8-4e3b-a41b-071e31ece504c728f7d18e2406a97/408c79c8-e182-4519-9c6a-08f0d06675a1.zip?sv=2021-08-06&se=2023-11-15T04%3A07%3A52Z&sr=c&sp=rl&sig=Hsfqx%2FuqPQgzYVmHZmAJaJ263yGP6TK14viyjqDgPVs%3D>)
### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False))

#### ETL Files

#### Histogram

#### Description of detection logic
```
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 259.77145843179125 > 159.59479019439289.
IsChangePoint: Marked as a change because one of 10/14/2023 5:24:18 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -31.465767323895786 (T) = (0 -263.45592408926706) / Math.Sqrt((47.69462150191671 / (35)) + (39.990956994292006 / (4))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (4) - 2, .025) and -0.6738014109962288 = (157.39975026814017 - 263.45592408926706) / 157.39975026814017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```
#### JIT Disasms



### Docs

[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

</details>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Perf] Linux/x64: 1 Regression on 10/15/2023 11:56:34 AM #23222

performanceautofiler
openedon Oct 17, 2023

Run Information

Regressions in System.Globalization.Tests.StringSearch

Repro

Prerequisites (Build files either built locally or downloaded from payload above)

Payloads

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False))

ETL Files

Histogram

Description of detection logic

JIT Disasms

Docs

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	765798c49d5d908e512af0931b18fed782b4cf00
Compare	b9159ab91a5dbd124dd15d78ed073c52eba8c31b
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono