[Perf] Linux/arm64: 2 Regressions on 4/3/2024 11:48:33 PM

### Run Information

Name | Value
-- | --
Architecture | arm64
OS | ubuntu 22.04
Queue | AmpereUbuntu
Baseline | [0b542b960dc18883347e62e51d0840476a1ba966](https://github.com/dotnet/runtime/commit/0b542b960dc18883347e62e51d0840476a1ba966)
Compare | [223e9e71954433544106d5c90e06b5c8113e51d9](https://github.com/dotnet/runtime/commit/223e9e71954433544106d5c90e06b5c8113e51d9)
Diff | [Diff](https://github.com/dotnet/runtime/compare/0b542b960dc18883347e62e51d0840476a1ba966...223e9e71954433544106d5c90e06b5c8113e51d9)
Configs | CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono


### Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio
-- | -- | -- | -- | -- | -- | -- | -- | --
|<ul><li>[Email_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>)</li><li>📝 - [Benchmark Source](<https://github.com/dotnet/performance/blob/main/src/benchmarks/micro/libraries/System.Text.RegularExpressions/Perf.Regex.Common.cs#L47-#L49>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWU3W7TQBCF75F4h72zDW7qhqqUSEakoUAgoSG1uEEo2toTe9H+WDPr1q4Qz87aCaFNpWJurN3j78zMjnc8B04VggJt6clPdlMAAjurhMyWUBoWx8zLjNVgD7HSVijw7lNnyHVadBzCmg4L4BkdKi70HjjGtBAWUuuyMaGZ79Unx17IPI7KLYIdnQDZz1xBG/ONd9mQBTVIoLaDJeSV5HhelwhEwmhyOlkaLADXK/cW6tXEKGX04NxVIFdTmnObFv5FaVt6xKa5NggTThAyR5ZCQhb8LXRZ6YnRa5FXyDvHN29Ddbu5ycD7zuJfzLPC4Zn39DHnbPZ1/gfHCh6H50ab8UXyX/xUW0DXCvfc+tZc0j+MTvokdLY1KJGiWbmGmTsuQaBK2/gP3GNjgx7Y1NBMXqvuq/fhP9K5zoWGPuwiN0lT9kOLhkTK5QKNMq3S1/ihuUKRvZfmiktx22l9bMvhsm8Gd4MB3Szp7qLfv/oT4+YMMDHldnV/jBJnu7Rclew1y7iFdiT9YTQ8PoiGB9GrJDodDV+OoqPB8OTkKIpOn0fRKIraFFQpxVHcAuOYrxSv/S5gW0HIngXsqtnLEO5GMWxv0pcKKnBxSjQ/3Bg/gJdAlbQxuqoz/85JNvr4GpDnLtKLIGRdqDZw/A4F6Ew2O6VtVrdpS0b3DpC1Z0wLjpbdCFswn1zzgOKdJ2R1amSldLxfU7PRKd4U4UArrITYe+ta534+zXbbWpivKWg1KqWwMa+Bgt8znO/ZHgUAAA==>)</li></ul> | 323.62 ns | 851.24 ns | 2.63 | 0.02 | True | | |
|<ul><li>[Email_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 22.04_LLVM=true_MonoAOT=true_MonoInterpreter=false_RunKind=micro_mono/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>)</li><li>📝 - [Benchmark Source](<https://github.com/dotnet/performance/blob/main/src/benchmarks/micro/libraries/System.Text.RegularExpressions/Perf.Regex.Common.cs#L51-#L53>)</li><li>[ADX - Test Multi Config Graph](<https://dataexplorer.azure.com/clusters/dotnetperf.westus/databases/PerformanceData?query=H4sIAAAAAAAACpWU227TQBCG75F4h72zDW7qplVpIxmRhlICTRtSixuEoq09sRftwZpZt3aFeHbWTgg9SMXcWLu/v39mdrzjGXCqEBRoSy9+stsCENhJJWS2gNKwOGZeZqwGu4uVtkKB95A6Qa7TouMQVrRbAM9oV3GhH4FjTAthIbUuGxOa+V59eOCFzOOo3CLY0gmQveAK2pjvvKuGLKhBArUdLCCvJMfTukQgEkaT08nSYA64Wrq3UC8nRimjB6euArmc0oWxM27Twr8sbWsYsWmuDcKEE4TMwaWQkAV/a11UemL0SuQV8s7xzVtT3W5mMvC+s/gX86xweOa9fM55fv519gfHCp6HZ0ab8WXyX/xUW0DXDffc+FZc0j+MTvosdLYxKJGiWbqemXsuQaBK2/hP3GNjgx7Y1NC5vFHdh+/Df6JTnQsNfdh5bpKm7IcWDYmUyzkaZVqlr/Fjc40iO5Pmmktx12l9bIvhom8Gd4kB3Tjp7q4/vP0T40YNMDHlZvVwkhJnu7Jclewty7iFdir9YTQ82ImGO9FxEh2Nhm9G0d5g/2jvYHh8+DqKRlHUpqBKKY7iDhjHfKl47XcB2wpC9ipg182jDOF2GsP2Jn2poAIXp0Tzw03yE3gBVEkbo6s68++dZK2PbwB57iLtByHrQrWB4w8oQGey2Spts7pNWzK6d4CsPWNacLTsVtiC+eSaBxRvPSGrUyMrpePHNTVrneJ1EQ60wkqIvfeude7/02y2rYX5moJWo1IKG/MaKPgN12eJ2CEFAAA=>)</li></ul> | 363.45 ns | 932.62 ns | 2.57 | 0.02 | True | | |

![graph](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/autofilereports/04_09_2024/refs/heads/main_arm64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Text.RegularExpressions.Tests.Perf_Regex_Common_1.png>)
![graph](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/autofilereports/04_09_2024/refs/heads/main_arm64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Text.RegularExpressions.Tests.Perf_Regex_Common_2.png>)
[Test Report](<https://pvscmdupload.z22.web.core.windows.net/autofilereport/autofilereports/04_09_2024/refs/heads/main_arm64_ubuntu%2022.04_LLVM%3Dtrue_MonoAOT%3Dtrue_MonoInterpreter%3Dfalse_RunKind%3Dmicro_mono_Regression/System.Text.RegularExpressions.Tests.Perf_Regex_Common.html>)

### Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

<details>
   <summary>Repro Steps</summary>


#### Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))
- Libraries build extracted to `runtime/artifacts` or build instructions: [Libraries README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/libraries/README.md) args: `-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0`
- CoreCLR product build extracted to `runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release`, build instructions: [CoreCLR README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/README.md) args: `-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0`
- AOT MONO build extracted to `runtime/artifacts/bin/mono/$RunOS.$RunArch.Release`, build instructions: [MONO README](https://github.com/dotnet/runtime/blob/main/docs/workflow/building/mono/README.md) args: `-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false`
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder

Linux
```cmd
# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.RegularExpressions.Tests.Perf_Regex_Common* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200
```

Windows
```cmd
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.RegularExpressions.Tests.Perf_Regex_Common* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200
```

</details>

<details>

### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: IgnoreCase, Compiled)

#### ETL Files

#### Histogram

#### JIT Disasms

### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: IgnoreCase, Compiled)

#### ETL Files

#### Histogram

#### JIT Disasms



### Docs

[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

</details>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Perf] Linux/arm64: 2 Regressions on 4/3/2024 11:48:33 PM #32511

performanceautofiler
openedon Apr 9, 2024

Run Information

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Repro

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: IgnoreCase, Compiled)

ETL Files

Histogram

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: IgnoreCase, Compiled)

ETL Files

Histogram

JIT Disasms

Docs

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	0b542b960dc18883347e62e51d0840476a1ba966
Compare	223e9e71954433544106d5c90e06b5c8113e51d9
Diff	Diff
Configs	CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Email_IsMatch - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	323.62 ns	851.24 ns	2.63	0.02	True
Email_IsNotMatch - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	363.45 ns	932.62 ns	2.57	0.02	True

[Perf] Linux/arm64: 2 Regressions on 4/3/2024 11:48:33 PM #32511

Description

performanceautofileropenedon Apr 9, 2024

Run Information

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Repro

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: IgnoreCase, Compiled)

ETL Files

Histogram

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: IgnoreCase, Compiled)

ETL Files

Histogram

JIT Disasms

Docs

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

performanceautofiler
openedon Apr 9, 2024