Skip to content

[Perf] Linux/x64: 22 Improvements on 3/21/2023 4:52:34 PM #14531

Closed
@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 8c5cf110b4d2fa0b98cad4b60e4abba5a6562c58
Compare 7b82cdf019aff3227d882a15c12eeb3c189bb22d
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToArray - Duration of single invocation 86.39 ns 79.64 ns 0.92 0.16 False
ToArray - Duration of single invocation 93.40 ns 85.94 ns 0.92 0.11 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span&lt;Byte&gt;*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span&lt;Byte&gt;*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.ToArray(Size: 4)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.63540837268684 < 83.23548582145773.
IsChangePoint: Marked as a change because one of 2/6/2023 6:07:54 PM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 27.344942915312316 (T) = (0 -81.11803828592747) / Math.Sqrt((9.83308772259105 / (299)) + (2.117483245732087 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.09052363407624792 = (89.19202447172572 - 81.11803828592747) / 89.19202447172572 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.ToArray(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.94212743009594 < 90.76897139440236.
IsChangePoint: Marked as a change because one of 1/27/2023 3:53:39 PM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 15.050407232370599 (T) = (0 -88.06956383886245) / Math.Sqrt((6.543394424363484 / (299)) + (6.913067273822993 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.07085761696091941 = (94.78586430510313 - 88.06956383886245) / 94.78586430510313 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 8c5cf110b4d2fa0b98cad4b60e4abba5a6562c58
Compare 7b82cdf019aff3227d882a15c12eeb3c189bb22d
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Remove_IntInt - Duration of single invocation 121.90 ns 112.65 ns 0.92 0.12 False
Remove_IntInt - Duration of single invocation 123.32 ns 109.37 ns 0.89 0.18 False
Insert - Duration of single invocation 165.58 ns 130.22 ns 0.79 0.14 True
Join_Array - Duration of single invocation 869.41 ns 713.32 ns 0.82 0.07 True
Join_List - Duration of single invocation 904.69 ns 828.83 ns 0.92 0.07 False
Concat_str_str_str_str - Duration of single invocation 473.97 ns 440.67 ns 0.93 0.04 False
Substring_IntInt - Duration of single invocation 92.34 ns 83.21 ns 0.90 0.13 False
GetChars - Duration of single invocation 125.15 ns 115.58 ns 0.92 0.09 False
TrimStart_CharArr - Duration of single invocation 166.75 ns 147.43 ns 0.88 0.15 False
Substring_Int - Duration of single invocation 96.19 ns 82.55 ns 0.86 0.10 False
Substring_IntInt - Duration of single invocation 100.81 ns 91.93 ns 0.91 0.09 False
Substring_IntInt - Duration of single invocation 92.11 ns 84.05 ns 0.91 0.12 False
Insert - Duration of single invocation 171.55 ns 135.72 ns 0.79 0.14 True
Remove_Int - Duration of single invocation 108.56 ns 97.57 ns 0.90 0.08 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_String*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_String*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 112.64652883643473 < 117.18116423186719.
IsChangePoint: Marked as a change because one of 3/16/2023 9:46:36 PM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 76.94758755642222 (T) = (0 -111.63011778970314) / Math.Sqrt((31.490693779939654 / (299)) + (1.1302720914626398 / (38))) is greater than 1.967070609662329 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (38) - 2, .975) and 0.20212438049471332 = (139.90917263384745 - 111.63011778970314) / 139.90917263384745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 109.36992911361848 < 119.22033614339789.
IsChangePoint: Marked as a change because one of 3/16/2023 9:46:36 PM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 48.42583786474686 (T) = (0 -112.35989059636388) / Math.Sqrt((38.72597310019643 / (299)) + (9.667924038579743 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.20934467561534223 = (142.10982602793456 - 112.35989059636388) / 142.10982602793456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 130.2228122082001 < 156.90673318481677.
IsChangePoint: Marked as a change because one of 2/10/2023 11:47:22 AM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 58.14932916509284 (T) = (0 -133.61159207659213) / Math.Sqrt((31.414424400396047 / (299)) + (17.2040968957118 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.2433662708777424 = (176.586883367716 - 133.61159207659213) / 176.586883367716 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Tests.Perf_String.Join_Array

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 713.3189878582026 < 799.6034000589287.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 44.330522018788294 (T) = (0 -726.1858980506381) / Math.Sqrt((414.13771805094694 / (299)) + (254.62049293747097 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.1465615228582612 = (850.8942560015763 - 726.1858980506381) / 850.8942560015763 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Tests.Perf_String.Join_List

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 828.8279751788907 < 853.3300253692461.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 25.785621916484484 (T) = (0 -782.9980553889037) / Math.Sqrt((424.1064729510099 / (299)) + (656.1516500263571 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.123312648477001 = (893.1326019802426 - 782.9980553889037) / 893.1326019802426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Concat_str_str_str_str(size: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 440.67416681498617 < 452.6833995793008.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 15.82916480449482 (T) = (0 -447.257483926475) / Math.Sqrt((83.58919185208077 / (299)) + (148.5829940819434 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06678959985927858 = (479.26757337791327 - 447.257483926475) / 479.26757337791327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 83.21073431730177 < 89.96167659256858.
IsChangePoint: Marked as a change because one of 1/27/2023 11:36:27 AM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 20.500822088285393 (T) = (0 -85.61346141580444) / Math.Sqrt((7.367785667934834 / (299)) + (6.7463582015029315 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.09620948806161685 = (94.72710798013027 - 85.61346141580444) / 94.72710798013027 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.GetChars(size: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.58438310544464 < 118.33063588336606.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 22.421327443184833 (T) = (0 -113.27229702906773) / Math.Sqrt((7.978976367185842 / (299)) + (3.9524843786622865 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06614188621445019 = (121.29497549675888 - 113.27229702906773) / 121.29497549675888 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimStart_CharArr(s: " Test", c: [' ', ' '])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 147.43034932082608 < 149.09756584150895.
IsChangePoint: Marked as a change because one of 2/6/2023 11:23:26 AM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 9.555877232321794 (T) = (0 -150.3134595898578) / Math.Sqrt((16.394487884808857 / (299)) + (27.084165678050358 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.05215966920500105 = (158.58521177694848 - 150.3134595898578) / 158.58521177694848 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 7)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.55336368115597 < 88.55856778540316.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 31.716948664859064 (T) = (0 -84.66551367679708) / Math.Sqrt((5.596025874876032 / (299)) + (2.9783775745240826 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.10355333342520964 = (94.44567851458841 - 84.66551367679708) / 94.44567851458841 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 91.93411394195775 < 95.36936453113924.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 26.409636465819805 (T) = (0 -92.95053109108991) / Math.Sqrt((7.820958320955878 / (299)) + (3.3969777246214714 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.08727446485669646 = (101.83842514770457 - 92.95053109108991) / 101.83842514770457 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 84.05342164202729 < 88.32502985879222.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 34.97009036618064 (T) = (0 -84.76182547349472) / Math.Sqrt((9.624592947046866 / (299)) + (1.6950547903748856 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.10191217979646784 = (94.3803307056155 - 84.76182547349472) / 94.3803307056155 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 135.71825664341148 < 161.41120444773063.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 56.93195941340913 (T) = (0 -134.73570611202734) / Math.Sqrt((33.263601698484244 / (299)) + (21.929894272559057 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.2574907635423586 = (181.45997315107303 - 134.73570611202734) / 181.45997315107303 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 7)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 97.57352707006368 < 101.67663176620673.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 25.933201462331954 (T) = (0 -98.19460124404738) / Math.Sqrt((7.349172344572044 / (299)) + (3.5446408376155993 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.08235203215478562 = (107.00683125210222 - 98.19460124404738) / 107.00683125210222 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions