Closed
Description
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 8c5cf110b4d2fa0b98cad4b60e4abba5a6562c58 |
Compare | 7b82cdf019aff3227d882a15c12eeb3c189bb22d |
Diff | Diff |
Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono |
Improvements in System.Memory.Span<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToArray - Duration of single invocation | 86.39 ns | 79.64 ns | 0.92 | 0.16 | False | |||||
ToArray - Duration of single invocation | 93.40 ns | 85.94 ns | 0.92 | 0.11 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifacts
or build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
- CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release
, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
- Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release
, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Byte>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Byte>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages
Windows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Byte>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Byte>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages
Payloads
Histogram
System.Memory.Span<Byte>.ToArray(Size: 4)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.63540837268684 < 83.23548582145773.
IsChangePoint: Marked as a change because one of 2/6/2023 6:07:54 PM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 27.344942915312316 (T) = (0 -81.11803828592747) / Math.Sqrt((9.83308772259105 / (299)) + (2.117483245732087 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.09052363407624792 = (89.19202447172572 - 81.11803828592747) / 89.19202447172572 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Byte>.ToArray(Size: 33)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.94212743009594 < 90.76897139440236.
IsChangePoint: Marked as a change because one of 1/27/2023 3:53:39 PM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 15.050407232370599 (T) = (0 -88.06956383886245) / Math.Sqrt((6.543394424363484 / (299)) + (6.913067273822993 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.07085761696091941 = (94.78586430510313 - 88.06956383886245) / 94.78586430510313 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 8c5cf110b4d2fa0b98cad4b60e4abba5a6562c58 |
Compare | 7b82cdf019aff3227d882a15c12eeb3c189bb22d |
Diff | Diff |
Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono |
Improvements in System.Tests.Perf_String
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Remove_IntInt - Duration of single invocation | 121.90 ns | 112.65 ns | 0.92 | 0.12 | False | |||||
Remove_IntInt - Duration of single invocation | 123.32 ns | 109.37 ns | 0.89 | 0.18 | False | |||||
Insert - Duration of single invocation | 165.58 ns | 130.22 ns | 0.79 | 0.14 | True | |||||
Join_Array - Duration of single invocation | 869.41 ns | 713.32 ns | 0.82 | 0.07 | True | |||||
Join_List - Duration of single invocation | 904.69 ns | 828.83 ns | 0.92 | 0.07 | False | |||||
Concat_str_str_str_str - Duration of single invocation | 473.97 ns | 440.67 ns | 0.93 | 0.04 | False | |||||
Substring_IntInt - Duration of single invocation | 92.34 ns | 83.21 ns | 0.90 | 0.13 | False | |||||
GetChars - Duration of single invocation | 125.15 ns | 115.58 ns | 0.92 | 0.09 | False | |||||
TrimStart_CharArr - Duration of single invocation | 166.75 ns | 147.43 ns | 0.88 | 0.15 | False | |||||
Substring_Int - Duration of single invocation | 96.19 ns | 82.55 ns | 0.86 | 0.10 | False | |||||
Substring_IntInt - Duration of single invocation | 100.81 ns | 91.93 ns | 0.91 | 0.09 | False | |||||
Substring_IntInt - Duration of single invocation | 92.11 ns | 84.05 ns | 0.91 | 0.12 | False | |||||
Insert - Duration of single invocation | 171.55 ns | 135.72 ns | 0.79 | 0.14 | True | |||||
Remove_Int - Duration of single invocation | 108.56 ns | 97.57 ns | 0.90 | 0.08 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifacts
or build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
- CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release
, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
- Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release
, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_String*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages
Windows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_String*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages
Payloads
Histogram
System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 112.64652883643473 < 117.18116423186719.
IsChangePoint: Marked as a change because one of 3/16/2023 9:46:36 PM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 76.94758755642222 (T) = (0 -111.63011778970314) / Math.Sqrt((31.490693779939654 / (299)) + (1.1302720914626398 / (38))) is greater than 1.967070609662329 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (38) - 2, .975) and 0.20212438049471332 = (139.90917263384745 - 111.63011778970314) / 139.90917263384745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 109.36992911361848 < 119.22033614339789.
IsChangePoint: Marked as a change because one of 3/16/2023 9:46:36 PM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 48.42583786474686 (T) = (0 -112.35989059636388) / Math.Sqrt((38.72597310019643 / (299)) + (9.667924038579743 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.20934467561534223 = (142.10982602793456 - 112.35989059636388) / 142.10982602793456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 130.2228122082001 < 156.90673318481677.
IsChangePoint: Marked as a change because one of 2/10/2023 11:47:22 AM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 58.14932916509284 (T) = (0 -133.61159207659213) / Math.Sqrt((31.414424400396047 / (299)) + (17.2040968957118 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.2433662708777424 = (176.586883367716 - 133.61159207659213) / 176.586883367716 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Tests.Perf_String.Join_Array
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 713.3189878582026 < 799.6034000589287.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 44.330522018788294 (T) = (0 -726.1858980506381) / Math.Sqrt((414.13771805094694 / (299)) + (254.62049293747097 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.1465615228582612 = (850.8942560015763 - 726.1858980506381) / 850.8942560015763 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Tests.Perf_String.Join_List
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 828.8279751788907 < 853.3300253692461.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 25.785621916484484 (T) = (0 -782.9980553889037) / Math.Sqrt((424.1064729510099 / (299)) + (656.1516500263571 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.123312648477001 = (893.1326019802426 - 782.9980553889037) / 893.1326019802426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.Concat_str_str_str_str(size: 100)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 440.67416681498617 < 452.6833995793008.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 15.82916480449482 (T) = (0 -447.257483926475) / Math.Sqrt((83.58919185208077 / (299)) + (148.5829940819434 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06678959985927858 = (479.26757337791327 - 447.257483926475) / 479.26757337791327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 83.21073431730177 < 89.96167659256858.
IsChangePoint: Marked as a change because one of 1/27/2023 11:36:27 AM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 20.500822088285393 (T) = (0 -85.61346141580444) / Math.Sqrt((7.367785667934834 / (299)) + (6.7463582015029315 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.09620948806161685 = (94.72710798013027 - 85.61346141580444) / 94.72710798013027 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.GetChars(size: 100)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.58438310544464 < 118.33063588336606.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 22.421327443184833 (T) = (0 -113.27229702906773) / Math.Sqrt((7.978976367185842 / (299)) + (3.9524843786622865 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.06614188621445019 = (121.29497549675888 - 113.27229702906773) / 121.29497549675888 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.TrimStart_CharArr(s: " Test", c: [' ', ' '])
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 147.43034932082608 < 149.09756584150895.
IsChangePoint: Marked as a change because one of 2/6/2023 11:23:26 AM, 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 9.555877232321794 (T) = (0 -150.3134595898578) / Math.Sqrt((16.394487884808857 / (299)) + (27.084165678050358 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.05215966920500105 = (158.58521177694848 - 150.3134595898578) / 158.58521177694848 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 7)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.55336368115597 < 88.55856778540316.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 31.716948664859064 (T) = (0 -84.66551367679708) / Math.Sqrt((5.596025874876032 / (299)) + (2.9783775745240826 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.10355333342520964 = (94.44567851458841 - 84.66551367679708) / 94.44567851458841 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 91.93411394195775 < 95.36936453113924.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 26.409636465819805 (T) = (0 -92.95053109108991) / Math.Sqrt((7.820958320955878 / (299)) + (3.3969777246214714 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.08727446485669646 = (101.83842514770457 - 92.95053109108991) / 101.83842514770457 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 84.05342164202729 < 88.32502985879222.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 34.97009036618064 (T) = (0 -84.76182547349472) / Math.Sqrt((9.624592947046866 / (299)) + (1.6950547903748856 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.10191217979646784 = (94.3803307056155 - 84.76182547349472) / 94.3803307056155 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 135.71825664341148 < 161.41120444773063.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 56.93195941340913 (T) = (0 -134.73570611202734) / Math.Sqrt((33.263601698484244 / (299)) + (21.929894272559057 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.2574907635423586 = (181.45997315107303 - 134.73570611202734) / 181.45997315107303 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 7)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 97.57352707006368 < 101.67663176620673.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 3/28/2023 8:13:28 AM falls between 3/19/2023 2:30:51 PM and 3/28/2023 8:13:28 AM.
IsImprovementStdDev: Marked as improvement because 25.933201462331954 (T) = (0 -98.19460124404738) / Math.Sqrt((7.349172344572044 / (299)) + (3.5446408376155993 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.08235203215478562 = (107.00683125210222 - 98.19460124404738) / 107.00683125210222 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository