[Perf] Windows/x64: 9 Regressions on 6/8/2023 9:27:53 AM #18696
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 79effd2c09a7a3049d0b29c998e444ba51864131 |
| Compare | 9b53f4b5238e4994cc65ecee7e3d9023c7197770 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Numerics.Tests.Perf_BigInteger
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Parse - Duration of single invocation | 1.50 μs | 1.65 μs | 1.10 | 0.06 | False | |||||
| ToByteArray - Duration of single invocation | 10.54 ns | 13.02 ns | 1.24 | 0.07 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'Payloads
Histogram
System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6512824458977617 > 1.5581484651052109.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -13.765416913800705 (T) = (0 -1698.4709850486781) / Math.Sqrt((568.4663623786745 / (23)) + (3727.6387787701256 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.14076351644647855 = (1488.8896432623296 - 1698.4709850486781) / 1488.8896432623296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 123)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.020977993155844 > 11.0731713613289.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -14.18749931118298 (T) = (0 -13.093076394431893) / Math.Sqrt((0.608359901915123 / (28)) + (0.021766873956958848 / (13))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (13) - 2, .025) and -0.19869932553910163 = (10.922736098598728 - 13.093076394431893) / 10.922736098598728 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 79effd2c09a7a3049d0b29c998e444ba51864131 |
| Compare | 9b53f4b5238e4994cc65ecee7e3d9023c7197770 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Collections.CreateAddAndClear<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| List - Duration of single invocation | 1.20 μs | 1.32 μs | 1.11 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'Payloads
Histogram
System.Collections.CreateAddAndClear<Int32>.List(Size: 512)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.322639286760637 > 1.2650694855171205.
IsChangePoint: Marked as a change because one of 5/27/2023 2:31:56 PM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -6.813721543795807 (T) = (0 -1316.2242560244715) / Math.Sqrt((2050.456800240801 / (23)) + (888.3461671786545 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.06487581496834331 = (1236.035448944439 - 1316.2242560244715) / 1236.035448944439 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 79effd2c09a7a3049d0b29c998e444ba51864131 |
| Compare | 9b53f4b5238e4994cc65ecee7e3d9023c7197770 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Memory.ReadOnlySpan
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IndexOfString - Duration of single invocation | 7.50 ns | 9.41 ns | 1.26 | 0.01 | False | |||||
| IndexOfString - Duration of single invocation | 8.52 ns | 9.95 ns | 1.17 | 0.11 | False | |||||
| IndexOfString - Duration of single invocation | 9.71 ns | 10.82 ns | 1.11 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'Payloads
Histogram
System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.414878783162923 > 7.939471702915738.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -49.5333280879812 (T) = (0 -9.365118048953883) / Math.Sqrt((0.016600248413385886 / (23)) + (0.0091753725006387 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.2279121617097903 = (7.626863175549582 - 9.365118048953883) / 7.626863175549582 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.949742613109567 > 8.936788096627017.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/17/2023 11:29:15 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -16.976696619670403 (T) = (0 -10.068058936319073) / Math.Sqrt((0.2759391959284162 / (23)) + (0.030012349820857404 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.24549990812806016 = (8.08354851784051 - 10.068058936319073) / 8.08354851784051 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.818295540563492 > 10.546717122576734.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -12.977948150668956 (T) = (0 -11.117068240134408) / Math.Sqrt((0.048958932771129164 / (23)) + (0.12385319857790736 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.12461397581741802 = (9.885230380543726 - 11.117068240134408) / 9.885230380543726 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 79effd2c09a7a3049d0b29c998e444ba51864131 |
| Compare | 9b53f4b5238e4994cc65ecee7e3d9023c7197770 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_Enum
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Parse_Flags - Duration of single invocation | 165.23 ns | 179.21 ns | 1.08 | 0.22 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'Payloads
Histogram
System.Tests.Perf_Enum.Parse_Flags(text: "Red, Orange, Yellow, Green, Blue")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 179.21464460712772 > 174.0045730853184.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -12.127693407802253 (T) = (0 -180.41879902453772) / Math.Sqrt((28.49786501897951 / (23)) + (25.99757149435359 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.12373698395332006 = (160.55251504655675 - 180.41879902453772) / 160.55251504655675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 79effd2c09a7a3049d0b29c998e444ba51864131 |
| Compare | 9b53f4b5238e4994cc65ecee7e3d9023c7197770 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Globalization.Tests.Perf_NumberCultureInfo
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToString - Duration of single invocation | 98.68 ns | 108.97 ns | 1.10 | 0.07 | False | |||||
| ToString - Duration of single invocation | 100.33 ns | 105.77 ns | 1.05 | 0.08 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'Payloads
Histogram
System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: )
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 108.96561716913666 > 103.36878133995924.
IsChangePoint: Marked as a change because one of 3/31/2023 8:05:26 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -13.312826185260775 (T) = (0 -107.50960224853989) / Math.Sqrt((2.9779874790886027 / (23)) + (8.822377876853734 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.10799949244673543 = (97.03037138684265 - 107.50960224853989) / 97.03037138684265 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.76987261880637 > 103.75141318354456.
IsChangePoint: Marked as a change because one of 3/31/2023 8:05:26 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -10.3447287304074 (T) = (0 -106.6890107273297) / Math.Sqrt((10.37459499065862 / (23)) + (5.1120496192723985 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.09066866592229494 = (97.81981830119872 - 106.6890107273297) / 97.81981830119872 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository