Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in System.Collections.IterateFor<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ImmutableList - Duration of single invocation | 10.91 μs | 12.02 μs | 1.10 | 0.03 | False | |||||
ReadOnlySpan - Duration of single invocation | 174.94 ns | 229.20 ns | 1.31 | 0.03 | False | |||||
Span - Duration of single invocation | 181.80 ns | 233.38 ns | 1.28 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor<String>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Collections.IterateFor<String>.ImmutableList(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 12.015544961946787 > 11.479808973732194.
IsChangePoint: Marked as a change because one of 9/3/2022 11:44:47 AM, 9/7/2022 1:05:52 PM, 10/10/2022 3:36:43 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -45.190373934483304 (T) = (0 -12017.913080523016) / Math.Sqrt((9488.831139762908 / (26)) + (2504.333491692935 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.09302411895706615 = (10995.103284628505 - 12017.913080523016) / 10995.103284628505 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.IterateFor<String>.ReadOnlySpan(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 229.19755406254157 > 183.81344391353085.
IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -935.8026626157678 (T) = (0 -229.38432863226546) / Math.Sqrt((0.0390402017718691 / (26)) + (0.03166299391386547 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.3099653923245103 = (175.1071669345608 - 229.38432863226546) / 175.1071669345608 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.IterateFor<String>.Span(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 233.37568339947686 > 189.14312176200895.
IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -495.86548527331774 (T) = (0 -233.54050095202567) / Math.Sqrt((0.17548571793951576 / (25)) + (0.07777650790385317 / (17))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (17) - 2, .025) and -0.2963901212754425 = (180.14677612805227 - 233.54050095202567) / 180.14677612805227 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in ByteMark
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
BenchNumericSortJagged - Duration of single invocation | 945.43 ms | 1.01 secs | 1.07 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'ByteMark*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
ByteMark.BenchNumericSortJagged
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.0072694792142858 > 991.2216186144232.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -45.3368673696619 (T) = (0 -1007302956.6448178) / Math.Sqrt((40029832692645.72 / (26)) + (5756409130033.691 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.06573779659177932 = (945169590.3684418 - 1007302956.6448178) / 945169590.3684418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in Benchstone.MDBenchF.MDSqMtx
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 707.20 ms | 770.24 ms | 1.09 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDSqMtx*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
Benchstone.MDBenchF.MDSqMtx.Test
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 770.2367917692308 > 742.7226221999999.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -435.67171563088385 (T) = (0 -770508306.4795625) / Math.Sqrt((204445020130.40134 / (26)) + (222226019656.2836 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.08910281771091552 = (707470675.8164694 - 770508306.4795625) / 707470675.8164694 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | 4fc3eb443a3f8f8917038fbdba9049caf288020c |
Diff | Diff |
Regressions in Benchstone.BenchI.BenchE
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 245.96 ms | 276.01 ms | 1.12 | 0.10 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.BenchE*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
Benchstone.BenchI.BenchE.Test
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 276.0140003333333 > 258.36238718500005.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -85.8428241289489 (T) = (0 -276092160.46769685) / Math.Sqrt((3531782407073.49 / (26)) + (21100826606.789955 / (16))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (16) - 2, .025) and -0.1301331254961949 = (244300564.45471957 - 276092160.46769685) / 244300564.45471957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in System.Text.Perf_Utf8Encoding
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetByteCount - Duration of single invocation | 8.16 μs | 10.27 μs | 1.26 | 0.05 | False | |||||
GetByteCount - Duration of single invocation | 8.93 μs | 11.15 μs | 1.25 | 0.04 | False | |||||
GetByteCount - Duration of single invocation | 18.49 μs | 22.85 μs | 1.24 | 0.06 | False | |||||
GetByteCount - Duration of single invocation | 7.09 μs | 8.85 μs | 1.25 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Text.Perf_Utf8Encoding.GetByteCount(Input: Chinese)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 10.267148920239597 > 8.582776780285263.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -326.5865160691413 (T) = (0 -10196.162173219669) / Math.Sqrt((81.97499416573011 / (26)) + (598.1781699274131 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.24739480049200627 = (8173.965587477218 - 10196.162173219669) / 8173.965587477218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Greek)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 11.153700850882498 > 9.3907775193202.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -185.3564745063029 (T) = (0 -11137.888360720259) / Math.Sqrt((3375.0352302401534 / (25)) + (54.17386678929373 / (17))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (17) - 2, .025) and -0.24321229844247833 = (8958.959282074375 - 11137.888360720259) / 8958.959282074375 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishMostlyAscii)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 22.850955701913758 > 19.554777491629466.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -231.57345920851665 (T) = (0 -22720.38068754999) / Math.Sqrt((4791.012087869746 / (25)) + (2014.0294453503732 / (17))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (17) - 2, .025) and -0.2187497514362704 = (18642.367443172407 - 22720.38068754999) / 18642.367443172407 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Cyrillic)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 8.852897015893229 > 7.450123206141866.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -195.39129114145314 (T) = (0 -8849.195825613026) / Math.Sqrt((1993.2347533712712 / (26)) + (46.63484749889969 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.24495196595833274 = (7108.062051856866 - 8849.195825613026) / 7108.062051856866 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in System.Tests.Perf_String
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToLowerInvariant - Duration of single invocation | 46.33 ns | 50.55 ns | 1.09 | 0.04 | False | |||||
ToLower - Duration of single invocation | 51.39 ns | 56.46 ns | 1.10 | 0.05 | False | |||||
IndexerCheckPathLength - Duration of single invocation | 83.65 ns | 97.50 ns | 1.17 | 0.02 | False | |||||
ToUpperInvariant - Duration of single invocation | 46.50 ns | 50.01 ns | 1.08 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_String.ToLowerInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 50.545799525192535 > 48.187547409833755.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -33.53563789349041 (T) = (0 -50.68131422663356) / Math.Sqrt((0.3376877914047586 / (26)) + (0.12194176271111679 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.10369668507273612 = (45.91960355783215 - 50.68131422663356) / 45.91960355783215 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 56.46445334095001 > 54.70612215723706.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/6/2022 2:13:23 PM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -19.21305755287296 (T) = (0 -56.5556259574959) / Math.Sqrt((0.3895160298079307 / (26)) + (0.49786491679787004 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.0769786982758955 = (52.51322616504318 - 56.5556259574959) / 52.51322616504318 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.IndexerCheckPathLength
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 97.49978743656838 > 87.63263907332612.
IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -26.330869035838653 (T) = (0 -97.50603407935617) / Math.Sqrt((6.927228010948752 / (26)) + (0.006336374555690262 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.16209601933171588 = (83.90531630547083 - 97.50603407935617) / 83.90531630547083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.ToUpperInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 50.00848445655875 > 48.7161059783996.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -20.529752935831283 (T) = (0 -49.75293801343357) / Math.Sqrt((0.30417466699880563 / (26)) + (0.31689091783106804 / (16))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (16) - 2, .025) and -0.07902887255393468 = (46.108996041666806 - 49.75293801343357) / 46.108996041666806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetHashCodeBenchmark - Duration of single invocation | 35.46 ns | 39.19 ns | 1.11 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<UInt16>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<UInt16>.GetHashCodeBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 39.18856311817259 > 37.251443680021694.
IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -116.87775942355182 (T) = (0 -39.171547872495644) / Math.Sqrt((0.00964257668937183 / (26)) + (0.01031681264075556 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.10289830760733029 = (35.516917201075316 - 39.171547872495644) / 35.516917201075316 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in Benchstone.BenchI.IniArray
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 45.73 ms | 52.66 ms | 1.15 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.IniArray*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
Benchstone.BenchI.IniArray.Test
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 52.6622479625 > 48.123812953125004.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -33.38832045773455 (T) = (0 -51938684.60073529) / Math.Sqrt((553845115117.5769 / (26)) + (312162731217.78326 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.1468251495236333 = (45289104.98894231 - 51938684.60073529) / 45289104.98894231 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in System.Collections.ContainsTrueComparer<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SortedSet - Duration of single invocation | 24.23 μs | 26.52 μs | 1.09 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrueComparer<Int32>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Collections.ContainsTrueComparer<Int32>.SortedSet(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 26.522895723684215 > 25.445657586976047.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 12:47:34 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -12.257124664061116 (T) = (0 -26188.36072730335) / Math.Sqrt((183894.73594610873 / (26)) + (178400.89765174556 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.06613670272590538 = (24563.792485846116 - 26188.36072730335) / 24563.792485846116 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 10d342cecc5cbfc57419f8a92af4787f29d76c08 |
Compare | be7d6f557b80c211d10f29b57fa43dc55e8d66cf |
Diff | Diff |
Regressions in System.Collections.IterateForEachNonGeneric<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ArrayList - Duration of single invocation | 2.71 μs | 3.09 μs | 1.14 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric<String>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Collections.IterateForEachNonGeneric<String>.ArrayList(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.091022231527094 > 2.8468039977966693.
IsChangePoint: Marked as a change because one of 8/18/2022 3:00:51 AM, 8/23/2022 8:42:14 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -85.34694523055988 (T) = (0 -3071.2773108938677) / Math.Sqrt((75.27488031752902 / (26)) + (263.396778652285 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.13528663676453886 = (2705.2879963836463 - 3071.2773108938677) / 2705.2879963836463 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository