Description
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 0c419b05455970ea4b009c14984785b59ba7b624 |
Compare | 3e8f17a65a068fca3d19fa5cd43a7e1cd414a5ae |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Count - Duration of single invocation | 187.70 μs | 161.45 μs | 0.86 | 0.00 | True | |||||
Count - Duration of single invocation | 186.98 μs | 154.40 μs | 0.83 | 0.00 | True | |||||
Count - Duration of single invocation | 224.07 μs | 173.15 μs | 0.77 | 0.00 | True | |||||
Count - Duration of single invocation | 1.13 ms | 722.60 μs | 0.64 | 0.00 | True | |||||
Count - Duration of single invocation | 869.74 μs | 258.17 μs | 0.30 | 0.00 | True | |||||
Count - Duration of single invocation | 2.10 ms | 1.46 ms | 0.69 | 0.00 | True | |||||
Count - Duration of single invocation | 223.21 μs | 169.38 μs | 0.76 | 0.00 | True | |||||
Count - Duration of single invocation | 224.46 μs | 172.65 μs | 0.77 | 0.00 | True | |||||
Count - Duration of single invocation | 1.85 ms | 1.16 ms | 0.63 | 0.00 | True | |||||
Count - Duration of single invocation | 859.49 μs | 243.96 μs | 0.28 | 0.00 | True | |||||
Count - Duration of single invocation | 677.66 μs | 208.52 μs | 0.31 | 0.00 | True | |||||
Count - Duration of single invocation | 224.24 μs | 174.63 μs | 0.78 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: Compiled)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 161.45239613402063 < 178.54478942752624.
IsChangePoint: Marked as a change because one of 1/20/2023 1:51:38 AM, 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 189.0242713305524 (T) = (0 -161357.07914584648) / Math.Sqrt((2954666.7526430674 / (299)) + (53663.421190109046 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1384755485575155 = (187292.51256383953 - 161357.07914584648) / 187292.51256383953 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: Compiled)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 154.40302017973858 < 178.0077396527778.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 198.16137776886077 (T) = (0 -154150.79353758172) / Math.Sqrt((2727553.548352462 / (299)) + (87605.74128458113 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.17343775472392575 = (186496.29162059646 - 154150.79353758172) / 186496.29162059646 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: None)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 173.15448232600733 < 214.83578307791237.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 304.29025873059436 (T) = (0 -173457.17028515466) / Math.Sqrt((2830058.4405718627 / (299)) + (117704.11017444808 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.23027497551613113 = (225349.5271268653 - 173457.17028515466) / 225349.5271268653 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: Compiled)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 722.6039196969697 < 1.0667360553669543.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 267.94940492369204 (T) = (0 -719667.7800189393) / Math.Sqrt((63652316.863796055 / (299)) + (11676442.862547817 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.3536167926262408 = (1113376.3560209642 - 719667.7800189393) / 1113376.3560209642 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: NonBacktracking)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 258.17198953168037 < 827.065313702626.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 649.8388329296065 (T) = (0 -256615.96647262783) / Math.Sqrt((54312116.29575653 / (299)) + (4375510.596201315 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.7073361507184001 = (876828.3718762717 - 256615.96647262783) / 876828.3718762717 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: NonBacktracking)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4569572937254904 < 2.0230723778184774.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 105.75221277745864 (T) = (0 -1441455.7552643332) / Math.Sqrt((556202764.9988482 / (299)) + (234667126.87170035 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.3195429573603697 = (2118364.0772864004 - 1441455.7552643332) / 2118364.0772864004 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: NonBacktracking)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 169.37525595480224 < 212.01968220622993.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 218.64132909780284 (T) = (0 -170128.9606830052) / Math.Sqrt((3684122.589437716 / (299)) + (308114.07721851964 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.24487878267432373 = (225300.19919918402 - 170128.9606830052) / 225300.19919918402 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: None)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 172.6529139194139 < 214.82216088257445.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 318.48424301234644 (T) = (0 -172676.51533392203) / Math.Sqrt((2738784.8889064575 / (299)) + (117764.54391902909 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.238347900161157 = (226713.10874145615 - 172676.51533392203) / 226713.10874145615 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: None)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1626997375637755 < 1.7557553766232639.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 736.5461619091939 (T) = (0 -1161877.6961663833) / Math.Sqrt((196378240.41861135 / (299)) + (1482005.3002042996 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.3760371132134082 = (1862094.23792824 - 1161877.6961663833) / 1862094.23792824 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: None)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 243.96294205729166 < 816.9246630208333.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 1068.1672812902596 (T) = (0 -246137.84055718317) / Math.Sqrt((34762743.30056665 / (299)) + (1322783.1629978896 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.7157683220457917 = (865976.1020614938 - 246137.84055718317) / 865976.1020614938 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: Compiled)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 208.51603384615387 < 642.9217503069195.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 997.4031809102406 (T) = (0 -208972.83323031137) / Math.Sqrt((56246091.60047933 / (299)) + (128151.7516098705 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.6859752535713453 = (665466.1315928784 - 208972.83323031137) / 665466.1315928784 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: NonBacktracking)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 174.62825424836598 < 212.1489064785805.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 181.59033506379646 (T) = (0 -173903.22003131604) / Math.Sqrt((3834234.634671544 / (299)) + (382823.80076115887 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.22423668172069236 = (224170.46015664216 - 173903.22003131604) / 224170.46015664216 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 0c419b05455970ea4b009c14984785b59ba7b624 |
Compare | 3e8f17a65a068fca3d19fa5cd43a7e1cd414a5ae |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Count - Duration of single invocation | 3.46 secs | 1.93 secs | 0.56 | 0.00 | True | |||||
Count - Duration of single invocation | 3.49 secs | 1.94 secs | 0.55 | 0.00 | True | |||||
Count - Duration of single invocation | 2.90 secs | 1.71 secs | 0.59 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*'
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.9328833213333332 < 3.2816533564583334.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 563.6309770995017 (T) = (0 -1936268818.277778) / Math.Sqrt((673323160696068.5 / (299)) + (32481419330733.51 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.44626967749579566 = (3496772236.566612 - 1936268818.277778) / 3496772236.566612 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase, NonBacktracking)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.9370483136666667 < 3.31820412239375.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 480.09961596847285 (T) = (0 -1928990069.2222223) / Math.Sqrt((724630184723321.2 / (299)) + (52645495348257.336 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.45440172014697106 = (3535550129.926813 - 1928990069.2222223) / 3535550129.926813 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase, Compiled)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.710718181 < 2.7467378577583337.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 572.6325801912895 (T) = (0 -1710630719.3194447) / Math.Sqrt((798602819988746.8 / (299)) + (7913616771229.172 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.4007126782938584 = (2854441696.596155 - 1710630719.3194447) / 2854441696.596155 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 0c419b05455970ea4b009c14984785b59ba7b624 |
Compare | 3e8f17a65a068fca3d19fa5cd43a7e1cd414a5ae |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Count - Duration of single invocation | 8.00 ms | 5.50 ms | 0.69 | 0.01 | True | |||||
Count - Duration of single invocation | 9.35 ms | 5.57 ms | 0.60 | 0.00 | True | |||||
Count - Duration of single invocation | 9.46 ms | 5.62 ms | 0.59 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: Compiled)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.495775105555555 < 7.603646308802082.
IsChangePoint: Marked as a change because one of 3/17/2023 7:47:41 PM, 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 262.0589586272643 (T) = (0 -5492602.111226852) / Math.Sqrt((13733737711.388384 / (299)) + (213302725.23995885 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.30103111472788496 = (7858149.664399627 - 5492602.111226852) / 7858149.664399627 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: NonBacktracking)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.571410901515152 < 8.94796995453649.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 506.2865147039224 (T) = (0 -5590962.161386083) / Math.Sqrt((8062674531.893575 / (299)) + (208126361.9338655 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.41555652735357906 = (9566300.973590523 - 5590962.161386083) / 9566300.973590523 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: None)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.619493005555555 < 8.95024752312271.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 238.67019508300424 (T) = (0 -5594351.001633725) / Math.Sqrt((7954246667.587818 / (299)) + (1497408857.2665954 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.4148576061205017 = (9560666.019331018 - 5594351.001633725) / 9560666.019331018 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | ubuntu 20.04 |
Queue | AmpereUbuntu |
Baseline | 0c419b05455970ea4b009c14984785b59ba7b624 |
Compare | 3e8f17a65a068fca3d19fa5cd43a7e1cd414a5ae |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_BigInteger
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Ctor_ByteArray - Duration of single invocation | 144.75 ns | 135.66 ns | 0.94 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'
Payloads
Histogram
System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 135.6565623972308 < 137.4694783668496.
IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 4/18/2023 2:18:01 AM, 5/1/2023 3:45:46 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 17.634123699680227 (T) = (0 -134.01392205478814) / Math.Sqrt((7.329755325302433 / (299)) + (1.7233452258943962 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06843999604622368 = (143.85967783717544 - 134.01392205478814) / 143.85967783717544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository