[Perf] Windows/arm64: 9 Improvements on 5/9/2023 6:35:22 PM #17921
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | aaa1de18a7f97774fd44a1728dafc0d86894800e |
| Compare | 3747edb1812106f0b88a42b724c3e44b69dc79c4 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Tests.Perf_StringBuilder
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| AppendLine_Strings - Duration of single invocation | 300.03 ns | 261.27 ns | 0.87 | 0.18 | False | |||||
| Append_Strings - Duration of single invocation | 251.10 ns | 233.15 ns | 0.93 | 0.13 | False | |||||
| Append_Strings - Duration of single invocation | 109.84 μs | 85.83 μs | 0.78 | 0.03 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'Payloads
Histogram
System.Text.Tests.Perf_StringBuilder.AppendLine_Strings
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 261.27455128205133 < 285.9362508149946.
IsChangePoint: Marked as a change because one of 5/9/2023 1:22:51 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 25.58770751363999 (T) = (0 -262.6421898709379) / Math.Sqrt((144.15690377914453 / (299)) + (55.62016354545464 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.1445926310447028 = (307.0375582474826 - 262.6421898709379) / 307.0375582474826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Tests.Perf_StringBuilder.Append_Strings(repeat: 1)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 233.14818251295233 < 239.2556145702455.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/9/2023 11:02:55 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 19.973673394075686 (T) = (0 -228.18113551257903) / Math.Sqrt((120.82644246500902 / (299)) + (36.22886629345964 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.1152730452661608 = (257.91136383001344 - 228.18113551257903) / 257.91136383001344 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Tests.Perf_StringBuilder.Append_Strings(repeat: 1000)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 85.83410177595627 < 104.65329159902596.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 5/9/2023 1:22:51 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 80.59750307004008 (T) = (0 -86913.82013732608) / Math.Sqrt((10314708.560047647 / (299)) + (853436.3994183076 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.20066945511888717 = (108733.26522290359 - 86913.82013732608) / 108733.26522290359 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | aaa1de18a7f97774fd44a1728dafc0d86894800e |
| Compare | 3747edb1812106f0b88a42b724c3e44b69dc79c4 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Enum
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| InterpolateIntoStringBuilder_Flags - Duration of single invocation | 114.62 ns | 106.80 ns | 0.93 | 0.00 | False | |||||
| GetName_Generic_NonFlags - Duration of single invocation | 3.73 ns | 1.95 ns | 0.52 | 0.65 | False | |||||
| ToString_NonFlags_Small - Duration of single invocation | 38.65 ns | 35.08 ns | 0.91 | 0.12 | False | |||||
| InterpolateIntoStringBuilder_NonFlags - Duration of single invocation | 124.87 ns | 118.14 ns | 0.95 | 0.00 | False | |||||
| InterpolateIntoStringBuilder_Flags - Duration of single invocation | 118.00 ns | 109.95 ns | 0.93 | 0.00 | False | |||||
| InterpolateIntoStringBuilder_NonFlags - Duration of single invocation | 142.08 ns | 134.46 ns | 0.95 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'Payloads
Histogram
System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: Red)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 106.79534124341879 < 108.67092721381478.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 4/12/2023 8:39:35 PM, 5/9/2023 1:22:51 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 70.4172375268466 (T) = (0 -106.59419395913523) / Math.Sqrt((2.5672561147599766 / (299)) + (0.23474926589570216 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.08397360503741919 = (116.36585424319519 - 106.59419395913523) / 116.36585424319519 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Enum.GetName_Generic_NonFlags
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.951063875012198 < 3.1455825330684015.
IsChangePoint: Marked as a change because one of 4/11/2023 12:48:48 AM, 5/9/2023 11:02:55 PM, 5/10/2023 10:36:48 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 13.367210595355502 (T) = (0 -2.2124497349919405) / Math.Sqrt((0.243318382005946 / (299)) + (0.11314203939542682 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.34917158763141515 = (3.3994363075516127 - 2.2124497349919405) / 3.3994363075516127 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Enum.ToString_NonFlags_Small(value: -1)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.07765968307145 < 36.76732871160749.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 7.953205621649113 (T) = (0 -35.630339055844125) / Math.Sqrt((1.1373546075769823 / (299)) + (0.572056216895199 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.05009062404062573 = (37.509198200995506 - 35.630339055844125) / 37.509198200995506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Enum.InterpolateIntoStringBuilder_NonFlags(value: 42)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 118.1399986681866 < 119.85883923520441.
IsChangePoint: Marked as a change because one of 4/11/2023 12:48:48 AM, 5/9/2023 1:22:51 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 58.43565730437548 (T) = (0 -118.16201364494559) / Math.Sqrt((4.059321136798374 / (299)) + (0.40700138088930216 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08238299897490296 = (128.77051483673833 - 118.16201364494559) / 128.77051483673833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: 32)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 109.95232130032667 < 111.86486264975075.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 5/9/2023 1:22:51 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 58.371244987056144 (T) = (0 -109.57865369236457) / Math.Sqrt((5.457922050050556 / (299)) + (0.44850836119221205 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09478646427327121 = (121.05282275125504 - 109.57865369236457) / 121.05282275125504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Enum.InterpolateIntoStringBuilder_NonFlags(value: Saturday)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 134.45926367317216 < 135.01565729958648.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/9/2023 1:22:51 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 57.43680057739822 (T) = (0 -134.54643197453314) / Math.Sqrt((2.5986129498042057 / (299)) + (0.39142497436027796 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.06495808261252536 = (143.89347629511465 - 134.54643197453314) / 143.89347629511465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository