-
Couldn't load subscription status.
- Fork 4
Closed
Description
Run Information
| Architecture | x86 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 285b8259af30e255980b1ebfb2ddd60ca291a83a |
| Compare | 0c64bdf3afc1fdc9166ad6c22c1990d3e0a6c509 |
| Diff | Diff |
Improvements in System.Memory.ReadOnlySpan
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IndexOfString - Duration of single invocation | 37.37 ns | 34.47 ns | 0.92 | 0.03 | False | |||||
| IndexOfString - Duration of single invocation | 219.19 ns | 19.87 ns | 0.09 | 0.01 | True | |||||
| IndexOfString - Duration of single invocation | 36.85 ns | 28.04 ns | 0.76 | 0.00 | True | |||||
| IndexOfString - Duration of single invocation | 17.20 ns | 14.36 ns | 0.83 | 0.28 | False | |||||
| IndexOfString - Duration of single invocation | 26.79 ns | 23.72 ns | 0.89 | 0.10 | False | |||||
| IndexOfString - Duration of single invocation | 41.14 ns | 37.83 ns | 0.92 | 0.02 | False | |||||
| IndexOfString - Duration of single invocation | 37.48 ns | 33.29 ns | 0.89 | 0.04 | False | |||||
| IndexOfString - Duration of single invocation | 30.66 ns | 18.35 ns | 0.60 | 0.02 | True | |||||
| IndexOfString - Duration of single invocation | 48.11 ns | 44.64 ns | 0.93 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'Payloads
Histogram
System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.467144309664 < 35.49049367549209.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 73.72300878275536 (T) = (0 -34.331740111577375) / Math.Sqrt((0.2641244459343681 / (299)) + (0.0166083927349269 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.08743061838012758 = (37.62096428288687 - 34.331740111577375) / 37.62096428288687 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.87106681071309 < 208.10935903589518.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 885.4311350782195 (T) = (0 -20.153759571273948) / Math.Sqrt((0.15255121986127915 / (299)) + (0.7483541159518351 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.9079457485759327 = (218.9335012723248 - 20.153759571273948) / 218.9335012723248 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.036081790924072 < 34.69392756396552.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 205.14619174496946 (T) = (0 -28.1039511988592) / Math.Sqrt((0.2001114405713706 / (299)) + (0.01590891618809219 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.23289361614631823 = (36.6363151062236 - 28.1039511988592) / 36.6363151062236 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.358982979464882 < 17.12686539144705.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 19.034080936244386 (T) = (0 -14.552364307654507) / Math.Sqrt((1.301271835603534 / (299)) + (0.4439484827675951 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.19419553731576616 = (18.05942381999076 - 14.552364307654507) / 18.05942381999076 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.7239035878413 < 25.877334979095767.
IsChangePoint: Marked as a change because one of 4/9/2022 2:03:45 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 17.459988707210996 (T) = (0 -23.97665501213095) / Math.Sqrt((0.4471211651676712 / (299)) + (0.3791046862505219 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11308482093243676 = (27.03376329328158 - 23.97665501213095) / 27.03376329328158 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAA5AAAA", value: "5", comparisonType: InvariantCulture)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.834723154703774 < 39.08707259385519.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 28.379976621285074 (T) = (0 -38.10865611549452) / Math.Sqrt((0.2146064682708601 / (299)) + (0.18505117555429126 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.07841577299177706 = (41.35124603771511 - 38.10865611549452) / 41.35124603771511 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.28648742518219 < 35.56568474629773.
IsChangePoint: Marked as a change because one of 4/9/2022 9:54:15 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 29.756660465268254 (T) = (0 -33.49778728011992) / Math.Sqrt((0.24061720381238794 / (299)) + (0.25887504308089615 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.10984161550486113 = (37.631266371903564 - 33.49778728011992) / 37.631266371903564 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.347445801347497 < 28.436444450121577.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 286.95823590711234 (T) = (0 -18.27040743641531) / Math.Sqrt((0.25207306337744945 / (299)) + (0.009397019983967211 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.37581374911374105 = (29.270762389389123 - 18.27040743641531) / 29.270762389389123 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.63501654015502 < 45.659438502770534.
IsChangePoint: Marked as a change because one of 4/9/2022 6:06:57 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 34.99580545749906 (T) = (0 -44.66082144945497) / Math.Sqrt((0.3745279339203872 / (299)) + (0.10981773898775478 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.07401110800436475 = (48.230407335885715 - 44.66082144945497) / 48.230407335885715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x86 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 285b8259af30e255980b1ebfb2ddd60ca291a83a |
| Compare | 0c64bdf3afc1fdc9166ad6c22c1990d3e0a6c509 |
| Diff | Diff |
Improvements in System.Globalization.Tests.StringSearch
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IndexOf_Word_NotFound - Duration of single invocation | 668.98 ns | 49.11 ns | 0.07 | 0.00 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'Payloads
Histogram
System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.10615855148156 < 636.6422063735923.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 1803.4316183237086 (T) = (0 -49.12551182765844) / Math.Sqrt((31.785052613464053 / (299)) + (0.15713288333333225 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.9261730036809689 = (665.4139309063953 - 49.12551182765844) / 665.4139309063953 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository








