Closed
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | cd9d54165b0de56d3974f9969d3e70412c871c4b |
| Compare | 5d54b08d5fc40d0b1c156f430a487a94c1e34f79 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Int32
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| ParseHex - Duration of single invocation | 19.41 ns | 16.43 ns | 0.85 | 0.07 | False | |||
| ParseSpan - Duration of single invocation | 9.67 ns | 7.10 ns | 0.73 | 0.34 | False | |||
| ParseHex - Duration of single invocation | 13.51 ns | 11.75 ns | 0.87 | 0.10 | False | |||
| ToString - Duration of single invocation | 20.81 ns | 17.84 ns | 0.86 | 0.13 | False | |||
| TryParse - Duration of single invocation | 10.72 ns | 6.89 ns | 0.64 | 0.31 | False | |||
| Parse - Duration of single invocation | 10.28 ns | 7.28 ns | 0.71 | 0.27 | False | |||
| Parse - Duration of single invocation | 13.45 ns | 10.65 ns | 0.79 | 0.12 | False | |||
| TryParse - Duration of single invocation | 12.99 ns | 10.68 ns | 0.82 | 0.12 | False | |||
| ParseHex - Duration of single invocation | 9.48 ns | 7.21 ns | 0.76 | 0.42 | False | |||
| TryParseSpan - Duration of single invocation | 9.45 ns | 7.09 ns | 0.75 | 0.26 | False | |||
| ParseSpan - Duration of single invocation | 13.59 ns | 10.99 ns | 0.81 | 0.10 | False | |||
| ParseHex - Duration of single invocation | 18.60 ns | 16.69 ns | 0.90 | 0.06 | False | |||
| TryParseSpan - Duration of single invocation | 12.97 ns | 11.19 ns | 0.86 | 0.15 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'Payloads
System.Tests.Perf_Int32.ParseHex(value: "80000000")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.43087879718381 < 17.869810625296186.
IsChangePoint: Marked as a change because one of 4/21/2023 8:59:32 PM, 5/19/2023 10:30:38 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 10.892491902825501 (T) = (0 -16.428016124906467) / Math.Sqrt((1.2253215612022847 / (299)) + (0.10047838189559993 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0672580231852976 = (17.612605129028132 - 16.428016124906467) / 17.612605129028132 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.ParseSpan(value: "4")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.102458808508215 < 8.875771045462496.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/4/2023 10:13:35 PM, 5/19/2023 10:30:38 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 39.74091776435523 (T) = (0 -7.084208878712596) / Math.Sqrt((0.5578688567768099 / (299)) + (0.04701104692612442 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2934621697551492 = (10.026651900942888 - 7.084208878712596) / 10.026651900942888 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.ParseHex(value: "3039")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.748038843389581 < 12.79094217734065.
IsChangePoint: Marked as a change because one of 5/15/2023 6:46:12 PM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 18.823832190339598 (T) = (0 -11.286068828847476) / Math.Sqrt((0.15872620882717206 / (299)) + (0.07882559330928413 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11928533052059484 = (12.81467110740727 - 11.286068828847476) / 12.81467110740727 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.ToString(value: -2147483648)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.835204623584175 < 18.683001576136334.
IsChangePoint: Marked as a change because one of 5/19/2023 12:51:11 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 35.67067487216894 (T) = (0 -18.09800892213429) / Math.Sqrt((1.3114064351151213 / (299)) + (0.04249087431683207 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1470781854051046 = (21.21883695837952 - 18.09800892213429) / 21.21883695837952 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.TryParse(value: "4")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.890637232766367 < 9.270993134231823.
IsChangePoint: Marked as a change because one of 4/28/2023 10:42:00 AM, 5/19/2023 10:30:38 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 27.502698542739445 (T) = (0 -7.121580198492501) / Math.Sqrt((0.646330616599192 / (299)) + (0.10155846309223526 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2783339587648195 = (9.868249012110134 - 7.121580198492501) / 9.868249012110134 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.Parse(value: "4")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.278565946885125 < 8.918785359528602.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 31.527635042993097 (T) = (0 -7.228019428473042) / Math.Sqrt((0.7953680960233378 / (299)) + (0.0680544021240841 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2793153216067404 = (10.029378513481998 - 7.228019428473042) / 10.029378513481998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.Parse(value: "12345")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.648086561950164 < 12.493985416633405.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 12:51:11 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 17.87422998896138 (T) = (0 -10.824857329874545) / Math.Sqrt((0.4759209480685141 / (299)) + (0.19849815648566585 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17655459714437027 = (13.14581062974543 - 10.824857329874545) / 13.14581062974543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.TryParse(value: "12345")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.681938070186158 < 11.973767007152542.
IsChangePoint: Marked as a change because one of 5/2/2023 10:35:24 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 26.465830813162388 (T) = (0 -10.199369789912868) / Math.Sqrt((0.5506613641357704 / (299)) + (0.1297419176221494 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22004993436473388 = (13.076952281048305 - 10.199369789912868) / 13.076952281048305 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.ParseHex(value: "4")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.213935885642106 < 9.084499797125945.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/18/2023 7:10:13 PM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 32.44941022949678 (T) = (0 -6.817394862566341) / Math.Sqrt((0.3958095676398056 / (299)) + (0.07709850969153233 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2884617951978205 = (9.581207047711093 - 6.817394862566341) / 9.581207047711093 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.TryParseSpan(value: "4")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.090217298618347 < 9.216688572400889.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/2/2023 9:04:55 PM, 5/19/2023 10:30:38 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 26.519305362855622 (T) = (0 -7.213816048259828) / Math.Sqrt((0.6032463185911902 / (299)) + (0.13044350533959942 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2875318726702686 = (10.125107035028476 - 7.213816048259828) / 10.125107035028476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.ParseSpan(value: "12345")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.987480815593935 < 12.089456792907983.
IsChangePoint: Marked as a change because one of 5/2/2023 9:04:55 PM, 5/19/2023 3:32:16 PM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 55.953939900289654 (T) = (0 -10.768474486484793) / Math.Sqrt((0.4950314405359531 / (299)) + (0.004674244856272068 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.189138585333861 = (13.280289691572714 - 10.768474486484793) / 13.280289691572714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.ParseHex(value: "7FFFFFFF")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.686941379767475 < 17.73792673131191.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 10.61917850337905 (T) = (0 -16.346807575367254) / Math.Sqrt((1.2671610625922691 / (299)) + (0.13117747964908696 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07215013347924694 = (17.61794463221128 - 16.346807575367254) / 17.61794463221128 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.TryParseSpan(value: "12345")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.190296759897086 < 11.726247819720102.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/19/2023 4:22:12 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 17.179526806325146 (T) = (0 -10.305329370080349) / Math.Sqrt((0.6491686312875796 / (299)) + (0.35726575680800643 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2230401505435326 = (13.263657545868785 - 10.305329370080349) / 13.263657545868785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | cd9d54165b0de56d3974f9969d3e70412c871c4b |
| Compare | 5d54b08d5fc40d0b1c156f430a487a94c1e34f79 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Byte
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| Parse - Duration of single invocation | 9.51 ns | 7.94 ns | 0.83 | 0.29 | False | |||
| TryParse - Duration of single invocation | 11.59 ns | 9.33 ns | 0.81 | 0.16 | False | |||
| TryParse - Duration of single invocation | 9.16 ns | 7.12 ns | 0.78 | 0.33 | False | |||
| Parse - Duration of single invocation | 11.53 ns | 9.77 ns | 0.85 | 0.13 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'Payloads
System.Tests.Perf_Byte.Parse(value: "0")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.935446427929865 < 8.541822379718385.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 34.78374936217735 (T) = (0 -7.220134267777635) / Math.Sqrt((0.3634133547384362 / (299)) + (0.055139991500967496 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26247303351103596 = (9.789654610392708 - 7.220134267777635) / 9.789654610392708 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Byte.TryParse(value: "255")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.334159910157778 < 11.038182264197877.
IsChangePoint: Marked as a change because one of 4/25/2023 8:51:41 AM, 5/19/2023 10:30:38 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 63.05781831268972 (T) = (0 -9.368375942599322) / Math.Sqrt((0.41393260185626735 / (299)) + (0.004967295100264391 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2205149201579262 = (12.018672563299589 - 9.368375942599322) / 12.018672563299589 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Byte.TryParse(value: "0")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.1204641323384665 < 9.012619930041962.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 27.854950295770703 (T) = (0 -7.0521204994447695) / Math.Sqrt((0.4088801087264405 / (299)) + (0.09263278053774522 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26686854721757325 = (9.619176032729351 - 7.0521204994447695) / 9.619176032729351 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Byte.Parse(value: "255")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.774554507122614 < 10.888307395982231.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 39.89050888824755 (T) = (0 -9.841559360303124) / Math.Sqrt((0.25530394872364615 / (299)) + (0.0313954117116353 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18814207044286482 = (12.12226795108308 - 9.841559360303124) / 12.12226795108308 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | cd9d54165b0de56d3974f9969d3e70412c871c4b |
| Compare | 5d54b08d5fc40d0b1c156f430a487a94c1e34f79 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.CtorDefaultSizeNonGeneric
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| SortedList - Duration of single invocation | 25.16 ns | 19.70 ns | 0.78 | 0.31 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSizeNonGeneric*'Payloads
System.Collections.CtorDefaultSizeNonGeneric.SortedList
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.698462724135982 < 22.37449712734328.
IsChangePoint: Marked as a change because one of 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 12.306062468446784 (T) = (0 -20.587279843087206) / Math.Sqrt((1.0140324470913296 / (299)) + (0.7024646311845714 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1252966811687009 = (23.536300137278662 - 20.587279843087206) / 23.536300137278662 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | cd9d54165b0de56d3974f9969d3e70412c871c4b |
| Compare | 5d54b08d5fc40d0b1c156f430a487a94c1e34f79 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_UInt32
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| TryParse - Duration of single invocation | 10.17 ns | 6.82 ns | 0.67 | 0.28 | False | |||
| ParseSpan - Duration of single invocation | 9.71 ns | 7.20 ns | 0.74 | 0.22 | False | |||
| ParseSpan - Duration of single invocation | 14.26 ns | 11.23 ns | 0.79 | 0.13 | False | |||
| Parse - Duration of single invocation | 9.52 ns | 7.16 ns | 0.75 | 0.27 | False | |||
| TryParse - Duration of single invocation | 14.03 ns | 11.60 ns | 0.83 | 0.10 | False | |||
| Parse - Duration of single invocation | 13.81 ns | 11.29 ns | 0.82 | 0.13 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'Payloads
System.Tests.Perf_UInt32.TryParse(value: "0")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.815687543535362 < 9.24659283639106.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 26.606226557152432 (T) = (0 -7.318587807112265) / Math.Sqrt((0.4925343633359694 / (299)) + (0.08404869952004651 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2466716252984667 = (9.715003513589767 - 7.318587807112265) / 9.715003513589767 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_UInt32.ParseSpan(value: "0")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.198896664304782 < 8.975879135133791.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 56.44453280185159 (T) = (0 -7.151168272973289) / Math.Sqrt((0.32442827169036703 / (299)) + (0.014533581395510371 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2703234512918797 = (9.800463350006668 - 7.151168272973289) / 9.800463350006668 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_UInt32.ParseSpan(value: "12345")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.233651922685524 < 12.649920471122632.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 43.54298708270854 (T) = (0 -11.320579617005212) / Math.Sqrt((0.3638605755780663 / (299)) + (0.030745178094824202 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18712520440483116 = (13.926596910556853 - 11.320579617005212) / 13.926596910556853 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_UInt32.Parse(value: "0")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.164513814826033 < 8.698096311835371.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 3:32:16 PM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 38.75141670413703 (T) = (0 -7.236095358109884) / Math.Sqrt((0.41233124701156654 / (299)) + (0.03595615503515023 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25638300682953435 = (9.730944053952102 - 7.236095358109884) / 9.730944053952102 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_UInt32.TryParse(value: "12345")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.596591956408373 < 12.813187680895432.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 47.80137873783611 (T) = (0 -11.582404451422267) / Math.Sqrt((0.4077120824650437 / (299)) + (0.006911175388418087 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15230391060020942 = (13.663392572240323 - 11.582404451422267) / 13.663392572240323 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_UInt32.Parse(value: "12345")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.29007798207396 < 12.792329295438464.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/2/2023 3:18:41 PM, 7/6/2023 1:53:22 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM.
IsImprovementStdDev: Marked as improvement because 18.886215104373377 (T) = (0 -11.479229339617412) / Math.Sqrt((0.4974083994666487 / (299)) + (0.1586094428306776 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16228471163721148 = (13.703019986721447 - 11.479229339617412) / 13.703019986721447 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment