Closed
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 4101144c8dde177addfb93ac46425fd1a8604f7a |
| Compare | af8172dc268dcaae71bfda421e88c98386c527c9 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Formats.Cbor.Tests.Perf_CborReader
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 57.25 ns | 50.63 ns | 0.88 | 0.03 | True | ||||
| 48.55 ns | 42.75 ns | 0.88 | 0.01 | True | ||||
| 46.72 ns | 39.65 ns | 0.85 | 0.02 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Formats.Cbor.Tests.Perf_CborReader*'Payloads
System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Text String, Strict))
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 50.62732310951853 < 54.53821031344897.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 9/24/2023 8:25:02 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM.
IsImprovementStdDev: Marked as improvement because 42.04351462666219 (T) = (0 -50.94211219871193) / Math.Sqrt((5.682800532702182 / (299)) + (0.1832252500493966 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11755468954525884 = (57.72835052232356 - 50.94211219871193) / 57.72835052232356 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Integer, Strict))
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 42.75389193209272 < 46.3174396559218.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 9/24/2023 8:25:02 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM.
IsImprovementStdDev: Marked as improvement because 87.87437883149575 (T) = (0 -42.723773770180635) / Math.Sqrt((0.8886655990141629 / (299)) + (0.058595461555697254 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.12943977885550845 = (49.076184200115826 - 42.723773770180635) / 49.076184200115826 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Byte String, Strict))
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 39.6473201129978 < 44.39636092118672.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 9/24/2023 8:25:02 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM.
IsImprovementStdDev: Marked as improvement because 48.91876559258193 (T) = (0 -40.19079499089878) / Math.Sqrt((1.5346071638422114 / (299)) + (0.37125686672631375 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1449935486881836 = (47.006423085153315 - 40.19079499089878) / 47.006423085153315 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment