-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Description
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 7b2dc26555db2ef75c4ef822613a63b8c1d25e85 |
Compare | 71738239969b184b66e72a927a527690a761ecbf |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Text.Json.Tests.Perf_Basic
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
WriteBasicUtf8 - Duration of single invocation | 1.87 ms | 2.34 ms | 1.25 | 0.02 | False | |||||
WriteBasicUtf16 - Duration of single invocation | 1.94 ms | 2.39 ms | 1.23 | 0.01 | False | |||||
WriteBasicUtf8 - Duration of single invocation | 520.95 ns | 600.44 ns | 1.15 | 0.01 | False | |||||
WriteBasicUtf16 - Duration of single invocation | 1.93 ms | 2.32 ms | 1.20 | 0.01 | False | |||||
WriteBasicUtf8 - Duration of single invocation | 1.83 ms | 2.39 ms | 1.30 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'
Payloads
Histogram
System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 100000)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3408806339031343 > 1.92154505949183.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -32.125924874135094 (T) = (0 -2311425.3056228026) / Math.Sqrt((916492552.4091985 / (38)) + (1219546242.0427265 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2651487721284224 = (1826998.8135341406 - 2311425.3056228026) / 1826998.8135341406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 100000)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3893149670329668 > 1.997571935011744.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -30.551419155101406 (T) = (0 -2415472.5066380375) / Math.Sqrt((2173440929.3461156 / (38)) + (1427914196.8943276 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.27764027109606826 = (1890573.23981017 - 2415472.5066380375) / 1890573.23981017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 10)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 600.4378005852681 > 550.2634042503744.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -13.555942787766599 (T) = (0 -603.4032247875114) / Math.Sqrt((729.6826798823377 / (38)) + (20.25989565113668 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.11950892195161956 = (538.9892058525174 - 603.4032247875114) / 538.9892058525174 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 100000)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3185141816239314 > 2.031272294717874.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -29.273890304780583 (T) = (0 -2319651.048315165) / Math.Sqrt((3274466224.0751367 / (38)) + (895885445.9605526 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2401709068259935 = (1870428.5317028742 - 2319651.048315165) / 1870428.5317028742 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 100000)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3887269498412698 > 1.9251214242859795.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -34.45603805502824 (T) = (0 -2398405.953772479) / Math.Sqrt((1965339907.7037337 / (38)) + (1110426259.2088099 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2838082011740852 = (1868196.4732574986 - 2398405.953772479) / 1868196.4732574986 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 7b2dc26555db2ef75c4ef822613a63b8c1d25e85 |
Compare | 71738239969b184b66e72a927a527690a761ecbf |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.IO.Tests.BinaryWriterExtendedTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
WriteAsciiCharArray - Duration of single invocation | 363.88 ns | 466.47 ns | 1.28 | 0.03 | False | |||||
WriteAsciiCharArray - Duration of single invocation | 105.67 μs | 114.49 μs | 1.08 | 0.03 | False | |||||
WriteAsciiString - Duration of single invocation | 363.79 ns | 466.84 ns | 1.28 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'
Payloads
Histogram
System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 466.4734139963222 > 383.15772596179494.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/11/2023 5:02:01 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -62.654031829913194 (T) = (0 -469.92013652447446) / Math.Sqrt((26.245500407837604 / (38)) + (12.25300946969347 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2827261375970982 = (366.3448671941504 - 469.92013652447446) / 366.3448671941504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.49047945559612 > 113.30206172402872.
IsChangePoint: Marked as a change because one of 4/29/2023 8:32:21 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -13.19588940047768 (T) = (0 -115702.83254668942) / Math.Sqrt((6515911.940266402 / (38)) + (867363.3301534103 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.06850743382361006 = (108284.5368072467 - 115702.83254668942) / 108284.5368072467 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 466.84031777918227 > 383.94951849473836.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/11/2023 5:02:01 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -54.041525304572716 (T) = (0 -468.97358170633976) / Math.Sqrt((15.48228807803085 / (38)) + (19.27088055444809 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.28077702098236795 = (366.1633321205534 - 468.97358170633976) / 366.1633321205534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 7b2dc26555db2ef75c4ef822613a63b8c1d25e85 |
Compare | 71738239969b184b66e72a927a527690a761ecbf |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SerializeToStream - Duration of single invocation | 5.35 μs | 6.79 μs | 1.27 | 0.01 | False | |||||
SerializeObjectProperty - Duration of single invocation | 6.23 μs | 7.77 μs | 1.25 | 0.01 | False | |||||
SerializeToString - Duration of single invocation | 5.95 μs | 7.61 μs | 1.28 | 0.01 | False | |||||
SerializeToUtf8Bytes - Duration of single invocation | 5.58 μs | 7.15 μs | 1.28 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>*'
Payloads
Histogram
System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToStream(Mode: SourceGen)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.785466742077336 > 5.620330566796158.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -38.14413042760439 (T) = (0 -6792.760940070647) / Math.Sqrt((36321.29587460243 / (37)) + (1437.6788092482238 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -0.24415224434961832 = (5459.750581908542 - 6792.760940070647) / 5459.750581908542 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.770807751279954 > 6.5013251281869975.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -30.080745057111375 (T) = (0 -7658.5811372684) / Math.Sqrt((45506.34930826978 / (37)) + (5038.4359784384305 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -0.21755943103309572 = (6290.108673192335 - 7658.5811372684) / 6290.108673192335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: SourceGen)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.610254915572233 > 6.255681012242412.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -32.68505020007108 (T) = (0 -7491.1123978439255) / Math.Sqrt((32617.196150649073 / (38)) + (6809.848114679876 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.24192928079182446 = (6031.834915002383 - 7491.1123978439255) / 6031.834915002383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.151032749657221 > 5.859381495053741.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -35.90800319403818 (T) = (0 -7121.446313133833) / Math.Sqrt((29809.964933499366 / (38)) + (4944.305579525699 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.25348767610064143 = (5681.305408033432 - 7121.446313133833) / 5681.305408033432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 7b2dc26555db2ef75c4ef822613a63b8c1d25e85 |
Compare | 71738239969b184b66e72a927a527690a761ecbf |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Reflection.Invoke
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
StaticMethod4_int_string_struct_class - Duration of single invocation | 49.76 ns | 52.27 ns | 1.05 | 0.09 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'
Payloads
Histogram
System.Reflection.Invoke.StaticMethod4_int_string_struct_class
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.266650549909954 > 50.215607555651395.
IsChangePoint: Marked as a change because one of 4/12/2023 9:47:42 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -17.12534861116317 (T) = (0 -52.382046518807556) / Math.Sqrt((1.4405795219331046 / (38)) + (0.04899866671257257 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.07547379950348426 = (48.70601826189616 - 52.382046518807556) / 48.70601826189616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 7b2dc26555db2ef75c4ef822613a63b8c1d25e85 |
Compare | 71738239969b184b66e72a927a527690a761ecbf |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_Enum
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
InterpolateIntoSpan_NonFlags - Duration of single invocation | 72.38 ns | 87.61 ns | 1.21 | 0.06 | False | |||||
InterpolateIntoSpan_Flags - Duration of single invocation | 75.00 ns | 83.24 ns | 1.11 | 0.04 | False | |||||
InterpolateIntoSpan_Flags - Duration of single invocation | 64.75 ns | 71.04 ns | 1.10 | 0.07 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
Payloads
Histogram
System.Tests.Perf_Enum.InterpolateIntoSpan_NonFlags(value: 42)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 87.6100956968936 > 76.50684204353036.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 5/21/2023 1:46:51 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -7.481037391105198 (T) = (0 -85.35887906495647) / Math.Sqrt((11.593569699738323 / (37)) + (7.602943293148558 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -0.12382509730703668 = (75.95388220951595 - 85.35887906495647) / 75.95388220951595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: 32)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.24290799504712 > 81.17197146223633.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -11.57857230318279 (T) = (0 -83.14123999437209) / Math.Sqrt((10.56402685678745 / (38)) + (0.907792668164346 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.10040904835775401 = (75.55484946116331 - 83.14123999437209) / 75.55484946116331 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: Red)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.03822988869678 > 68.96792473499272.
IsChangePoint: Marked as a change because one of 3/22/2023 1:04:34 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -11.429818712205602 (T) = (0 -72.97867041955617) / Math.Sqrt((3.5585694283905367 / (38)) + (2.4303351757881004 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.1243563971602053 = (64.90706203467059 - 72.97867041955617) / 64.90706203467059 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 18.04 |
Queue | TigerUbuntu |
Baseline | 7b2dc26555db2ef75c4ef822613a63b8c1d25e85 |
Compare | 71738239969b184b66e72a927a527690a761ecbf |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_Uri
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
EscapeDataString - Duration of single invocation | 6.16 μs | 12.80 μs | 2.08 | 0.05 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'
Payloads
Histogram
System.Tests.Perf_Uri.EscapeDataString(input: "{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.800517183048434 > 6.483500356773706.
IsChangePoint: Marked as a change because one of 5/10/2023 6:05:58 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -99.0120814232714 (T) = (0 -12786.393197676012) / Math.Sqrt((177525.3632517209 / (37)) + (124.92219905417701 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -1.1623430475639271 = (5913.212157562616 - 12786.393197676012) / 5913.212157562616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository