-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in System.Text.Json.Serialization.Tests.ReadJson<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
DeserializeFromUtf8Bytes - Duration of single invocation | 93.89 ns | 85.21 ns | 0.91 | 0.05 | False | |||||
DeserializeFromUtf8Bytes - Duration of single invocation | 90.23 ns | 84.13 ns | 0.93 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<Int32>*'
Payloads
Histogram
System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: SourceGen)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.20970505457376 < 89.1088766561678.
IsChangePoint: Marked as a change because one of 12/5/2022 3:14:07 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 34.104810308942206 (T) = (0 -85.32910318933537) / Math.Sqrt((4.044500669603929 / (212)) + (0.3567130638480651 / (30))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (212) + (30) - 2, .975) and 0.06571383867593349 = (91.33080069216408 - 85.32910318933537) / 91.33080069216408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: Reflection)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 84.13326843552649 < 85.64543916496137.
IsChangePoint: Marked as a change because one of 12/5/2022 3:14:07 PM, 12/21/2022 9:23:17 AM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 23.57350995032281 (T) = (0 -84.3535140581799) / Math.Sqrt((8.868451619954685 / (212)) + (1.375205813808394 / (30))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (212) + (30) - 2, .975) and 0.07642329671517059 = (91.3335229853296 - 84.3535140581799) / 91.3335229853296 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in System.Globalization.Tests.StringSearch
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IsSuffix_SecondHalf - Duration of single invocation | 152.12 ns | 141.53 ns | 0.93 | 0.00 | False | |||||
IsPrefix_FirstHalf - Duration of single invocation | 156.29 ns | 143.85 ns | 0.92 | 0.00 | False | |||||
IsPrefix_FirstHalf - Duration of single invocation | 156.30 ns | 144.06 ns | 0.92 | 0.01 | False | |||||
LastIndexOf_Word_NotFound - Duration of single invocation | 523.03 ns | 410.27 ns | 0.78 | 0.01 | False | |||||
IsPrefix_FirstHalf - Duration of single invocation | 154.70 ns | 143.69 ns | 0.93 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
Payloads
Histogram
System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreNonSpace, False))
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 141.53478791013893 < 144.19295089514353.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 32.15266621123199 (T) = (0 -142.36948410996519) / Math.Sqrt((0.3176423292308934 / (299)) + (2.2284907189910506 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.058372400469021764 = (151.19510534831284 - 142.36948410996519) / 151.19510534831284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 143.8493887377669 < 148.5307354644846.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 66.33741469648247 (T) = (0 -143.76580606503248) / Math.Sqrt((0.8713461899315349 / (299)) + (0.8749073740171796 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07633432969091569 = (155.6470167576158 - 143.76580606503248) / 155.6470167576158 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, None, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 144.05810298256017 < 148.5101016614829.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 81.4761338565778 (T) = (0 -143.85698615137252) / Math.Sqrt((0.9374310556460869 / (299)) + (0.5306371836784859 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07555343964997945 = (155.6141721127767 - 143.85698615137252) / 155.6141721127767 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 410.2653103279839 < 496.91457913480815.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/13/2023 4:18:46 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 200.99477330730764 (T) = (0 -410.3670199347482) / Math.Sqrt((97.56841615563015 / (299)) + (0.03679627889522617 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.21905846179680535 = (525.4772602811327 - 410.3670199347482) / 525.4772602811327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 143.69166620151574 < 148.3287937290018.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 68.41536216969016 (T) = (0 -143.82687737497216) / Math.Sqrt((1.5267323860976605 / (299)) + (0.7539673830245344 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07639729723157855 = (155.723751071606 - 143.82687737497216) / 155.723751071606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in System.Hashing
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetStringHashCode - Duration of single invocation | 389.32 ns | 368.43 ns | 0.95 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Hashing*'
Payloads
Histogram
System.Hashing.GetStringHashCode(BytesCount: 1000)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 368.42988927697394 < 369.9746705242599.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 279.4438753107469 (T) = (0 -368.5042468602642) / Math.Sqrt((0.22598904725683003 / (299)) + (0.13646159826786033 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.05233938744102421 = (388.8567721150604 - 368.5042468602642) / 388.8567721150604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in PerfLabTests.GetMember
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetMethod12 - Duration of single invocation | 841.44 μs | 781.55 μs | 0.93 | 0.00 | False | |||||
GetMethod20 - Duration of single invocation | 1.41 ms | 1.29 ms | 0.91 | 0.01 | False | |||||
GetMethod10 - Duration of single invocation | 706.41 μs | 659.09 μs | 0.93 | 0.02 | False | |||||
GetMethod2 - Duration of single invocation | 136.22 μs | 125.07 μs | 0.92 | 0.04 | False | |||||
GetMethod15 - Duration of single invocation | 1.05 ms | 986.96 μs | 0.94 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.GetMember*'
Payloads
Histogram
PerfLabTests.GetMember.GetMethod12
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 781.5505158730158 < 808.0918854166667.
IsChangePoint: Marked as a change because one of 11/3/2022 9:01:15 AM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 64.88340029601899 (T) = (0 -778311.9861276456) / Math.Sqrt((77284597.6994202 / (299)) + (27669830.263572462 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.0830631375636488 = (848817.4246366628 - 778311.9861276456) / 848817.4246366628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod20
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2927220486111113 < 1.3418383374368685.
IsChangePoint: Marked as a change because one of 11/2/2022 9:17:08 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 41.850126497711486 (T) = (0 -1307810.7866427556) / Math.Sqrt((217103758.72657248 / (299)) + (192468780.8328541 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07878013704206276 = (1419651.0944124856 - 1307810.7866427556) / 1419651.0944124856 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod10
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 659.0939756944443 < 669.2397741429766.
IsChangePoint: Marked as a change because one of 11/2/2022 9:17:08 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 53.34291505934309 (T) = (0 -648783.4343451747) / Math.Sqrt((66945657.736959726 / (299)) + (29057586.97007345 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08238774486603812 = (707034.4044723542 - 648783.4343451747) / 707034.4044723542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod2
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.0664857142857 < 126.02416591000808.
IsChangePoint: Marked as a change because one of 11/2/2022 9:17:08 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 40.76294043737647 (T) = (0 -125387.62066810903) / Math.Sqrt((3919491.163388816 / (299)) + (1025331.66093151 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.06602581728469749 = (134251.69880346698 - 125387.62066810903) / 134251.69880346698 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod15
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 986.9561764705882 < 0.9976992031249999.
IsChangePoint: Marked as a change because one of 11/2/2022 9:17:08 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 54.193159931445855 (T) = (0 -974301.0763346005) / Math.Sqrt((126250352.67666104 / (299)) + (49708944.14355533 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07424958807381864 = (1052444.6587146542 - 974301.0763346005) / 1052444.6587146542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | 5385f045f9aee91c22922d21552c4317198710f1 |
Diff | Diff |
Improvements in PerfLabTests.LowLevelPerf
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ForeachOverList100Elements - Duration of single invocation | 9.02 ms | 6.27 ms | 0.69 | 0.09 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
Payloads
Histogram
PerfLabTests.LowLevelPerf.ForeachOverList100Elements
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.265312980769231 < 8.588168017113096.
IsChangePoint: Marked as a change because one of 11/4/2022 9:11:54 AM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 35.57966911402608 (T) = (0 -6357071.632187439) / Math.Sqrt((197855673427.0797 / (299)) + (160998098586.72095 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.3029189834630732 = (9119559.249754269 - 6357071.632187439) / 9119559.249754269 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository