-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | afdd009ff95ea3da2e5b34f09f4a00c548f9be33 |
| Compare | a8ea8e67dc20689aa2e4c738a2f84ea3483ec872 |
| Diff | Diff |
Improvements in System.Buffers.Text.Tests.Utf8ParserTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| TryParseDouble - Duration of single invocation | 98.89 ns | 87.23 ns | 0.88 | 0.03 | False | |||||
| TryParseSingle - Duration of single invocation | 85.61 ns | 73.76 ns | 0.86 | 0.01 | False | |||||
| TryParseSingle - Duration of single invocation | 85.65 ns | 73.68 ns | 0.86 | 0.01 | False | |||||
| TryParseDouble - Duration of single invocation | 98.20 ns | 85.76 ns | 0.87 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'Related Issues
Regressions
- [Perf] Linux/arm64: 8 Regressions on 10/25/2022 10:16:52 PM
- [Perf] Windows/arm64: 11 Regressions on 10/25/2022 10:16:52 PM
- [Perf] Windows/arm64: 1 Regression on 10/25/2022 10:16:52 PM
Improvements
Payloads
Histogram
Edge Detector Info
System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: -1.7976931348623157e+308)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 87.22784024823544 < 93.97657617132974.
IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsImprovementStdDev: Marked as improvement because 93.46525353579817 (T) = (0 -86.93675526063932) / Math.Sqrt((0.4076149145336057 / (299)) + (0.08314312673675607 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1260319516018504 = (99.47360824000506 - 86.93675526063932) / 99.47360824000506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 3.4028235E+38)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.75650173563825 < 81.62062393659788.
IsChangePoint: Marked as a change because one of 8/15/2022 12:02:22 PM, 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsImprovementStdDev: Marked as improvement because 116.96708073487362 (T) = (0 -73.94160667802687) / Math.Sqrt((0.2224896896256413 / (299)) + (0.05304880231577487 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14424371067294373 = (86.4049818858738 - 73.94160667802687) / 86.4049818858738 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.67787041256258 < 81.36282500252932.
IsChangePoint: Marked as a change because one of 8/15/2022 12:02:22 PM, 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsImprovementStdDev: Marked as improvement because 118.92021196078389 (T) = (0 -73.7616596758135) / Math.Sqrt((0.33237235596759246 / (299)) + (0.050763466913849045 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14610977393565888 = (86.38307059185793 - 73.7616596758135) / 86.38307059185793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 1.7976931348623157e+308)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.76064890661729 < 93.50608766111907.
IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 8/15/2022 12:02:22 PM, 10/25/2022 5:34:05 PM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsImprovementStdDev: Marked as improvement because 99.90217883705706 (T) = (0 -86.18319378546778) / Math.Sqrt((1.0421148503120492 / (299)) + (0.06043039125479198 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1263699822554434 = (98.64953359543004 - 86.18319378546778) / 98.64953359543004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository



