-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | afdd009ff95ea3da2e5b34f09f4a00c548f9be33 |
| Compare | a8ea8e67dc20689aa2e4c738a2f84ea3483ec872 |
| Diff | Diff |
Regressions in System.MathBenchmarks.Double
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Min - Duration of single invocation | 3.20 μs | 3.91 μs | 1.22 | 0.10 | False | |||||
| Max - Duration of single invocation | 3.20 μs | 3.92 μs | 1.22 | 0.10 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.MathBenchmarks.Double.Min
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.9112465633591604 > 3.3613824468818176.
IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -805.9513298093725 (T) = (0 -3912.044641391327) / Math.Sqrt((0.7281573251552954 / (11)) + (24.892235018989997 / (35))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (35) - 2, .025) and -0.22196646265909933 = (3201.4337225576523 - 3912.044641391327) / 3201.4337225576523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.MathBenchmarks.Double.Max
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.917612632324747 > 3.3613880315061473.
IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/4/2022 3:13:24 PM, 10/25/2022 5:34:05 PM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -742.5063928856842 (T) = (0 -3911.2508864019233) / Math.Sqrt((1.1005957247345743 / (11)) + (28.49591242747083 / (35))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (35) - 2, .025) and -0.22176829960288164 = (3201.3032975836904 - 3911.2508864019233) / 3201.3032975836904 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | afdd009ff95ea3da2e5b34f09f4a00c548f9be33 |
| Compare | a8ea8e67dc20689aa2e4c738a2f84ea3483ec872 |
| Diff | Diff |
Regressions in System.MathBenchmarks.Single
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Tanh - Duration of single invocation | 42.07 μs | 44.85 μs | 1.07 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.MathBenchmarks.Single.Tanh
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 44.84646642648288 > 44.167029117555835.
IsChangePoint: Marked as a change because one of 9/21/2022 8:02:52 PM, 10/25/2022 5:34:05 PM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -25.06730764524348 (T) = (0 -47809.88944999557) / Math.Sqrt((2340.994184910757 / (11)) + (1825380.9932489477 / (35))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (35) - 2, .025) and -0.13634070651890942 = (42073.551687202525 - 47809.88944999557) / 42073.551687202525 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | afdd009ff95ea3da2e5b34f09f4a00c548f9be33 |
| Compare | a8ea8e67dc20689aa2e4c738a2f84ea3483ec872 |
| Diff | Diff |
Regressions in System.Text.Perf_Utf8Encoding
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| GetByteCount - Duration of single invocation | 9.36 μs | 10.10 μs | 1.08 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishAllAscii)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 10.101537966537968 > 9.82649029280747.
IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -221.69186776834746 (T) = (0 -10090.414050652405) / Math.Sqrt((94.31822123232439 / (11)) + (82.3358556277748 / (34))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (34) - 2, .025) and -0.07857980552260792 = (9355.278115710003 - 10090.414050652405) / 9355.278115710003 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | afdd009ff95ea3da2e5b34f09f4a00c548f9be33 |
| Compare | a8ea8e67dc20689aa2e4c738a2f84ea3483ec872 |
| Diff | Diff |
Regressions in System.Text.Json.Tests.Perf_Strings
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| WriteStringsUtf16 - Duration of single invocation | 14.73 ms | 15.77 ms | 1.07 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Strings*'Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: OneEscaped)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 15.77137 > 15.531980387443438.
IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/25/2022 5:34:05 PM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -26.712326533095318 (T) = (0 -15656612.869505497) / Math.Sqrt((9050694372.389854 / (11)) + (4438123363.225064 / (35))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (35) - 2, .025) and -0.055493002493681005 = (14833459.655834364 - 15656612.869505497) / 14833459.655834364 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository




