[Perf] Changes at 5/23/2022 7:50:08 PM #70029
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | a0ff92c8d3cb3cb3b35357117e629cdd19f7080a |
| Compare | cd6512d071d0a32c90fa81a64abd258d707cfa5f |
| Diff | Diff |
Regressions in System.Text.Json.Reader.Tests.Perf_Base64
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation | 77.53 ns | 93.50 ns | 1.21 | 0.09 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'Payloads
Histogram
System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 100)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 93.49766283232752 > 81.41699218742718.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/22/2022 6:17:37 AM, 5/21/2022 9:00:51 AM, 5/23/2022 3:43:39 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -4.529068678788347 (T) = (0 -92.55227148494835) / Math.Sqrt((57.02585325930479 / (21)) + (9.256823177502445 / (45))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (45) - 2, .025) and -0.09127193112937368 = (84.81137363183585 - 92.55227148494835) / 84.81137363183585 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | a0ff92c8d3cb3cb3b35357117e629cdd19f7080a |
| Compare | cd6512d071d0a32c90fa81a64abd258d707cfa5f |
| Diff | Diff |
Regressions in System.Tests.Perf_Version
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| TryParse3 - Duration of single invocation | 38.32 ns | 40.55 ns | 1.06 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'Payloads
Histogram
System.Tests.Perf_Version.TryParse3
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.547358483873694 > 40.44499077653907.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/9/2022 5:23:46 PM, 5/23/2022 3:43:39 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -25.15196605288161 (T) = (0 -40.827940105493056) / Math.Sqrt((0.08083014823191696 / (20)) + (0.15635369342812014 / (45))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (45) - 2, .025) and -0.056421697438722315 = (38.64738882633682 - 40.827940105493056) / 38.64738882633682 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository