Closed
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | e4df4329df8e78c21530ae1f4af32dd87e950825 |
| Compare | e985f786c548e87f299e211026cba58cba900be8 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in Benchstone.BenchI.TreeInsert
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 56.63 μs | 52.62 μs | 0.93 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.TreeInsert*'Payloads
Benchstone.BenchI.TreeInsert.Test
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 52.62369336433758 < 53.68501385065426.
IsChangePoint: Marked as a change because one of 5/21/2023 3:58:09 AM, 9/20/2023 12:12:52 PM, 9/26/2023 10:37:55 AM falls between 9/14/2023 3:34:28 AM and 9/26/2023 10:37:55 AM.
IsImprovementStdDev: Marked as improvement because 27.360746477745916 (T) = (0 -51835.15602813079) / Math.Sqrt((250720.18290102007 / (299)) + (276125.13762061385 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.08568165898749543 = (56692.678800174996 - 51835.15602813079) / 56692.678800174996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | e4df4329df8e78c21530ae1f4af32dd87e950825 |
| Compare | e985f786c548e87f299e211026cba58cba900be8 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_BigInteger
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 1.43 ms | 1.34 ms | 0.94 | 0.00 | True | ||||
| 1.50 μs | 1.41 μs | 0.94 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'Payloads
System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 65536,65536 bits)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3445871056547618 < 1.3465486246330491.
IsChangePoint: Marked as a change because one of 9/20/2023 12:12:52 PM, 9/26/2023 10:37:55 AM falls between 9/14/2023 3:34:28 AM and 9/26/2023 10:37:55 AM.
IsImprovementStdDev: Marked as improvement because 157.48418833338812 (T) = (0 -1344929.2997462607) / Math.Sqrt((9266352.365721341 / (299)) + (1731054.9035547916 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.05243492400924862 = (1419352.9645866642 - 1344929.2997462607) / 1419352.9645866642 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 1024,1024 bits)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4104632096821292 < 1.428993100793316.
IsChangePoint: Marked as a change because one of 9/20/2023 12:12:52 PM, 9/26/2023 10:37:55 AM falls between 9/14/2023 3:34:28 AM and 9/26/2023 10:37:55 AM.
IsImprovementStdDev: Marked as improvement because 309.9347053922419 (T) = (0 -1409.356702080023) / Math.Sqrt((10.18000855746157 / (299)) + (0.5497114084350789 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.06351817612523358 = (1504.9482714450346 - 1409.356702080023) / 1504.9482714450346 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository