Description
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | e500dad300c739a35eb88edb3605f71eca18a4ac |
Compare | cc072997b68b2412c0f7667737b0619f4f04241d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in Benchstone.MDBenchI.MDMulMatrix
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
864.12 ms | 767.01 ms | 0.89 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDMulMatrix*'
Payloads
Benchstone.MDBenchI.MDMulMatrix.Test
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 9/28/2023 11:34:37 PM, 10/4/2023 1:30:06 AM falls between 9/25/2023 11:21:57 AM and 10/4/2023 1:30:06 AM.
IsImprovementStdDev: Marked as improvement because 59.83473080767967 (T) = (0 -765699225.4761906) / Math.Sqrt((1034816784797317.1 / (299)) + (609128529647.6758 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12853445703434066 = (878633965.1140555 - 765699225.4761906) / 878633965.1140555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | e500dad300c739a35eb88edb3605f71eca18a4ac |
Compare | cc072997b68b2412c0f7667737b0619f4f04241d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in Benchstone.MDBenchI.MDGeneralArray
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
19.83 ms | 17.08 ms | 0.86 | 0.00 | False | ||||
19.49 ms | 17.12 ms | 0.88 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDGeneralArray*'
Payloads
Benchstone.MDBenchI.MDGeneralArray.Test2
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 9/28/2023 11:34:37 PM, 10/4/2023 1:30:06 AM falls between 9/25/2023 11:21:57 AM and 10/4/2023 1:30:06 AM.
IsImprovementStdDev: Marked as improvement because 136.19311959860167 (T) = (0 -17118156.037414964) / Math.Sqrt((49808219493.3453 / (299)) + (840326518.6440237 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12228274228950332 = (19503041.425968133 - 17118156.037414964) / 19503041.425968133 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Benchstone.MDBenchI.MDGeneralArray.Test
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 9/28/2023 11:34:37 PM, 10/4/2023 1:30:06 AM falls between 9/25/2023 11:21:57 AM and 10/4/2023 1:30:06 AM.
IsImprovementStdDev: Marked as improvement because 161.3844481419925 (T) = (0 -17136012.81746032) / Math.Sqrt((44554998569.195694 / (299)) + (405716131.93686694 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12174092174057617 = (19511341.518292412 - 17136012.81746032) / 19511341.518292412 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | e500dad300c739a35eb88edb3605f71eca18a4ac |
Compare | cc072997b68b2412c0f7667737b0619f4f04241d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in Benchstone.MDBenchI.MDXposMatrix
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
44.93 μs | 41.42 μs | 0.92 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDXposMatrix*'
Payloads
Benchstone.MDBenchI.MDXposMatrix.Test
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/28/2023 11:34:37 PM, 10/4/2023 1:30:06 AM falls between 9/25/2023 11:21:57 AM and 10/4/2023 1:30:06 AM.
IsImprovementStdDev: Marked as improvement because 202.31111554292247 (T) = (0 -41446.34695795105) / Math.Sqrt((12569.892553917614 / (299)) + (1672.142223478621 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.08039085680868933 = (45069.524661444964 - 41446.34695795105) / 45069.524661444964 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | e500dad300c739a35eb88edb3605f71eca18a4ac |
Compare | cc072997b68b2412c0f7667737b0619f4f04241d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in Benchstone.MDBenchI.MDArray2
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
1.94 secs | 1.71 secs | 0.88 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDArray2*'
Payloads
Benchstone.MDBenchI.MDArray2.Test
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/3/2023 1:18:01 AM, 6/21/2023 4:19:00 AM, 9/28/2023 11:34:37 PM, 10/4/2023 1:30:06 AM falls between 9/25/2023 11:21:57 AM and 10/4/2023 1:30:06 AM.
IsImprovementStdDev: Marked as improvement because 149.11949297251678 (T) = (0 -1712979988.4157507) / Math.Sqrt((575164561828289.6 / (299)) + (670673094432.7651 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11047541134070307 = (1925725280.9588733 - 1712979988.4157507) / 1925725280.9588733 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository