Skip to content

[Perf] Changes at 7/5/2022 10:32:06 PM #6733

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDXposMatrix

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 34.26 μs 24.45 μs 0.71 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDXposMatrix*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDXposMatrix.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.45421042118093 < 32.55272206504033.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 583.6000010622305 (T) = (0 -24537.33125379975) / Math.Sqrt((16553.90605766711 / (299)) + (10586.087139488065 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.2866684194961015 = (34398.212450465944 - 24537.33125379975) / 34398.212450465944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in ByteMark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BenchNumericSortRectangular - Duration of single invocation 974.34 ms 780.60 ms 0.80 0.00 False
BenchNeural - Duration of single invocation 586.96 ms 501.94 ms 0.86 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

Histogram

ByteMark.BenchNumericSortRectangular


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 780.6019800769232 < 925.463784275.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 320.40610047927186 (T) = (0 -779774264.7631271) / Math.Sqrt((23548174345917.758 / (299)) + (12616061500759.107 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1962289422396519 = (970144741.1355089 - 779774264.7631271) / 970144741.1355089 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### ByteMark.BenchNeural

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 501.935362 < 544.3153957384616.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 61.19961434157426 (T) = (0 -505393165.410965) / Math.Sqrt((40973128916618.43 / (299)) + (52109624252056.63 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.12007229979057063 = (574357603.8016279 - 505393165.410965) / 574357603.8016279 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDAddArray2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 16.42 ms 10.88 ms 0.66 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDAddArray2*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDAddArray2.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.878526305803572 < 15.606283548666667.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 349.44310911324885 (T) = (0 -10716633.942103762) / Math.Sqrt((4986767616.621469 / (299)) + (11657502308.781933 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.34891506649465576 = (16459655.861497212 - 10716633.942103762) / 16459655.861497212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchF.MDInProd

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 1.35 secs 1.18 secs 0.87 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDInProd*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDInProd.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1816533965 < 1.285773078298077.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 187.57439559610685 (T) = (0 -1187572352.3035078) / Math.Sqrt((38764396733194.16 / (299)) + (26998946916218.39 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1179359175567149 = (1346356093.5549898 - 1187572352.3035078) / 1346356093.5549898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDMidpoint

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 442.55 ms 415.21 ms 0.94 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDMidpoint*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDMidpoint.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 415.2065742857143 < 420.1461699265384.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 5/23/2022 10:48:42 PM, 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 22.84961124253839 (T) = (0 -412026184.0876931) / Math.Sqrt((386748016928479.44 / (299)) + (39085060853064.61 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.07508957407800027 = (445476851.098271 - 412026184.0876931) / 445476851.098271 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchF.MDRomber

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 150.84 ms 125.51 ms 0.83 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDRomber*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDRomber.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.50942542307693 < 143.30489618571426.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 233.23863439697644 (T) = (0 -125582429.31150663) / Math.Sqrt((367041668021.4863 / (299)) + (494114271154.9436 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.16887654140207087 = (151099608.62295836 - 125582429.31150663) / 151099608.62295836 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDGeneralArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 10.88 ms 9.30 ms 0.85 0.01 False
Test2 - Duration of single invocation 10.60 ms 9.30 ms 0.88 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDGeneralArray*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDGeneralArray.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.301361461538463 < 10.328925606160713.
IsChangePoint: Marked as a change because one of 6/14/2022 10:05:19 AM, 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 103.49999241528238 (T) = (0 -9300823.578864444) / Math.Sqrt((47229734811.12752 / (299)) + (6984232.149951507 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.12275038923620418 = (10602254.437897654 - 9300823.578864444) / 10602254.437897654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Benchstone.MDBenchI.MDGeneralArray.Test2

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.302007109375 < 10.072081192885046.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 122.41772392182695 (T) = (0 -9303092.569888964) / Math.Sqrt((32111961171.875072 / (299)) + (82308306.18607251 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1208791389102665 = (10582268.015295545 - 9303092.569888964) / 10582268.015295545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchF.MDLLoops

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 604.87 ms 572.30 ms 0.95 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDLLoops*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDLLoops.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 572.3012537857143 < 577.6544251313095.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 46.993955995024926 (T) = (0 -571582237.1537386) / Math.Sqrt((29013262355255.766 / (299)) + (24073119779601.887 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.060818507214710694 = (608596146.2662795 - 571582237.1537386) / 608596146.2662795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDLogicArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 352.47 ms 293.53 ms 0.83 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDLogicArray*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDLogicArray.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 293.5326173846154 < 334.54280510102564.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 2962.672146326839 (T) = (0 -293611714.3854914) / Math.Sqrt((26999400692.321907 / (299)) + (13754354976.873549 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1660363841934536 = (352067774.68526894 - 293611714.3854914) / 352067774.68526894 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDArray2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 1.07 secs 919.27 ms 0.86 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDArray2*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDArray2.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 919.2742426923077 < 1.0203072489772618.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 570.9822030792318 (T) = (0 -919853871.2594603) / Math.Sqrt((6737283087307.014 / (299)) + (2348044803693.1206 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1441097472534866 = (1074733434.9324467 - 919853871.2594603) / 1074733434.9324467 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchF.MDInvMt

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 4.64 ms 2.52 ms 0.54 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDInvMt*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDInvMt.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5183204791666665 < 4.402104256529017.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 5889.06679597039 (T) = (0 -2515151.8836436435) / Math.Sqrt((3075126.4715031087 / (299)) + (5478197.044016493 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.45716722720541136 = (4633382.5253314115 - 2515151.8836436435) / 4633382.5253314115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchF.MDSqMtx

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 752.22 ms 594.85 ms 0.79 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDSqMtx*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDSqMtx.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 594.8503986153846 < 714.8829040633332.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 1244.2540846235213 (T) = (0 -595300734.4144526) / Math.Sqrt((154400526873.38232 / (299)) + (709224673801.1129 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.20876012738626928 = (752364428.3091732 - 595300734.4144526) / 752364428.3091732 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions