Skip to content

[Perf] Changes at 3/17/2022 11:24:40 PM #4226

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SquareRootBenchmark - Duration of single invocation 17.58 ns 16.55 ns 0.94 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int32>.SquareRootBenchmark


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.554757138428847 < 16.706015610659417.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 21.019370594404055 (T) = (0 -15.95763017083663) / Math.Sqrt((0.1616294064999156 / (170)) + (0.09044411330990544 / (18))) is greater than 1.9728001139952875 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (170) + (18) - 2, .975) and 0.09241130872914859 = (17.582447119842307 - 15.95763017083663) / 17.582447119842307 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 6.25 μs 5.55 μs 0.89 0.02 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 12.33 μs 11.54 μs 0.94 0.00 False
ReadReturnBytes - Duration of single invocation 11.12 μs 9.86 μs 0.89 0.02 False
ReadSpanEmptyLoop - Duration of single invocation 6.03 μs 5.37 μs 0.89 0.01 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 126.60 ns 117.53 ns 0.93 0.01 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 17.11 μs 15.74 μs 0.92 0.01 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Reader*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json4KB)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.551768018018017 < 5.926229735235667.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 2/19/2022 11:51:48 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 56.93319914290109 (T) = (0 -5539.6876120516245) / Math.Sqrt((21665.071600079904 / (299)) + (936.0952907696674 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.10286327544061188 = (6174.853241876079 - 5539.6876120516245) / 6174.853241876079 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.540485985916575 < 11.752832052688978.
IsChangePoint: Marked as a change because one of 2/3/2022 10:14:29 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 22.06643742166797 (T) = (0 -11794.769272487909) / Math.Sqrt((11850.154431635086 / (299)) + (15563.677755922072 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.053262802094854296 = (12458.335109876643 - 11794.769272487909) / 12458.335109876643 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.856135401548137 < 10.528504413100002.
IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 23.622387278306906 (T) = (0 -10122.609524586826) / Math.Sqrt((37792.54416648731 / (299)) + (28527.529772447233 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.0880372194090987 = (11099.805540340074 - 10122.609524586826) / 11099.805540340074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.365364780415091 < 5.714108779182342.
IsChangePoint: Marked as a change because one of 3/3/2022 7:35:51 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 63.224934496382026 (T) = (0 -5363.165006203291) / Math.Sqrt((18880.351601861545 / (299)) + (1842.6106135893697 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.13169113185874115 = (6176.563666433494 - 5363.165006203291) / 6176.563666433494 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 117.5315937175253 < 120.330655493205.
IsChangePoint: Marked as a change because one of 2/3/2022 10:14:29 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 45.83557614087597 (T) = (0 -118.65781720999588) / Math.Sqrt((8.29912501923317 / (299)) + (0.4490544931296403 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.08145668116972103 = (129.18042598263187 - 118.65781720999588) / 129.18042598263187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.744445563458594 < 16.134442639520334.
IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 26.74152403564988 (T) = (0 -15787.851295405999) / Math.Sqrt((104430.83001145754 / (299)) + (25947.421965827503 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.06688384311785231 = (16919.49193995148 - 15787.851295405999) / 16919.49193995148 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Slice_Repeat_StartPosition_And_EndPosition - Duration of single invocation 59.72 ns 54.59 ns 0.91 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySequence*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Multiple)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.587170621264356 < 56.17968383300961.
IsChangePoint: Marked as a change because one of 2/7/2022 7:01:56 PM, 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 27.453783934676984 (T) = (0 -55.223097692235925) / Math.Sqrt((6.461339904115026 / (299)) + (0.4775503432056187 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.09834998958440933 = (61.24671109001858 - 55.223097692235925) / 61.24671109001858 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseThenWrite - Duration of single invocation 6.70 μs 6.21 μs 0.93 0.00 False
ParseThenWrite - Duration of single invocation 33.85 μs 31.10 μs 0.92 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.208050652404985 < 6.353410345591179.
IsChangePoint: Marked as a change because one of 1/28/2022 6:30:08 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 25.091281218424424 (T) = (0 -6236.649343328205) / Math.Sqrt((12340.67198074131 / (299)) + (2556.5404651410545 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.0516561843335292 = (6576.358953683115 - 6236.649343328205) / 6576.358953683115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.103239390193654 < 32.04225141257814.
IsChangePoint: Marked as a change because one of 2/3/2022 10:14:29 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 18.20204016033657 (T) = (0 -31710.49151842977) / Math.Sqrt((135094.98731410247 / (299)) + (179904.5566683824 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.05541700412816427 = (33570.88964867663 - 31710.49151842977) / 33570.88964867663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractJsonSerializer_ - Duration of single invocation 3.44 ms 3.04 ms 0.88 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.DataContractJsonSerializer_


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.0426386991869925 < 3.1770781066924068.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 21.79312978274459 (T) = (0 -3087309.767880334) / Math.Sqrt((2392821900.001763 / (299)) + (2144507618.1805644 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.07372644399572159 = (3333043.1899602604 - 3087309.767880334) / 3333043.1899602604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScalarValueTypeObj - Duration of single invocation 467.74 μs 374.20 μs 0.80 0.00 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 374.1961309523809 < 444.35514049369743.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 7055.713620995293 (T) = (0 -374207.33816585754) / Math.Sqrt((14063.699920641104 / (299)) + (2315.6639870096546 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.1999463925664517 = (467727.83059657516 - 374207.33816585754) / 467727.83059657516 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MatchesWords - Duration of single invocation 77.93 μs 63.27 μs 0.81 0.01 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 63.267496190959896 < 73.07768309590115.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 43.85161211377051 (T) = (0 -65628.87698921865) / Math.Sqrt((1799365.329751168 / (299)) + (1328606.2871632732 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.15880639854504333 = (78018.75439340569 - 65628.87698921865) / 78018.75439340569 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Xml.Linq.Perf_XElement

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateElementWithNamespace - Duration of single invocation 89.74 ns 79.31 ns 0.88 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Linq.Perf_XElement*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XElement.CreateElementWithNamespace


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.30733506262119 < 84.35948757538644.
IsChangePoint: Marked as a change because one of 2/5/2022 8:17:17 PM, 3/4/2022 6:17:23 AM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 13.152715257288577 (T) = (0 -77.77341033416123) / Math.Sqrt((5.924392758616491 / (299)) + (2.9470270449735607 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.06755684204873998 = (83.40820528410863 - 77.77341033416123) / 83.40820528410863 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 238.54 ns 218.05 ns 0.91 0.00 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 218.04983680507775 < 226.61565306432746.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 245.63794734034894 (T) = (0 -217.7803977319733) / Math.Sqrt((0.23660074301290349 / (299)) + (0.11600413719322263 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.08754601049081093 = (238.6754841733092 - 217.7803977319733) / 238.6754841733092 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DotBenchmark - Duration of single invocation 149.38 ns 106.38 ns 0.71 0.07 False
GetHashCodeBenchmark - Duration of single invocation 124.40 ns 109.43 ns 0.88 0.07 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<SByte>.DotBenchmark


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 106.37980177382353 < 141.51654080503062.
IsChangePoint: Marked as a change because one of 2/4/2022 1:40:45 AM, 2/23/2022 2:01:04 AM, 3/4/2022 3:52:45 PM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 22.449554431778033 (T) = (0 -105.53708400447884) / Math.Sqrt((220.5254379452198 / (254)) + (0.17637410414690632 / (18))) is greater than 1.968789021834149 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (254) + (18) - 2, .975) and 0.16619456837843438 = (126.57279504552129 - 105.53708400447884) / 126.57279504552129 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GetHashCodeBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 109.4332133567641 < 118.19747423091593.
IsChangePoint: Marked as a change because one of 2/4/2022 6:00:25 PM, 2/7/2022 2:49:00 PM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 7.587512765695804 (T) = (0 -110.41316530548266) / Math.Sqrt((77.58722499079457 / (254)) + (5.549829269453942 / (18))) is greater than 1.968789021834149 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (254) + (18) - 2, .975) and 0.05108731643182754 = (116.3575608351011 - 110.41316530548266) / 116.3575608351011 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 1.63 μs 1.54 μs 0.94 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400B)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5352765213121795 < 1.549574396821219.
IsChangePoint: Marked as a change because one of 1/28/2022 6:30:08 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 29.87658412438519 (T) = (0 -1534.7688970416343) / Math.Sqrt((563.0582186158472 / (299)) + (143.7365421508863 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.05762829763909287 = (1628.6237088789965 - 1534.7688970416343) / 1628.6237088789965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions