Skip to content

System.Buffers.Binary.Tests.BinaryReadAndWriteTests #83735

Closed
@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 673afec8a76cb4437eba318d7132140b7da9b111
Compare 2f8a43af7eec94a3a26ece0e5411d64577410f1a
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Formats.Cbor.Tests.Perf_CborWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteCoseKey - Duration of single invocation 630.33 ns 719.87 ns 1.14 0.03 False
WriteCoseKey - Duration of single invocation 572.83 ns 639.91 ns 1.12 0.02 False
WriteCoseKey - Duration of single invocation 582.88 ns 675.83 ns 1.16 0.03 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Formats.Cbor.Tests.Perf_CborWriter*'

Payloads

Baseline
Compare

Histogram

System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P521)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 719.8683337060743 > 661.4982808663523.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 3/17/2023 10:06:19 AM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsRegressionStdDev: Marked as regression because -11.564558545499509 (T) = (0 -723.5504004583727) / Math.Sqrt((77.13595081478127 / (45)) + (826.9238493075813 / (14))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (14) - 2, .025) and -0.1423435476868618 = (633.391243749303 - 723.5504004583727) / 633.391243749303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P256)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 639.9148703209471 > 583.3844880597635.
IsChangePoint: Marked as a change because one of 1/10/2023 11:45:30 PM, 3/17/2023 10:06:19 AM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsRegressionStdDev: Marked as regression because -37.29938971421864 (T) = (0 -638.0509799833275) / Math.Sqrt((102.62054988361416 / (45)) + (32.111171060456776 / (14))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (14) - 2, .025) and -0.14289087515525523 = (558.2781294816549 - 638.0509799833275) / 558.2781294816549 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P384)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 675.8325912216577 > 615.2575497147488.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 3/17/2023 10:06:19 AM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsRegressionStdDev: Marked as regression because -26.04047026489414 (T) = (0 -673.6124891543647) / Math.Sqrt((383.7306749649151 / (45)) + (14.65403765677355 / (14))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (14) - 2, .025) and -0.13586713548559687 = (593.0381011212083 - 673.6124891543647) / 593.0381011212083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 673afec8a76cb4437eba318d7132140b7da9b111
Compare 2f8a43af7eec94a3a26ece0e5411d64577410f1a
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Xml.Linq.Perf_XElement

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateElementsWithNamespace - Duration of single invocation 104.63 ns 114.87 ns 1.10 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElement*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XElement.CreateElementsWithNamespace


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.86529673670462 > 109.50399425578388.
IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/24/2023 10:20:22 AM, 2/20/2023 10:31:48 AM, 3/17/2023 10:06:19 AM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsRegressionStdDev: Marked as regression because -4.389396318593573 (T) = (0 -111.42071450088524) / Math.Sqrt((0.6531766938200125 / (45)) + (37.068680835806504 / (14))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (14) - 2, .025) and -0.06869402016414386 = (104.25876106593336 - 111.42071450088524) / 104.25876106593336 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 673afec8a76cb4437eba318d7132140b7da9b111
Compare 2f8a43af7eec94a3a26ece0e5411d64577410f1a
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadStructAndReverseLE - Duration of single invocation 1.30 ns 8.17 ns 6.27 0.37 False
ReadStructAndReverseBE - Duration of single invocation 13.46 ns 15.78 ns 1.17 0.63 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseLE


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.167938846535769 > 1.3975800016097022.
IsChangePoint: Marked as a change because one of 3/17/2023 10:06:19 AM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsRegressionStdDev: Marked as regression because -97.92977094233322 (T) = (0 -8.652844892749059) / Math.Sqrt((0.006749899736193148 / (45)) + (0.07569390271517575 / (14))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (14) - 2, .025) and -5.396080167201258 = (1.3528355909484007 - 8.652844892749059) / 1.3528355909484007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseBE

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.777489774253384 > 13.928980752590498.
IsChangePoint: Marked as a change because one of 3/17/2023 10:06:19 AM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsRegressionStdDev: Marked as regression because -10.040053091498793 (T) = (0 -15.331183594915322) / Math.Sqrt((1.787631447493644 / (45)) + (0.12942140707381405 / (14))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (14) - 2, .025) and -0.16947892033693288 = (13.109414225694918 - 15.331183594915322) / 13.109414225694918 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 673afec8a76cb4437eba318d7132140b7da9b111
Compare 2f8a43af7eec94a3a26ece0e5411d64577410f1a
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCustomAttributesClassMiss - Duration of single invocation 103.14 ns 110.28 ns 1.07 0.12 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Attributes.GetCustomAttributesClassMiss


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.27518213204208 > 108.52837332133602.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 3/5/2023 11:54:18 PM, 3/8/2023 9:36:36 PM, 3/17/2023 10:06:19 AM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsRegressionStdDev: Marked as regression because -6.990544502612381 (T) = (0 -110.24449758453878) / Math.Sqrt((9.034063724976647 / (45)) + (15.947561128677734 / (14))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (14) - 2, .025) and -0.07921218789802174 = (102.15275440806654 - 110.24449758453878) / 102.15275440806654 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions