Skip to content

[Perf] Windows/x64: 3 Regressions on 10/20/2022 5:12:08 PM #9144

Open
@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 10d342cecc5cbfc57419f8a92af4787f29d76c08
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Regressions in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedDictionary - Duration of single invocation 36.13 μs 38.38 μs 1.06 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.TryGetValueTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 38.38328022875818 > 37.98091143466438.
IsChangePoint: Marked as a change because one of 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -17.540059820998522 (T) = (0 -38313.4622386043) / Math.Sqrt((318933.3801365225 / (28)) + (2872.1994614546493 / (17))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (17) - 2, .025) and -0.05176881010446284 = (36427.646333036784 - 38313.4622386043) / 36427.646333036784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 10d342cecc5cbfc57419f8a92af4787f29d76c08
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 23.97 μs 25.89 μs 1.08 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableDictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 25.888337465564735 > 25.139038045558966.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -16.545307969557907 (T) = (0 -25871.44223232675) / Math.Sqrt((301926.18161508447 / (28)) + (13698.227430679168 / (17))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (17) - 2, .025) and -0.07393544212446665 = (24090.314200961355 - 25871.44223232675) / 24090.314200961355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 10d342cecc5cbfc57419f8a92af4787f29d76c08
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Regressions in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateActivityLinksSmall - Duration of single invocation 90.04 ns 106.38 ns 1.18 0.20 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Diagnostics.Perf_Activity*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 106.37662364090936 > 98.94466503206003.
IsChangePoint: Marked as a change because one of 9/3/2022 11:44:47 AM, 10/19/2022 7:27:15 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsRegressionStdDev: Marked as regression because -12.451776575587383 (T) = (0 -106.48961217219139) / Math.Sqrt((21.67003235356752 / (28)) + (3.607629063964601 / (17))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (17) - 2, .025) and -0.13137086420530064 = (94.12440742584616 - 106.48961217219139) / 94.12440742584616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions