Skip to content

[Perf] Windows/arm64: 5 Regressions on 10/25/2022 10:16:52 PM #9468

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline afdd009ff95ea3da2e5b34f09f4a00c548f9be33
Compare a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Diff Diff

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Min - Duration of single invocation 3.20 μs 3.91 μs 1.22 0.10 False
Max - Duration of single invocation 3.20 μs 3.92 μs 1.22 0.10 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.MathBenchmarks.Double.Min


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.9112465633591604 > 3.3613824468818176.
IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -805.9513298093725 (T) = (0 -3912.044641391327) / Math.Sqrt((0.7281573251552954 / (11)) + (24.892235018989997 / (35))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (35) - 2, .025) and -0.22196646265909933 = (3201.4337225576523 - 3912.044641391327) / 3201.4337225576523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.Max

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.917612632324747 > 3.3613880315061473.
IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/4/2022 3:13:24 PM, 10/25/2022 5:34:05 PM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -742.5063928856842 (T) = (0 -3911.2508864019233) / Math.Sqrt((1.1005957247345743 / (11)) + (28.49591242747083 / (35))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (35) - 2, .025) and -0.22176829960288164 = (3201.3032975836904 - 3911.2508864019233) / 3201.3032975836904 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline afdd009ff95ea3da2e5b34f09f4a00c548f9be33
Compare a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Diff Diff

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Tanh - Duration of single invocation 42.07 μs 44.85 μs 1.07 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.MathBenchmarks.Single.Tanh


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 44.84646642648288 > 44.167029117555835.
IsChangePoint: Marked as a change because one of 9/21/2022 8:02:52 PM, 10/25/2022 5:34:05 PM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -25.06730764524348 (T) = (0 -47809.88944999557) / Math.Sqrt((2340.994184910757 / (11)) + (1825380.9932489477 / (35))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (35) - 2, .025) and -0.13634070651890942 = (42073.551687202525 - 47809.88944999557) / 42073.551687202525 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline afdd009ff95ea3da2e5b34f09f4a00c548f9be33
Compare a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Diff Diff

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetByteCount - Duration of single invocation 9.36 μs 10.10 μs 1.08 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishAllAscii)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 10.101537966537968 > 9.82649029280747.
IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -221.69186776834746 (T) = (0 -10090.414050652405) / Math.Sqrt((94.31822123232439 / (11)) + (82.3358556277748 / (34))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (34) - 2, .025) and -0.07857980552260792 = (9355.278115710003 - 10090.414050652405) / 9355.278115710003 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline afdd009ff95ea3da2e5b34f09f4a00c548f9be33
Compare a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf16 - Duration of single invocation 14.73 ms 15.77 ms 1.07 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: OneEscaped)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 15.77137 > 15.531980387443438.
IsChangePoint: Marked as a change because one of 9/29/2022 1:36:12 PM, 10/25/2022 5:34:05 PM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -26.712326533095318 (T) = (0 -15656612.869505497) / Math.Sqrt((9050694372.389854 / (11)) + (4438123363.225064 / (35))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (35) - 2, .025) and -0.055493002493681005 = (14833459.655834364 - 15656612.869505497) / 14833459.655834364 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions