Skip to content

[Perf] Windows/arm64: 4 Improvements on 9/9/2022 7:27:51 AM #8571

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e8b149109c8b04161b83d0e4c997b63759c6d936
Compare 071fe7ce868a63113143b949c3c4185f9e5578cf
Diff Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 141.02 ns 131.15 ns 0.93 0.01 False
GetChars - Duration of single invocation 36.81 ns 33.23 ns 0.90 0.06 False
GetString - Duration of single invocation 30.03 ns 27.50 ns 0.92 0.09 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "ascii")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 131.15346502817857 < 135.80823307955913.
IsChangePoint: Marked as a change because one of 8/5/2022 7:33:33 PM, 8/25/2022 6:55:57 AM, 9/9/2022 3:12:19 AM, 9/15/2022 1:10:12 AM falls between 9/6/2022 1:06:44 PM and 9/15/2022 1:10:12 AM.
IsImprovementStdDev: Marked as improvement because 40.668492545161385 (T) = (0 -128.6437170525642) / Math.Sqrt((9.401643272122854 / (299)) + (3.9899083692475577 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11803520683212075 = (145.8603767963301 - 128.6437170525642) / 145.8603767963301 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.22709199408358 < 35.262121894835204.
IsChangePoint: Marked as a change because one of 8/5/2022 7:33:33 PM, 9/9/2022 3:12:19 AM, 9/15/2022 1:10:12 AM falls between 9/6/2022 1:06:44 PM and 9/15/2022 1:10:12 AM.
IsImprovementStdDev: Marked as improvement because 16.014054726837138 (T) = (0 -34.001992958339144) / Math.Sqrt((0.6653790591544279 / (299)) + (0.6259015759526859 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06982894666870948 = (36.554559332463946 - 34.001992958339144) / 36.554559332463946 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.49872023572498 < 28.395346284073206.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/15/2022 1:10:12 AM falls between 9/6/2022 1:06:44 PM and 9/15/2022 1:10:12 AM.
IsImprovementStdDev: Marked as improvement because 21.590366391885 (T) = (0 -27.022869716040017) / Math.Sqrt((0.5299985950803874 / (299)) + (0.4894218252976796 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.101290104886048 = (30.068512501037556 - 27.022869716040017) / 30.068512501037556 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline e8b149109c8b04161b83d0e4c997b63759c6d936
Compare 071fe7ce868a63113143b949c3c4185f9e5578cf
Diff Diff

Improvements in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 51.96 μs 46.70 μs 0.90 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetString(Input: EnglishAllAscii)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.700133972468045 < 49.424169435738776.
IsChangePoint: Marked as a change because one of 8/5/2022 7:33:33 PM, 8/25/2022 6:55:57 AM, 9/9/2022 3:12:19 AM, 9/15/2022 1:10:12 AM falls between 9/6/2022 1:06:44 PM and 9/15/2022 1:10:12 AM.
IsImprovementStdDev: Marked as improvement because 17.31763566303028 (T) = (0 -48262.96817134697) / Math.Sqrt((1593566.3430199767 / (299)) + (764944.5811950695 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.061765912296340715 = (51440.220307355536 - 48262.96817134697) / 51440.220307355536 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions