Skip to content

[Perf] Changes at 6/23/2022 3:07:42 AM #6511

Closed

Description

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline c9007c8cd39a5b086a74dd5580b34311041a68ff
Compare b1221494186e7b8e5199d892dae76481a52f5935
Diff Diff

Improvements in System.Net.NetworkInformation.Tests.PhysicalAddressTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
PAMedium - Duration of single invocation 31.92 ns 25.58 ns 0.80 0.34 False
PALong - Duration of single invocation 513.33 ns 419.96 ns 0.82 0.24 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.NetworkInformation.Tests.PhysicalAddressTests*'

Payloads

Baseline
Compare

Histogram

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PAMedium


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.57949245959389 < 31.240040208456747.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 14.655056924276687 (T) = (0 -27.254385022303538) / Math.Sqrt((1.828745253279086 / (233)) + (2.9783686782977528 / (27))) is greater than 1.9692013862338849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (233) + (27) - 2, .975) and 0.15600052103070614 = (32.291945316822996 - 27.254385022303538) / 32.291945316822996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.NetworkInformation.Tests.PhysicalAddressTests.PALong

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 419.96493141626473 < 501.5647459559442.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 86.87746971362286 (T) = (0 -413.85674701661264) / Math.Sqrt((263.78399320546396 / (232)) + (21.44797325249935 / (28))) is greater than 1.9692013862338849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (232) + (28) - 2, .975) and 0.22455687488791326 = (533.7035478350415 - 413.85674701661264) / 533.7035478350415 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline c9007c8cd39a5b086a74dd5580b34311041a68ff
Compare b1221494186e7b8e5199d892dae76481a52f5935
Diff Diff

Improvements in System.Threading.Tests.Perf_CancellationToken

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateTokenDispose - Duration of single invocation 20.86 ns 17.16 ns 0.82 0.48 False
CancelAfter - Duration of single invocation 179.98 ns 147.47 ns 0.82 0.21 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_CancellationToken*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_CancellationToken.CreateTokenDispose


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.161924729031707 < 17.925526941118264.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 14.356744423126749 (T) = (0 -15.089309993586422) / Math.Sqrt((1.9788018829961884 / (232)) + (2.247908320568229 / (27))) is greater than 1.9692374962334795 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (232) + (27) - 2, .975) and 0.22375457711367644 = (19.438839249421456 - 15.089309993586422) / 19.438839249421456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_CancellationToken.CancelAfter

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 147.46864472549697 < 170.32933300746686.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 14.984808693380927 (T) = (0 -155.46704162905615) / Math.Sqrt((36.07588071865975 / (231)) + (60.116498868016166 / (28))) is greater than 1.9692374962334795 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (231) + (28) - 2, .975) and 0.1276109650909242 = (178.20838571780033 - 155.46704162905615) / 178.20838571780033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline c9007c8cd39a5b086a74dd5580b34311041a68ff
Compare b1221494186e7b8e5199d892dae76481a52f5935
Diff Diff

Improvements in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ObjInt - Duration of single invocation 1.49 ms 871.40 μs 0.58 0.56 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.ObjInt


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 871.3976238425925 < 1.0389110539499007.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 8.41811306759192 (T) = (0 -905180.5424303995) / Math.Sqrt((25755867772.11898 / (233)) + (47387851557.976204 / (27))) is greater than 1.9692013862338849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (233) + (27) - 2, .975) and 0.2865769617334435 = (1268785.1301098529 - 905180.5424303995) / 1268785.1301098529 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline c9007c8cd39a5b086a74dd5580b34311041a68ff
Compare b1221494186e7b8e5199d892dae76481a52f5935
Diff Diff

Improvements in Microsoft.Extensions.Logging.ScopesOverheadBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NotFiltered_InsideScope - Duration of single invocation 212.44 ns 188.46 ns 0.89 0.31 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Logging.ScopesOverheadBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered_InsideScope(HasISupportLoggingScopeLogger: True, CaptureScopes: True)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 188.4566222677596 < 197.5847321968716.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 13.789136545115117 (T) = (0 -182.7432702084048) / Math.Sqrt((70.70339262132887 / (233)) + (82.87994538139557 / (27))) is greater than 1.9692013862338849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (233) + (27) - 2, .975) and 0.12171502025927142 = (208.06830860565435 - 182.7432702084048) / 208.06830860565435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline c9007c8cd39a5b086a74dd5580b34311041a68ff
Compare b1221494186e7b8e5199d892dae76481a52f5935
Diff Diff

Improvements in System.IO.Tests.Perf_Directory

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCurrentDirectory - Duration of single invocation 715.29 ns 668.86 ns 0.94 0.23 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Directory*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Directory.GetCurrentDirectory


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 668.8646193152102 < 679.889231663684.
IsChangePoint: Marked as a change because one of 5/26/2022 4:44:18 AM, 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 8.435075122019342 (T) = (0 -672.6569296310379) / Math.Sqrt((1096.5239694684935 / (233)) + (1120.4681740147712 / (27))) is greater than 1.9692013862338849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (233) + (27) - 2, .975) and 0.0785441821007212 = (729.9936866908618 - 672.6569296310379) / 729.9936866908618 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions