[Perf] Changes at 7/7/2022 11:40:51 PM

[performanceautofiler]

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	21a69aa7f70cad997959cce2c33e60d821daca61
Compare	4a63cb28b69e1c48bccf592150be7ba297b67950
Diff	Diff

Regressions in System.Threading.Tests.Perf_Monitor

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EnterExit - Duration of single invocation	19.89 ns	22.04 ns	1.11	0.21	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_Monitor*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Monitor.EnterExit

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.04067352234627 > 20.58504262920396.
IsChangePoint: Marked as a change because one of 7/7/2022 7:47:15 PM, 7/11/2022 5:54:09 PM falls between 7/3/2022 4:26:07 AM and 7/11/2022 5:54:09 PM.
IsRegressionStdDev: Marked as regression because -7.2688174840201105 (T) = (0 -21.97191346917649) / Math.Sqrt((1.6872586448060853 / (30)) + (0.1767391199318404 / (16))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (16) - 2, .025) and -0.0938709626300066 = (20.086385158583205 - 21.97191346917649) / 20.086385158583205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[AndyAyersMS]

Maybe from dotnet/runtime#70992? I can no longer see the diffs.

[AndyAyersMS]

@jakobbotsch maybe take a look?

[jakobbotsch]

On my Raspberry Pi 400 I get:

Method	Job	Toolchain	Mean	Error	StdDev	Median	Min	Max	Ratio	RatioSD	Allocated	Alloc Ratio
EnterExit	Job-DXHTWZ	/base/corerun	47.79 ns	0.008 ns	0.007 ns	47.79 ns	47.78 ns	47.81 ns	1.00	0.00	-	NA
EnterExit	Job-CINUBA	/diff-no-71512/corerun	49.37 ns	0.201 ns	0.178 ns	49.43 ns	48.94 ns	49.52 ns	1.03	0.00	-	NA
EnterExit	Job-PAERIH	/diff/corerun	58.67 ns	0.823 ns	0.770 ns	58.96 ns	56.76 ns	58.97 ns	1.23	0.02	-	NA

So this seems to be caused by dotnet/runtime#71512 (the benchmark just calls Monitor.Enter followed by Monitor.Exit). cc @kunalspathak

base = 21a69aa7f70cad997959cce2c33e60d821daca61
diff = 4a63cb28b69e1c48bccf592150be7ba297b67950
diff-no-71512 = 4a63cb28b69e1c48bccf592150be7ba297b67950 with PR linked above reverted

[kunalspathak]

On my Raspberry Pi 400

Does it have atomics?

[jakobbotsch]

Does it have atomics?

This is what I get out of lscpu, so I don't think so:

Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
Vendor ID:                       ARM
Model:                           3
Model name:                      Cortex-A72
Stepping:                        r0p3
CPU max MHz:                     1800.0000
CPU min MHz:                     600.0000
BogoMIPS:                        108.00
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Vulnerable
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fp asimd evtstrm crc32 cpuid

Do the benchmark machines have atomics? I assume so since there were improvements, which means maybe the uncontended case regressed in either case?

[kunalspathak]

Model name: Cortex-A72
Flags: fp asimd evtstrm crc32 cpuid

https://developer.arm.com/Processors/Cortex-A72 is based on Armv8-A and doesn't have atomics so yes, dotnet/runtime#71512 regresses such scenarios, because in addition to the logic we had, we would be doing an extra check to see if atomics is present. However, the machine where the regression is reported has atomics, so unlikely that it is because of dotnet/runtime#71512.

Narrowing down the commit range for the improvement and regression, this is the commit range:

1. dotnet/runtime@21a69aa...79d5e09
2. dotnet/runtime@79d5e09...8cd9d63