Performance-ci centos tests failing in BDN

[LoopedBard3]

CentOS runs on the performance-ci are failing due to a type initializer exception. This can be seen in the pipeline here https://dev.azure.com/dnceng/public/_build/results?buildId=1838682&view=logs&jobId=c691a557-b8b7-5576-3a35-6adbad78c2b5, the full error is as follows:

[2022/06/22 12:09:18][INFO] System.TypeInitializationException: A type initializer threw an exception. To determine which type, inspect the InnerException's StackTrace property.
[2022/06/22 12:09:18][INFO]  ---> EETypeRva:0x030E3B60(System.Reflection.MissingRuntimeArtifactException): This object cannot be invoked because no code was generated for it: 'BenchmarkDotNet.Jobs.EnvironmentMode.PlatformCharacteristic'.
[2022/06/22 12:09:18][INFO]    at System.Reflection.Runtime.FieldInfos.RuntimeFieldInfo.get_FieldAccessor() + 0x11e
[2022/06/22 12:09:18][INFO]    at System.Reflection.Runtime.FieldInfos.RuntimeFieldInfo.GetValue(Object) + 0x10
[2022/06/22 12:09:18][INFO]    at BenchmarkDotNet.Characteristics.CharacteristicHelper.<>c.<GetThisTypeCharacteristicsCore>b__8_1(FieldInfo) + 0x19
[2022/06/22 12:09:18][INFO]    at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext() + 0x80
[2022/06/22 12:09:18][INFO]    at System.Linq.Enumerable.ConcatIterator`1.MoveNext() + 0x61
[2022/06/22 12:09:18][INFO]    at System.Collections.Generic.HashSet`1.UnionWith(IEnumerable`1) + 0x3f
[2022/06/22 12:09:18][INFO]    at System.Collections.Generic.HashSet`1..ctor(IEnumerable`1, IEqualityComparer`1) + 0xfa
[2022/06/22 12:09:18][INFO]    at System.Linq.Enumerable.DistinctIterator`1.ToArray() + 0x36
[2022/06/22 12:09:18][INFO]    at System.Linq.Buffer`1..ctor(IEnumerable`1) + 0x34
[2022/06/22 12:09:18][INFO]    at System.Linq.OrderedEnumerable`1.ToArray() + 0x38
[2022/06/22 12:09:18][INFO]    at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey, Func`2) + 0x8d
[2022/06/22 12:09:18][INFO]    at BenchmarkDotNet.Characteristics.CharacteristicHelper.FillAllCharacteristicsCore(Type, List`1, HashSet`1) + 0x28
[2022/06/22 12:09:18][INFO]    at BenchmarkDotNet.Characteristics.CharacteristicHelper.GetAllCharacteristicsCore(Type) + 0x47
[2022/06/22 12:09:18][INFO]    at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey, Func`2) + 0x8d
[2022/06/22 12:09:18][INFO]    at BenchmarkDotNet.Characteristics.CharacteristicObject.GetCharacteristicsToApply() + 0x2c
[2022/06/22 12:09:18][INFO]    at BenchmarkDotNet.Characteristics.CharacteristicObject.GetCharacteristicsToApply(CharacteristicObject) + 0x1c
[2022/06/22 12:09:18][INFO]    at BenchmarkDotNet.Jobs.Job..cctor() + 0x204
[2022/06/22 12:09:18][INFO]    at System.Runtime.CompilerServices.ClassConstructorRunner.EnsureClassConstructorRun(StaticClassConstructionContext*) + 0xb9
[2022/06/22 12:09:18][INFO]    --- End of inner exception stack trace ---
[2022/06/22 12:09:18][INFO]    at System.Runtime.CompilerServices.ClassConstructorRunner.EnsureClassConstructorRun(StaticClassConstructionContext*) + 0x153
[2022/06/22 12:09:18][INFO]    at System.Runtime.CompilerServices.ClassConstructorRunner.CheckStaticClassConstructionReturnGCStaticBase(StaticClassConstructionContext*, Object) + 0x9
[2022/06/22 12:09:18][INFO]    at BenchmarkDotNet.Jobs.Job..ctor(String) + 0x57
[2022/06/22 12:09:18][INFO]    at BenchmarkDotNet.Autogenerated.Runnable_0.Run(IHost, String) + 0x1b4
[2022/06/22 12:09:18][INFO]    at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[]) + 0x256

[LoopedBard3]

@adamsitnik @naricc Have y'all seen an error like this before with BDN? Here is a log with the error: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-performance-refs-heads-main-367830c2c6914864b5/Partition0/1/console.f1d1238a.log?helixlogtype=result
Thanks!

[adamsitnik]

Hi @LoopedBard3 !

@MichalStrehovsky has recently fixed this issue with dotnet/BenchmarkDotNet#2020

I've just fixed a recent breaking change: dotnet/BenchmarkDotNet#2045

To fix these builds we need is to upload 0.13.1.1818 to our internal feed and update BDN version in perf repo. Could you please do that?

BTW BenchmarkDotNet has recently updated all it's dependencies like TraceEvent, ClrMd and Iced. Some of them might not be in our internal feed(s) yet and you might need to upload them as well.

[LoopedBard3]

Taking a look!

[LoopedBard3]

It looks like there is a new 0.13.1.18.19 version, any reason not to take that one?

[MichalStrehovsky]

0.13.1.1820 should have my fix. Can we try with that one?

[adamsitnik]

@MichalStrehovsky I've verified that your fix has solved this problem in #2534, but a new one popped out: dotnet/roslyn#62864

@LoopedBard3 could you please update to 0.13.1.1820? It's going to help and then as soon as dotnet/roslyn#62864 gets fixed the CI should become green again

[MichalStrehovsky]

@MichalStrehovsky I've verified that your fix has solved this problem in #2534, but a new one popped out: dotnet/roslyn#62864

If it's blocking, downgrading the Roslyn used to build this repo to a build from 2 weeks ago would work around.

[LoopedBard3]

Updated to 0.13.1.1823 in the #2532 PR.

[LoopedBard3]

Looks like the BDN changes are running properly, but there are some tests that are unsupported:

[2022/07/22 15:50:58][INFO] // Found 1 benchmarks:
[2022/07/22 15:50:58][INFO] //   MandelBrot_7.MandelBrot_7: Job-CLDRFN(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=NativeAOT 7.0, Toolchain=ILCompiler 7.0.0-*, InvocationCount=1, IterationCount=1, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=0) [size=4000, lineLength=500, checksum=C7-E6-66-43-6(...)7-2F-FC-A1-D3 [47]]
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] // **************************
[2022/07/22 15:50:58][INFO] // Benchmark: MandelBrot_7.MandelBrot_7: Job-CLDRFN(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=NativeAOT 7.0, Toolchain=ILCompiler 7.0.0-*, InvocationCount=1, IterationCount=1, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=0) [size=4000, lineLength=500, checksum=C7-E6-66-43-6(...)7-2F-FC-A1-D3 [47]]
[2022/07/22 15:50:58][INFO] // *** Execute ***
[2022/07/22 15:50:58][INFO] // Launch: 1 / 1
[2022/07/22 15:50:58][INFO] // Execute: /home/helixbot/work/A626091F/w/BCD109E7/e/artifacts/bin/MicroBenchmarks/Release/net7.0/91d5ac24-fe26-461f-afdd-5e7aeda1a89d/bin/Release/net7.0/linux-x64/publish/91d5ac24-fe26-461f-afdd-5e7aeda1a89d --benchmarkName "BenchmarksGame.MandelBrot_7.Bench(size: 4000, lineLength: 500, checksum: \"C7-E6-66-43-66-73-F8-A8-D3-B4-D7-97-2F-FC-A1-D3\")" --job "PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=NativeAOT 7.0, Toolchain=ILCompiler 7.0.0-*, InvocationCount=1, IterationCount=1, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=0" --benchmarkId 9 in
[2022/07/22 15:50:58][INFO] Failed to set up high priority. Make sure you have the right permissions. Message: Permission denied
[2022/07/22 15:50:58][INFO] // BeforeAnythingElse
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] // Benchmark Process Environment Information:
[2022/07/22 15:50:58][INFO] // Runtime=.NET 7.0.0-rc.1.22372.1, X64 NativeAOT
[2022/07/22 15:50:58][INFO] // GC=Non-concurrent Workstation
[2022/07/22 15:50:58][INFO] // Job: Job-UDTYPC(PowerPlanMode=00000000-0000-0000-0000-000000000000, InvocationCount=1, IterationCount=1, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=0)
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] // BeforeActualRun
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] System.PlatformNotSupportedException: Operation is not supported on this platform.
[2022/07/22 15:50:58][INFO]    at Internal.Runtime.CompilerHelpers.ThrowHelpers.ThrowPlatformNotSupportedException() + 0x20
[2022/07/22 15:50:58][INFO]    at BenchmarksGame.MandelBrot_7.DoBench(Int32, Int32) + 0x8a
[2022/07/22 15:50:58][INFO]    at BenchmarkDotNet.Autogenerated.Runnable_9.WorkloadActionUnroll(Int64) + 0x4d
[2022/07/22 15:50:58][INFO]    at BenchmarkDotNet.Engines.Engine.RunIteration(IterationData) + 0x1d3
[2022/07/22 15:50:58][INFO]    at BenchmarkDotNet.Engines.EngineStage.RunIteration(IterationMode, IterationStage, Int32, Int64, Int32) + 0x87
[2022/07/22 15:50:58][INFO]    at BenchmarkDotNet.Engines.EngineActualStage.RunSpecific(Int64, IterationMode, Int32, Int32) + 0x79
[2022/07/22 15:50:58][INFO]    at BenchmarkDotNet.Engines.Engine.Run() + 0x291
[2022/07/22 15:50:58][INFO]    at BenchmarkDotNet.Autogenerated.Runnable_9.Run(IHost, String) + 0x6af
[2022/07/22 15:50:58][INFO]    at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[]) + 0x2ee
[2022/07/22 15:50:58][INFO] // AfterAll
[2022/07/22 15:50:58][INFO] No Workload Results were obtained from the run.
[2022/07/22 15:50:58][INFO] // Benchmark Process 9718 has exited with code 255.
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] // ** Remained 852 (98.8%) benchmark(s) to run. Estimated finish 2022-07-22 16:13 (0h 22m from now) **
[2022/07/22 15:50:58][INFO] // ***** BenchmarkRunner: Finish  *****
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] // * Export *
[2022/07/22 15:50:58][INFO]   home/helixbot/work/A626091F/w/BCD109E7/e/artifacts/bin/MicroBenchmarks/Release/net7.0/BenchmarkDotNet.Artifacts/results/BenchmarksGame.MandelBrot_7-report-github.md
[2022/07/22 15:50:58][INFO]   home/helixbot/work/A626091F/w/BCD109E7/e/artifacts/bin/MicroBenchmarks/Release/net7.0/BenchmarkDotNet.Artifacts/results/BenchmarksGame.MandelBrot_7-report-full.json
[2022/07/22 15:50:58][INFO]   home/helixbot/work/A626091F/w/BCD109E7/e/artifacts/bin/MicroBenchmarks/Release/net7.0/BenchmarkDotNet.Artifacts/results/BenchmarksGame.MandelBrot_7-perf-lab-report.json
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] // * Detailed results *
[2022/07/22 15:50:58][INFO] MandelBrot_7.MandelBrot_7: Job-CLDRFN(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=NativeAOT 7.0, Toolchain=ILCompiler 7.0.0-*, InvocationCount=1, IterationCount=1, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=0) [size=4000, lineLength=500, checksum=C7-E6-66-43-6(...)7-2F-FC-A1-D3 [47]]
[2022/07/22 15:50:58][INFO] Runtime = .NET 7.0.0-rc.1.22372.1, X64 NativeAOT; GC = Non-concurrent Workstation
[2022/07/22 15:50:58][INFO] There are not any results runs
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] // * Summary *
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] BenchmarkDotNet=v0.13.1.1823-nightly, OS=centos 7
[2022/07/22 15:50:58][INFO] AMD EPYC 7452, 1 CPU, 2 logical cores and 1 physical core
[2022/07/22 15:50:58][INFO] .NET SDK=7.0.100-rc.1.22371.5
[2022/07/22 15:50:58][INFO]   [Host]     : .NET 7.0.0 (7.0.22.36704), X64 RyuJIT
[2022/07/22 15:50:58][INFO]   Job-CLDRFN : .NET 7.0.0-rc.1.22372.1, X64 NativeAOT
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] PowerPlanMode=00000000-0000-0000-0000-000000000000  Runtime=NativeAOT 7.0  Toolchain=ILCompiler 7.0.0-*
[2022/07/22 15:50:58][INFO] InvocationCount=1  IterationCount=1  IterationTime=250.0000 ms
[2022/07/22 15:50:58][INFO] MaxIterationCount=20  MinIterationCount=15  RunStrategy=ColdStart
[2022/07/22 15:50:58][INFO] UnrollFactor=1  WarmupCount=0
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] |       Method | size | lineLength |                             checksum | Mean | Error | Median | Min | Max |
[2022/07/22 15:50:58][INFO] |------------- |----- |----------- |------------------------------------- |-----:|------:|-------:|----:|----:|
[2022/07/22 15:50:58][INFO] | MandelBrot_7 | 4000 |        500 | C7-E6-66-43-6(...)7-2F-FC-A1-D3 [47] |   NA |    NA |     NA |  NA |  NA |
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] Benchmarks with issues:
[2022/07/22 15:50:58][INFO]   MandelBrot_7.MandelBrot_7: Job-CLDRFN(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=NativeAOT 7.0, Toolchain=ILCompiler 7.0.0-*, InvocationCount=1, IterationCount=1, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=0) [size=4000, lineLength=500, checksum=C7-E6-66-43-6(...)7-2F-FC-A1-D3 [47]]
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] // * Legends *
[2022/07/22 15:50:58][INFO]   size       : Value of the 'size' parameter
[2022/07/22 15:50:58][INFO]   lineLength : Value of the 'lineLength' parameter
[2022/07/22 15:50:58][INFO]   checksum   : Value of the 'checksum' parameter
[2022/07/22 15:50:58][INFO]   Mean       : Arithmetic mean of all measurements
[2022/07/22 15:50:58][INFO]   Error      : Half of 99.9% confidence interval
[2022/07/22 15:50:58][INFO]   Median     : Value separating the higher half of all measurements (50th percentile)
[2022/07/22 15:50:58][INFO]   Min        : Minimum
[2022/07/22 15:50:58][INFO]   Max        : Maximum
[2022/07/22 15:50:58][INFO]   1 ns       : 1 Nanosecond (0.000000001 sec)
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] // * Diagnostic Output - MemoryDiagnoser *
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] 
[2022/07/22 15:50:58][INFO] // ***** BenchmarkRunner: End *****
[2022/07/22 15:50:58][INFO] Run time: 00:00:00 (0.04 sec), executed benchmarks: 1

If y'all see something immediately obvious as to why this error was hit (pertaining to what isn't supported, not that something is not supported), please respond.