PerfCollect diagnoser #2117

adamsitnik · 2022-09-22T17:44:54Z

I am opening the PR to start a discussion. I know I need to write some docs and test it before it gets merged.

Must have:

Resolving Framework Symbols, it just works OOTB without following https://github.com/dotnet/runtime/blob/main/docs/project/linux-performance-tracing.md#resolving-framework-symbols on my machine, we need to test it for more configs.
enable BDN events by following https://github.com/dotnet/runtime/blob/main/docs/project/linux-performance-tracing.md#filtering

Nice to have:

https://github.com/dotnet/runtime/blob/main/docs/project/linux-performance-tracing.md#getting-symbols-for-the-native-runtime

Please see dotnet/runtime#71786 (comment) for .NET 7 limitations

cc @janvorli

…enerate the path if it has failed

# Conflicts: # src/BenchmarkDotNet.Diagnostics.Windows/Sessions.cs # src/BenchmarkDotNet/Diagnosers/DiagnosersLoader.cs # src/BenchmarkDotNet/Extensions/CommonExtensions.cs

… asked for permission to install stuff (and redirecting input and all that stuff)

# Conflicts: # src/BenchmarkDotNet/BenchmarkDotNet.csproj # src/BenchmarkDotNet/Extensions/CommonExtensions.cs # src/BenchmarkDotNet/Extensions/ProcessExtensions.cs

adamsitnik · 2022-09-22T17:48:30Z

Sample trace file:

BenchmarkDotNet.Samples.IntroPerfCollect.WriteAllText-20220922-193514..trace.zip

BenchmarkDotNet.Samples.IntroPerfCollectProfiler.WriteAllText-20220922-202400..trace.zip

BenchmarkDotNet.Samples.IntroPerfCollectProfiler.WriteAllText-20220926-121051..trace.zip

BenchmarkDotNet.Samples.IntroPerfCollectProfiler.WriteAllText-20220926-154943.trace.zip

standalone.trace.zip

standaloneEnvFull.trace.zip

standaloneEnvSuffix.trace.zip

net7.trace.zip

adamsitnik · 2022-09-22T20:03:41Z

src/BenchmarkDotNet/Extensions/ProcessExtensions.cs

+            {
+                start.EnvironmentVariables["COMPlus_PerfMapEnabled"] = "1";
+                start.EnvironmentVariables["COMPlus_EnableEventLog"] = "1";
+                start.EnvironmentVariables["COMPlus_EventSourceFilter"] = EngineEventSource.SourceName;


@brianrob if I want to enable all events for given event source, is setting COMPlus_EventSourceFilter enough?

The definition of event source that I care about:

BenchmarkDotNet/src/BenchmarkDotNet/Engines/EngineEventSource.cs

Lines 7 to 10 in 21a2940

[EventSource(Name = EngineEventSource.SourceName)]

public class EngineEventSource : EventSource

{

public const string SourceName = "BenchmarkDotNet.EngineEventSource";

Yes, that and setting COMPlus_EnableEventLog=1. If you set COMPlus_EventSourceFilter to the EventSource that you want to capture, this will disable all other EventSources from being sent through to LTTng to avoid the cost of emitting all other EventSource events that may not be needed.

…kely due to failed install)

…rid of extra dot

…crossgen(2)

…times and code refactor

lewing · 2022-10-14T22:22:29Z

This appears to be breaking the wasm tests now that it has made it into dotnet/performance

AndyAyersMS · 2023-07-25T16:50:34Z

I finally got around to trying this out, looks like maybe it does not work with locally built runtimes?

    dotnet run -c Release -f net7.0 -- -filter System.Collections.Sort<IntStruct>.List(Size: 512)  --corerun /home/andy/bugs/r87194/base-rel/corerun --iterationCount 25 -p perf
// Validating benchmarks:
// ***** BenchmarkRunner: Start   *****
// ***** Found 1 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start   *****
// start dotnet  restore /p:EnableUnsafeBinaryFormatterSerialization=true /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in /home/andy/repos/performance/artifacts/bin/MicroBenchmarks/Release/net7.0/b3768f1b-a8be-4010-9bb8-f2b99c0cee65
// command took 3.74s and exited with 0
// start dotnet  build -c Release /p:EnableUnsafeBinaryFormatterSerialization=true --no-restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in /home/andy/repos/performance/artifacts/bin/MicroBenchmarks/Release/net7.0/b3768f1b-a8be-4010-9bb8-f2b99c0cee65
// command took 5.74s and exited with 0
// start dotnet  publish -c Release /p:EnableUnsafeBinaryFormatterSerialization=true --no-build --no-restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true --output "/home/andy/repos/performance/artifacts/bin/MicroBenchmarks/Release/net7.0/b3768f1b-a8be-4010-9bb8-f2b99c0cee65/bin/Release/net7.0/publish" in /home/andy/repos/performance/artifacts/bin/MicroBenchmarks/Release/net7.0/b3768f1b-a8be-4010-9bb8-f2b99c0cee65
// command took 2.93s and exited with 0
Copying /home/andy/repos/performance/artifacts/bin/MicroBenchmarks/Release/net7.0/b3768f1b-a8be-4010-9bb8-f2b99c0cee65/bin/Release/net7.0/publish/CommandLine.dll to /home/andy/bugs/r87194/858959aa-3872-429d-882c-44a1bbe34dcc/CommandLine.dll
// ***** Done, took 00:00:39 (39.01 sec)   *****
// Found 1 benchmarks:
//   Sort<IntStruct>.List: Job-PLNYLF(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, Toolchain=CoreRun, InvocationCount=5000, IterationCount=25, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, MinWarmupIterationCount=6, UnrollFactor=1, WarmupCount=-1) [Size=512]

// **************************
// Benchmark: Sort<IntStruct>.List: Job-PLNYLF(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, Toolchain=CoreRun, InvocationCount=5000, IterationCount=25, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, MinWarmupIterationCount=6, UnrollFactor=1, WarmupCount=-1) [Size=512]
// *** Execute ***
// Launch: 1 / 1
// Execute: /home/andy/bugs/r87194/858959aa-3872-429d-882c-44a1bbe34dcc/corerun b3768f1b-a8be-4010-9bb8-f2b99c0cee65.dll --anonymousPipes 228 229 --benchmarkName "System.Collections.Sort<IntStruct>.List(Size: 512)" --job "PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, Toolchain=CoreRun, InvocationCount=5000, IterationCount=25, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, MinWarmupIterationCount=6, UnrollFactor=1, WarmupCount=-1" --benchmarkId 0 in /home/andy/repos/performance/artifacts/bin/MicroBenchmarks/Release/net7.0/b3768f1b-a8be-4010-9bb8-f2b99c0cee65/bin/Release/net7.0/publish
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at BenchmarkDotNet.Diagnosers.PerfCollectProfiler.EnsureSymbolsForNativeRuntime(DiagnoserActionParameters parameters)
   at BenchmarkDotNet.Diagnosers.PerfCollectProfiler.StartCollection(DiagnoserActionParameters parameters)
   at BenchmarkDotNet.Diagnosers.PerfCollectProfiler.Handle(HostSignal signal, DiagnoserActionParameters parameters)
   at BenchmarkDotNet.Diagnosers.CompositeDiagnoser.Handle(HostSignal signal, DiagnoserActionParameters parameters)
   at BenchmarkDotNet.Toolchains.DotNetCli.DotNetCliExecutor.Execute(BenchmarkCase benchmarkCase, BenchmarkId benchmarkId, ILogger logger, ArtifactsPaths artifactsPaths, IDiagnoser diagnoser, String executableName, IResolver resolver, Int32 launchIndex)
   at BenchmarkDotNet.Toolchains.DotNetCli.DotNetCliExecutor.Execute(ExecuteParameters executeParameters)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.RunExecute(ILogger logger, BenchmarkCase benchmarkCase, BenchmarkId benchmarkId, IToolchain toolchain, BuildResult buildResult, IResolver resolver, IDiagnoser diagnoser, Int32 launchIndex)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.Execute(ILogger logger, BenchmarkCase benchmarkCase, BenchmarkId benchmarkId, IToolchain toolchain, BuildResult buildResult, IResolver resolver)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.RunCore(BenchmarkCase benchmarkCase, BenchmarkId benchmarkId, ILogger logger, IResolver resolver, BuildResult buildResult)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.Run(BenchmarkRunInfo benchmarkRunInfo, Dictionary`2 buildResults, IResolver resolver, ILogger logger, List`1 artifactsToCleanup, String resultsFolderPath, String logFilePath, Int32 totalBenchmarkCount, StartedClock& runsChronometer, Int32& benchmarksToRunCount, TaskbarProgress taskbarProgress)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.Run(BenchmarkRunInfo[] benchmarkRunInfos)
   at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config, Boolean askUserForInput)
   at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config)
   at MicroBenchmarks.Program.Main(String[] args) in /home/andy/repos/performance/src/benchmarks/micro/Program.cs:line 43

adamsitnik · 2023-07-26T08:43:15Z

@AndyAyersMS could you please log an issue? I suspect that we should just skip EnsureSymbolsForNativeRuntime for corerun toolchain

adamsitnik added 15 commits June 9, 2019 18:05

initial LttngProfiler implementation

7c09e43

download the script to the right folder

c4921ad

escepe the arguments, wait until it starts actual collection, don't g…

c59b96d

…enerate the path if it has failed

wait until the script ends post-processing

4171885

use Mono.Posix to send SIGINT (Ctrl+C) to the script

77e3a52

more changes

11c5e06

Merge remote-tracking branch 'origin/master' into perfCollectDiagnoser

eedf8c0

# Conflicts: # src/BenchmarkDotNet.Diagnostics.Windows/Sessions.cs # src/BenchmarkDotNet/Diagnosers/DiagnosersLoader.cs # src/BenchmarkDotNet/Extensions/CommonExtensions.cs

remove duplicated code

2c63c4c

add perfcollect to the resources, don't download it

c77e7df

update doc link

93330c2

use the new start and stop commands

bd7ceb0

use the new install -force option which allows us to avoid user being…

8472f20

… asked for permission to install stuff (and redirecting input and all that stuff)

Merge branch 'master' into perfCollectDiagnoser

f08473b

# Conflicts: # src/BenchmarkDotNet/BenchmarkDotNet.csproj # src/BenchmarkDotNet/Extensions/CommonExtensions.cs # src/BenchmarkDotNet/Extensions/ProcessExtensions.cs

refresh perfcollect

c2d8bf7

use collect command, stop in with Ctrl+C by sending SIGINT

4ffdf5d

adamsitnik added the Area:Diagnosers label Sep 22, 2022

adamsitnik added this to the v0.13.3 milestone Sep 22, 2022

adamsitnik added 2 commits September 22, 2022 20:17

add an attribute and a sample

a6086f8

enable BDN event source

e85c830

adamsitnik commented Sep 22, 2022

View reviewed changes

adamsitnik added 4 commits September 23, 2022 12:44

emit an error when perfcollect finishes sooner than expected (most li…

d2db654

…kely due to failed install)

escape the arguments, store the result only if file was created, get …

220dcde

…rid of extra dot

turn off precompiled code to resolve framework symbols without using …

194f4bf

…crossgen(2)

install dotnet symbols to get symbols for native runtime parts

c0d692a

adamsitnik force-pushed the perfCollectDiagnoser branch from 5b3a633 to c0d692a Compare September 27, 2022 20:33

adamsitnik added 4 commits September 28, 2022 12:51

add workaround for dotnet/runtime#71786

7191b6b

download symbols for all .so files

bb0c66d

polishing: new short name (perf instead PC), running for multiple run…

5b04d07

…times and code refactor

don't turn off precompiled code

5201ed1

adamsitnik mentioned this pull request Sep 28, 2022

perfcollect docs: run two commands instead of three dotnet/runtime#76325

Merged

final polishing before merging

7819a20

adamsitnik merged commit d03287b into master Sep 29, 2022

adamsitnik deleted the perfCollectDiagnoser branch September 29, 2022 14:42

This was referenced Sep 29, 2022

perfcollect & PerfView: unable to get events working microsoft/perfview#1718

Open

Wishlist of features I'd find useful #1954

Open

mawosoft mentioned this pull request Oct 9, 2022

Build warning MSB3245: Could not locate the assembly "Mono.Posix". mawosoft/Mawosoft.Extensions.BenchmarkDotNet#140

Closed

adamsitnik mentioned this pull request Oct 11, 2022

Update provider_compute_event_enable_mask so EventSouces with no keywords show up by default dotnet/runtime#75248

Merged

lewing mentioned this pull request Oct 14, 2022

Bumped BDN version dotnet/performance#2639

Merged

AndyAyersMS mentioned this pull request Jul 26, 2023

-p perf doesn't seem to work right with locally built runtime #2382

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

PerfCollect diagnoser #2117

PerfCollect diagnoser #2117

Uh oh!

adamsitnik commented Sep 22, 2022 •

edited

Loading

Uh oh!

adamsitnik commented Sep 22, 2022 •

edited

Loading

Uh oh!

adamsitnik Sep 22, 2022

Uh oh!

brianrob Sep 22, 2022

Uh oh!

lewing commented Oct 14, 2022

Uh oh!

AndyAyersMS commented Jul 25, 2023

Uh oh!

adamsitnik commented Jul 26, 2023

Uh oh!

Uh oh!

	[EventSource(Name = EngineEventSource.SourceName)]
	public class EngineEventSource : EventSource
	{
	public const string SourceName = "BenchmarkDotNet.EngineEventSource";

Uh oh!

PerfCollect diagnoser #2117

PerfCollect diagnoser #2117

Uh oh!

Conversation

adamsitnik commented Sep 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamsitnik commented Sep 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamsitnik Sep 22, 2022

Choose a reason for hiding this comment

Uh oh!

brianrob Sep 22, 2022

Choose a reason for hiding this comment

Uh oh!

lewing commented Oct 14, 2022

Uh oh!

AndyAyersMS commented Jul 25, 2023

Uh oh!

adamsitnik commented Jul 26, 2023

Uh oh!

Uh oh!

adamsitnik commented Sep 22, 2022 •

edited

Loading

adamsitnik commented Sep 22, 2022 •

edited

Loading