Skip to content

Update High CPU Tutorial #29112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 28, 2022
Merged

Update High CPU Tutorial #29112

Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3a0464a
Update High CPU Tutorial
mikelle-rogers Apr 19, 2022
33a3a73
Add additional perfview images for clarity
mikelle-rogers Apr 20, 2022
4c22c05
Revert "Add additional perfview images for clarity"
mikelle-rogers Apr 20, 2022
9b6dde8
Update high CPU to clarify process
mikelle-rogers Apr 20, 2022
fc6194f
Remove trailing spaces and lines
mikelle-rogers Apr 25, 2022
1b3ff39
Explain why we chose perf for Linux
mikelle-rogers Apr 26, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 28 additions & 9 deletions docs/core/diagnostics/debug-highcpu.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,32 +83,50 @@ With the web app running, immediately after startup, the CPU isn't being consume

`https://localhost:5001/api/diagscenario/highcpu/60000`

Now, rerun the [dotnet-counters](dotnet-counters.md) command. To monitor just the `cpu-usage`, specify `System.Runtime[cpu-usage]` as part of the command.
Now, rerun the [dotnet-counters](dotnet-counters.md) command. If interested in monitoring just the `cpu-usage` counter, add '--counters System.Runtime[cpu-usage]` to the previous command. We are unsure if the CPU is being consumed, so we will monitor the same list of counters as above to verify counter values are within expected range for our application.

```dotnetcli
dotnet-counters monitor --counters System.Runtime[cpu-usage] -p 22884 --refresh-interval 1
dotnet-counters monitor -p 22884 --refresh-interval 1
```

You should see an increase in CPU usage as shown below:
You should see an increase in CPU usage as shown below (depending on the host machine, expect varying CPU usage):

```console
Press p to pause, r to resume, q to quit.
Status: Running

[System.Runtime]
% Time in GC since last GC (%) 0
Allocation Rate / 1 sec (B) 0
CPU Usage (%) 25
Exception Count / 1 sec 0
GC Heap Size (MB) 4
Gen 0 GC Count / 60 sec 0
Gen 0 Size (B) 0
Gen 1 GC Count / 60 sec 0
Gen 1 Size (B) 0
Gen 2 GC Count / 60 sec 0
Gen 2 Size (B) 0
LOH Size (B) 0
Monitor Lock Contention Count / 1 sec 0
Number of Active Timers 1
Number of Assemblies Loaded 140
ThreadPool Completed Work Item Count / 1 sec 3
ThreadPool Queue Length 0
ThreadPool Thread Count 7
Working Set (MB) 63
```

Throughout the duration of the request, the CPU usage will hover around 25% . Depending on the host machine, expect varying CPU usage.
Throughout the duration of the request, the CPU usage will hover around the increased percentage.

> [!TIP]
> To visualize an even higher CPU usage, you can exercise this endpoint in multiple browser tabs simultaneously.

At this point, you can safely say the CPU is running higher than you expect.
At this point, you can safely say the CPU is running higher than you expect. Identifying the effects of a problem is key to finding the cause. We will use the effect of high CPU consumption in addition to diagnostic tools to find the cause of the problem.

## Trace generation
## Analyze High CPU with Profiler

When analyzing a slow request, you need a diagnostics tool that can provide insights into what the code is doing. The usual choice is a profiler, and there are different profiler options to choose from.
When analyzing an app with high CPU usage, you need a diagnostics tool that can provide insights into what the code is doing. The usual choice is a profiler, and there are different profiler options to choose from.

### [Linux](#tab/linux)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we explain why we suggest using the Linux perf tool and not dotnet-trace? I'm guessing the reason is something like: Although Dotnet-Trace is cross-platform but since Event Pipe "CPU" traces only measure elapsed time and not actual CPU time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had the same question. @josalem Why are we using the Linux perf tool and not dotnet-trace?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dotnet-trace is great for CPU investigations to a certain point. The limitations of safe-point bias and managed-only callstacks means that it can't give exact information the same way a kernel-aware profiler like perf can.

If your performance investigation stops at managed code, generally dotnet-trace will be sufficient. If you are analyzing exact timing of your application and it involves native code, you may need something like perf.

I think of it like two differently powered magnifying glasses. You can start with dotnet-trace and then look at a higher resolution using perf (or ETW on Windows).

Expand Down Expand Up @@ -150,15 +168,16 @@ This command generates a `flamegraph.svg` that you can view in the browser to in

### [Windows](#tab/windows)

On Windows, you can use the [dotnet-trace](dotnet-trace.md) tool as a profiler. Using the previous [sample debug target](/samples/dotnet/samples/diagnostic-scenarios), exercise the high CPU endpoint (`https://localhost:5001/api/diagscenario/highcpu/60000`) again. While it's running within the 1-minute request, use the `collect` command as follows:
On Windows, you can use the [dotnet-trace](dotnet-trace.md) tool as a profiler. Using the previous [sample debug target](/samples/dotnet/samples/diagnostic-scenarios), exercise the high CPU endpoint (`https://localhost:5001/api/diagscenario/highcpu/60000`) again. While it's running within the 1-minute request, use the `collect` command, with the `providers` option to specify the provider we want: [Microsoft-DotNetCore-SampleProfiler](/well-known-event-providers?msclkid=7a52966cc0d211ec96f7684d71861dc5#microsoft-dotnetcore-sampleprofiler-provider), to collect a trace of the app as follows:

```dotnetcli
dotnet-trace collect -p 22884 --providers Microsoft-DotNETCore-SampleProfiler
```

Let [dotnet-trace](dotnet-trace.md) run for about 20-30 seconds, and then press the <kbd>Enter</kbd> to exit the collection. The result is a `nettrace` file located in the same folder. The `nettrace` files are a great way to use existing analysis tools on Windows.

Open the `nettrace` with [`PerfView`](https://github.com/microsoft/perfview/blob/main/documentation/Downloading.md) as shown below.

Open the `nettrace` with [`PerfView`](https://github.com/microsoft/perfview/blob/main/documentation/Downloading.md) by navigating to samples/core/diagnostics/DiagnosticScenarios/ and clicking on the arrow by the `nettrace` file. Open the 'Thread Time (with StartStop Activities) Stacks' and choose the 'CallTree' tab near the top. After checking the box to the left of one of the threads, your file should look similar to the one pictured below.

[![PerfView image](media/perfview.jpg)](media/perfview.jpg#lightbox)

Expand Down