Small reorganization of docs to move logging up to its own topic, and…

[samsp-msft]

… adding an overview for tracing that points to all the other topics

Summary

Moves the logging topic out of tracing, and renames logging and tracing to performance tracing.
Adds a new overview for performance tracing pointing at the APIs and tools that can be used.

Fixes #2911

---

Internal previews

📄 File	🔗 Preview link
docs/core/diagnostics/logging-tracing.md	.NET logging and tracing
docs/core/diagnostics/performance-tracing.md	docs/core/diagnostics/performance-tracing

[noahfalk]

This organization feels more confusing to me now, but I'm hoping if I know a little more what you wanted to achieve we can find a good way to do it.

[noahfalk]

Today ILogger is the only story for logging, but in the near future (6 months?) that might not be the case with Blanch's work on the logging bridge API. I'm fine if we add this for now, but we probably have to take back out soon :)

Suggested change

	ILogger provides the logging story for the Open Telemetry implementation for .NET, which enables egress of logs from you application to a variety of APM systems for further analysis.
	ILogger provides the logging story for the Open Telemetry implementation for .NET, which enables egress of logs from your application to a variety of APM systems for further analysis.

[noahfalk]

I'm fine with the idea that we'd give people some information specifically through a lense of performance tracing, but I don't like that we'd home all of the EventSource, DiagnosticSource, and EventPipe information there. Most DiagnosticSource usage is not performance tracing and EventSource/EventPipe have significant uses that are both performance and non-performance related. To me this organization seems more confusing than before.

Is the goal to highlight ILogger better? What if we just made ILogger first under
logging and tracing?

Logging and Tracing
- ILogger 
- EventSource
  - ...
  - Well-known event providers
  - EventPipe
- DiagnosticSource

[samsp-msft]

We need to move ILogger out as its the #1 thing that users will use.
Then step back and ask yourself "When are users going to use EventSource or DiagnosticSource?" I would suggest that most of that is going to be for diagnosing performance.

[noahfalk]

We need to move ILogger out as its the #1 thing that users will use.

If you want to make ILogger extra prominent how about:

- ILogger
- Logging and Tracing
  - ILogger 
  - EventSource
    - ...
    - Well-known event providers
    - EventPipe
  - DiagnosticSource

I don't think making logger prominent requires anything else to be re-arranged.

Then step back and ask yourself "When are users going to use EventSource or DiagnosticSource?" I would suggest that most of that is going to be for diagnosing performance.

If a customer asked me I'd answer:
EventSource - 60% diagnosing performance, 20% functional logging, 20% generic out-of-proc transport for other types of telemetry. Prominent examples of its use for functional logging are OTel SDK and Azure SDK.
DiagnosticSource - 60% APM/distributed tracing, 20% custom in-proc diagnostic tools, 20% performance tracing (when combined with DiagnosticSourceEventSource)

I think characterizing these tools as being primarily for "performance tracing" is simplified for EventSource (I acknowledge there are times where simple is good) and wrong for DiagnosticSource. Saying they are "for diagnosing performance" is a somewhat better fit, but that is a much broader characterization than "performance tracing" which would also include technologies like metrics and distributed tracing too.

Over time I think we are going to see increasing scenario crossover between ILogger and EventSource, not less. For example I'd like to change dotnet-trace so that you can specify ILogger categories as providers and have options for text or binary formats. OpenTelemetry is trying to make EventSource be another API you can use to produce the logging signal. I expect creating a view where logging and performance tracing are treated as very different and APIs should only be thought of as doing one or the other will be an increasingly strained organizational principle over time.

[samsp-msft]

I think the amount of direct customer usage of EventSource in their code, either emitting or consuming events is going to be minimal - it's going to pretty much be relegated to us as part of the framework/runtime. The interaction that customers will have with EventSource is via a trace, which is most likely to be performance related.

[noahfalk]

To the best of my knowledge DiagnosticSource is not often used for performance tracing. The main uses I've seen are APM tools and OpenTelemetry. Occasionally it gets used in combination with DiagnosticSourceEventSource for things like VS profiler database profiling.

[samsp-msft]

I think we need a better title than just tracing - Event Tracing or Performance & Diagnostics tracing are ideas I had.

[IEvangelist]

We have to address the merge conflict with the docs/fundamentals/toc.yml.

[samsp-msft]

This organization feels more confusing to me now, but I'm hoping if I know a little more what you wanted to achieve we can find a good way to do it.

There are a couple of things here:

• ILogger doesn't appear in this area's TOC, as that is the primary API users should be pointed at, and with the .NET 8 work will be extended
  • I don't know if linking that in here is what is causing a TOC merge problem
• Moving the topic which talks about logging mechanisms up a level
• Adding an overview for "Tracing". I called it "Performance Tracing" as we need to distinguish between this and distributed tracing. If you don't like performance, how about "Event Tracing"?

[noahfalk]

ILogger doesn't appear in this area's TOC, as that is the primary API users should be pointed at, and with the .NET 8 work will be extended

I'd suggest we add it within the 'Logging and Tracing' node, and if that still doesn't feel prominent enough then we can add another link to the same content right at the top level:

Diagnostics and Instrumentation
- Overview
- ILogger
- Logging and Tracing
  - Overview
  - ILogger
  - ...

I also think the text of the main diagnostics overview page could be cleaned up a bunch to highlight key information faster and spend less time talking about diagnostics 101 information that isn't specific to .NET. We should be able to get links to logging and ILogger in the text users will see without having scrolled down.

• Moving the topic which talks about logging mechanisms up a level

From what I've seen an item in the ToC can either link to a page or it can expand to have children, but not both. IMO the tiny extra discoverability you might get from users not needing the extra click to expand the 'logging and tracing' node and hit the Overview link isn't worth the lost discoverability reparenting all the children somewhere else. There is also a nice symmetry of Logging, Metrics, and Distributed Tracing right now - I'm not sure why this would need to change for logging but not for the others.

• Adding an overview for "Tracing". I called it "Performance Tracing" as we need to distinguish between this and distributed tracing. If you don't like performance, how about "Event Tracing"?

I'm fine with either Performance Tracing or Event Tracing, I just think the natural place for anything named 'xxx tracing' would be under the 'Logging and Tracing' node. If you want to treat it as a separate scenario with a top-level item then I think we should call it 'Profiling' and put the focus on the tools like VS profiler, dotnet-trace, and PerfView. EventSource might make an appearance in such a page and have links, but I wouldn't make it the star or re-parent all our EventSource related information there. Users who create their own EventSource to customize the trace are probably in the advanced 10% of the scenario. Users who use a custom DiagnosticSource there are probably in the 1% or 0.1%.

[samsp-msft]

ILogger doesn't appear in this area's TOC, as that is the primary API users should be pointed at, and with the .NET 8 work will be extended

I'd suggest we add it within the 'Logging and Tracing' node, and if that still doesn't feel prominent enough then we can add another link to the same content right at the top level:

ILogger is a different style of technology - It's not being used the same way as EventSource - which is why I want to separate them, and move ILogger up to the root.

[noahfalk]

ILogger is a different style of technology - It's not being used the same way as EventSource - which is why I want to separate them, and move ILogger up to the root.

If we let people create OpenTelemetry logging using EventSource and write timestamped messages into nettrace and .etl files with ILogger would you still feel like they are 'different style of technology'? The story I'd like to tell in the future is that ILogger handles all these cases well and you don't need to instrument your code with two different APIs depending on whether you are diagnosing a functional issue or a performance issue.

If you want to treat performance tracing like it is a separate island from other logging I think you could tell a decent consistent story about it today, with two caveats:

• I would only put EventSource under that umbrella, not DiagnosticSource.
• Over time I think the story will get increasingly blurry. In particular at the point we have 1st class support for using ILogger in .etl and nettrace I hope the docs say "ILogger works well for all these things" and not "ILogger is logging, performance tracing is different so you need to use EventSource instead"