Logs: Support parsing State & Scopes and capture of formatted Message

[CodeBlanch]

Fixes #1834

Changes

[Originally part of #1833 but @cijothomas requested that it be split up.]

• Exposes scope, message, & parsed state features on LogRecord
• Default behavior is exactly as it is in 1.0 (no message, no scopes, not parsing state)
• When turned on state parsing uses some IL magic to not box structs or allocate enumerators. It will allocate a list to store the parsed KVPs. We do have pooled list capability but they have to be returned to the pool and the API is not very nice. Since these have to go to userland an allocation seemed unavoidable. I do pass the size/capacity to the list ctor in order to reduce the growth penalty.

Public API

public class OpenTelemetryLoggerOptions
{
        // Turns on parsing scopes onto tags. Default: False
        public bool IncludeScopes { get; set; }

        // Turns on calling the formatter. Default: False
        public bool IncludeMessage { get; set; }

        // Turns on parsing state onto tags. Default: False
        public bool ParseStateValues { get; set; }
}

public class LogRecord
{
// Existing-----------------------------------------------------------------------------------//
        // Will now be set to null if ParseStateValues is turned on. To avoid boxing.
        public object State { get; }

// New--------------------------------------------------------------------------------------//
        // Set to the result of the formatter if IncludeMessage is turned on.
        public string Message { get; }

        // Set to the parsing results if ParseStateValues is turned on.
        public IReadOnlyList<KeyValuePair<string, object>> StateValues { get; }

        // Invoke a callback for each registered scope. Does a no-op if IncludeScopes is turned off.
        public void ForEachScope<TState>(Action<object, TState> callback, TState state)
}

TODOs

• CHANGELOG.md updated for non-trivial changes
• Unit tests
• Design discussion issue #
• Changes in public API reviewed

[reyang]

No strong opinion - my personal preference would be splitting this into two "if" statements. In this way it is easier to put breakpoint on the selected return statement instead of putting a condition-breakpoint (which is much slower since it uses debugger expression-evaluation engine).

if (condition1)
{
    return;
}

if (condition2)
{
    return;
}

[reyang]

LGTM.

[codecov]

Codecov Report

Merging #1869 (215e625) into main (cb066cb) will decrease coverage by 0.42%.
The diff coverage is 78.36%.

@@            Coverage Diff             @@
##             main    #1869      +/-   ##
==========================================
- Coverage   83.77%   83.35%   -0.43%     
==========================================
  Files         187      189       +2     
  Lines        5967     6128     +161     
==========================================
+ Hits         4999     5108     +109     
- Misses        968     1020      +52     

Impacted Files	Coverage Δ
src/OpenTelemetry/Logs/OpenTelemetryLogger.cs	26.82% <20.54%> (-44.60%)	⬇️
src/OpenTelemetry/Logs/LogRecord.cs	82.50% <77.77%> (-5.00%)	⬇️
.../OpenTelemetry/Logs/OpenTelemetryLoggerProvider.cs	77.55% <88.23%> (+7.09%)	⬆️
...rc/OpenTelemetry/Trace/TracerProviderBuilderSdk.cs	88.46% <88.88%> (-2.28%)	⬇️
src/OpenTelemetry/Trace/ExceptionProcessor.cs	90.90% <90.90%> (ø)
src/OpenTelemetry/Trace/TracerProviderSdk.cs	95.16% <98.64%> (+4.38%)	⬆️
...metryProtocol/Implementation/ActivityExtensions.cs	86.89% <100.00%> (+0.05%)	⬆️
...ry.Instrumentation.AspNet/AspNetInstrumentation.cs	100.00% <100.00%> (ø)
...umentation.AspNet/Implementation/HttpInListener.cs	89.10% <100.00%> (ø)
...entation.AspNet/TracerProviderBuilderExtensions.cs	100.00% <100.00%> (ø)
... and 24 more

[cijothomas]

Suggested change

	public bool IncludeMessage { get; set; }
	public bool IncludeFormattedMessage { get; set; }

I think IncludeFormattedMessage is a better name.

[CodeBlanch]

Sounds good. What about LogRecord.Message you want that to be LogRecord.FormattedMessage or leave as LogRecord.Message?

[cijothomas]

Thanks!
Left couple of small non blocking comments.
We can iterate on this as needed. (and add Unit tests)