Description
Background and motivation
Currently, apps publish logs to capture telemetry data for monitoring the system's health. Typically, the logged data is sent to a backend for analysis. Often, these logs become useful only when an issue arises, necessitating a review to identify the problem. When the system is functioning normally, the logs are discarded, but the cost of collecting and transmitting them has already been incurred.
Log buffering addresses this by storing logs in memory instead of immediately sending them to the backend. This allows a decision to be made later on whether to send the buffered logs to their destination or discard them, thereby avoiding unnecessary costs.
This log buffering feature will be implemented in the upstream dotnet/extensions library as part of the existing ExtendedLogger design. To enable this feature to work with any logger provider, we need to introduce a buffering abstraction. Logger providers can opt to implement this buffering abstraction interface, which will enable buffering functionality automatically for those providers.
The proposal here focuses on the abstraction types needed in the runtime:
IBufferedLogger: an interface that logger providers must implement to support buffering.BufferedLogRecord: a proposed class used by the infrastructure to hold buffered state in memory. When ready to be flushed out, this object is passed to logging providers without copying.BufferedLogRecordis designed to be extendable over time.- The infrastructure will maintain a pool of
BufferedLogRecordobjects that are recycled over time. This is an implementation detail forExtendedLogger.
API Proposal
namespace Microsoft.Extensions.Logging.Abstractions;
/// <summary>
/// Logging providers can implement this interface to indicate they support buffered logging.
/// </summary>
/// <remarks>
/// A logging provider normally exposes an ILogger<T> interface that gets invoked by the
/// logging infrastructure whenever it’s time to log a piece of state.
///
/// The logging infrastructure will type-test the ILogger<T> object to determine if
/// it supports the <c>IBufferedLogger</c> interface also. If it does, that tells the
/// logging infrastructure that the logging provider supports buffering. Whenever log
/// buffering is enabled, buffered log records will be delivered to the logging provider
/// via the <c>IBufferedLogger</c> interface.
///
/// If a logging provider does not support log buffering, then it will always be given
/// unbuffered log records. In other words, whether or not buffering is requested by
/// the user, it will not happen for those log providers.
/// </remarks>
public interface IBufferedLogger
{
/// <remarks>
/// Once this function returns, it should no longer access the record objects.
/// </remarks>
void LogRecords(IReadOnlyList<Microsoft.Extensions.Logging.BufferedLogRecord> records);
}
/// <summary>
/// State representing a buffered log entry.
/// </summary>
/// <remarks>
/// Objects of this type are reused over time to reduce allocations.
/// </remarks>
public abstract class BufferedLogRecord
{
public abstract DateTimeOffset Timestamp { get; }
public abstract LogLevel LogLevel { get; }
public abstract EventId EventId { get; }
public virtual string? Exception { get => null; }
public virtual ActivitySpanId? ActivitySpanId { get => null; }
public virtual ActivityTraceId? ActivityTraceId { get => null; }
public virtual ActivityTraceFlags? ActivityTraceFlags { get => null; }
public virtual int? ManagedThreadId { get => null; }
public virtual string? FormattedMessage { get => null; }
public virtual string? MessageTemplate { get => null; }
public virtual IReadOnlyList<KeyValuePair<string, object?>>? Attributes { get => null; }
}API Usage
Alternative Designs
No response
Risks
No response