How to use with sinks that depend on ConfigureServices being run first?

[cmeeren]

I'm using this with Serilog.Sinks.ApplicationInsights, but AFAIK there seems to be a conflict:

• When calling .UseSerilog with the configureLogger parameter, configureLogger (which among other things calls Serilog.Sinks.ApplicationInsights' WriteTo.ApplicationInsights) is executed before ASP.NET Core's ConfigureServices.
• ConfigureServices sets up TelemetryConfiguration.Active
• WriteTo.ApplicationInsights needs TelemetryConfiguration.Active to be set up correctly

Is there a way to solve this?

[cmeeren]

cc @aloneguid

[cmeeren]

Actually, as far as the title and original description go, this is still a problem now that TelemetryConfiguration.Active is deprecated (microsoft/ApplicationInsights-dotnet#1152).

In short:

• The Application Insights telemetry config is set up and registered with DI in ConfigureServices (so that it picks up all telemetry initializers and processors that are also registered), and therefore only available after it has ConfigureServices has run
• The telemetry config is needed when configuring Serilog
• UseSerilog is configured before ConfigureServices is called

Related: serilog-contrib/serilog-sinks-applicationinsights#121

How can this be solved?

[JasonTheProgrammer]

If you're using services.Configure<TelemetryConfiguration>(..), as described in this issue regarding deprecation of TelemetryConfiguration.Active, then you can use services.PostConfigure<T>(..), loosely documented here.

So at a basic level, you could have something like this:

.ConfigureServices((context, services) =>
{
    string appInsightsKey = context.Configuration["APPINSIGHTS_INSTRUMENTATIONKEY"];

    services.Configure<TelemetryConfiguration>(tc =>
    {
        tc.InstrumentationKey = appInsightsKey;
    });

    services.PostConfigure<TelemetryConfiguration>(tc =>
    {
        Log.Logger = new LoggerConfiguration()
            .WriteTo.ApplicationInsights(tc, TelemetryConverter.Traces)
            .CreateLogger();
    });
})

[cmeeren]

Thanks! But aren't I then losing out on logging startup exceptions, as well as any custom setup performed by UseSerilog? Including this ConfigureServices stuff:

serilog-aspnetcore/src/Serilog.AspNetCore/SerilogWebHostBuilderExtensions.cs

Lines 139 to 158 in e6e14e6

	static void ConfigureServices(IServiceCollection collection, ILogger logger)
	{
	if (collection == null) throw new ArgumentNullException(nameof(collection));
	if (logger != null)
	{
	// This won't (and shouldn't) take ownership of the logger.
	collection.AddSingleton(logger);
	}
	// Registered to provide two services...
	var diagnosticContext = new DiagnosticContext(logger);
	// Consumed by e.g. middleware
	collection.AddSingleton(diagnosticContext);
	// Consumed by user code
	collection.AddSingleton<IDiagnosticContext>(diagnosticContext);
	}
	}

Case in point: If I call UseSerilog() (without args), and then set Log.Logger in ConfigureServices, then I lose e.g. the Microsoft start/end request messages. I can solve this by adding services.AddSingleton<ILogger>(Log.Logger) after setting Log.Logger, but issues such as this indicates to me that there might be more I may be missing when replacing Log.Logger after calling UseSerilog.

Also, it seems that using services.Configure<TelemetryConfiguration> makes you lose out on a lot of the default config you get with AddApplicationInsightsTelemetry, so I'm really hesitant to use anything but that method.

[cmeeren]

After checking library sources and experimenting for many hours today, here is a workaround I have arrived at which seems to work. Let me know if I'm shooting myself in the foot.

In doing this I have sacrificed logging early startup errors (in main or Startup.ConfigureServices up to and including where I configure the logger), but I am successfully logging "late" startup errors (e.g. in the Startup.Configure method) to Application Insights, and everything else seems to work as normal after (very) cursory testing.

Program.fs

[<EntryPoint>]
let main args =
  WebHost
    .CreateDefaultBuilder(args)
    .CaptureStartupErrors(true)
    .UseSerilog(dispose = true)
    .UseStartup<Startup>()
    .Build()
    .Run()
  0

Startup.fs

type Startup() =

  member __.ConfigureServices(services: IServiceCollection) : unit =
    services.AddApplicationInsightsTelemetry() |> ignore
    let sp = services.BuildServiceProvider()
    let telemetryConfig = sp.GetRequiredService<TelemetryConfiguration>()

    Log.Logger <-
      LoggerConfiguration()
        .WriteTo.ApplicationInsights(telemetryConfig, TelemetryConverter.Traces)
        .CreateLogger()

    services.AddSingleton<ILogger>(Log.Logger) |> ignore

[mikegoatly]

I can see how that would work - it feels very clunky though; building the service provider to configure something for the next time the service provider is built :)

I've got a feeling that calling IServiceCollection.BuildServiceProvider multiple times isn't great (the framework will call it for you once ConfigureServices exits unless you return a pre-built one) I can't remember where I read it - I'll do a bit of digging.

Edit: I suspect it'll be something to do with multiple singleton instance construction in addition to it being extra work at start-up time.

[mikegoatly]

Could you use a lambda ILogger registration instead? I'm unsure of the F# syntax, but in C# it would be:

services.AddSingleton<ILogger>(
    serviceProvider =>
    {
        Log.Logger = new LoggerConfiguration().WriteTo.ApplicationInsights(
                serviceProvider.GetRequiredService<TelemetryConfiguration>(),
                TelemetryConverter.Traces)
            .CreateLogger();

        return Log.Logger;
    });

[sungam3r]

Maybe related to #130

[mikegoatly]

@sungam3r Interesting - so I think that might make something like this possible? (untested)

WebHost.CreateDefaultBuilder(args)
                .UseSerilog((builder, hostingContext, loggerConfiguration) =>
                {
                    var config = loggerConfiguration
                        .ReadFrom.Configuration(hostingContext.Configuration)
                        .Enrich.FromLogContext();

                    builder.ConfigureServices(services => services.AddSingleton(
                        serviceProvider => 
                        {
                            cfg.WriteTo.ApplicationInsights(
                                serviceProvider.GetRequiredService<TelemetryConfiguration>(),
                                TelemetryConverter.Traces);
                            return config;
                        }));
                })
                .UseStartup<Startup>();

[sungam3r]

Yes, that's exactly how I wanted to use it. cfg.WriteTo should be changed to config.WriteTo.
Indeed, you may not immediately see the working solution because of the approach with multiple delegates. Fluent API is convenient, but it’s not always easy to figure out what place in the call chain needs to be delegated to and what can be used inside this delegate. In this case, we have delegates of triple nesting :).

[JasonTheProgrammer]

Regarding my earlier suggestion of using services.PostConfigure<TelemetryConfiguration>(..), I'm finding that callback does not get called when using the AspNetCore WebHostBuilder. I.e. WebHost.CreateDefaultBuilder() from Microsoft.AspNetCore v2.2.0.

But PostConfigure does get called if using generic host, i.e. HostBuilder from Microsoft.Extensions.Hosting v2.2.0. (I'm rather looking forward to the consolidation of Http workloads into the generic host in 3.0).

In the case of WebHost, I've found that if you're using a startup class, you can request an instance of TelemetryConfiguration in its constructor, and use it in the Configure method to configure the Serilog sink. As to whether that's early enough in the pipeline to meet your needs, that's up to you, but it's at least a pretty tidy option. In total, that would mean creating WebHost like:

public class Program
{
    public static void Main(string[] args)
    {
        CreateWebHostBuilder(args).Build().Run();
    }

    public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
        WebHost.CreateDefaultBuilder(args)
            .UseApplicationInsights() // Necessary
            .UseStartup<Startup>();
}

public class Startup
{
    private TelemetryConfiguration _telemetryConfiguration;

    public Startup(TelemetryConfiguration telemetryConfiguration)
    {
        _telemetryConfiguration = telemetryConfiguration;
    }

    public void Configure(IApplicationBuilder app, IHostingEnvironment env)
    {
        // Use _telemetryConfiguration as necessary here to configure Serilog sink
    }
}

EDIT: And now I've just discovered that IWebHostBuilder.UseApplicationInsights() was deprecated in v2.8.0. I'm getting woozy trying to follow all these changes! 🤪

[cmeeren]

Could you use a lambda ILogger registration instead?

I just tried your suggestion, but then I don't get anything logged at all.

EDIT: And now I've just discovered that IWebHostBuilder.UseApplicationInsights() was deprecated in v2.8.0. I'm getting woozy trying to follow all these changes! 🤪

Not sure if by the edit you intended to retract your previous suggestion:

In the case of WebHost, I've found that if you're using a startup class, you can request an instance of TelemetryConfiguration in its constructor, and use it in the Configure method to configure the Serilog sink.

But if you still stand by that: Wouldn't that mean that Serilog would use an unconfigured TelemetryConfiguration, since it gets injected (to Startup) before it's configured?

[JasonTheProgrammer]

In the case of WebHostBuilder in .NET Core 2.2, I've had good results injecting an instance of TelemetryConfiguration into the Startup class constructor. That was in conjunction with IServiceCollection.AddApplicationInsightsTelemetry(string instrumentationKey) from NuGet package Microsoft.ApplicationInsights.AspNetCore. And if custom ITelemetryInitializer implementations are needed, also configuring them in services. So, a complete example being:

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
    WebHost.CreateDefaultBuilder(args)
        .ConfigureServices((context, services) =>
        {
            services.AddSingleton<ITelemetryInitializer>(new MyCustomTelemInit());
            services.AddApplicationInsightsTelemetry(appInsightsKey);
        })
        .UseStartup<Startup>();

Then in Startup, take TelemetryConfiguration as a constructor parameter, and I configure logging in the class's ConfigureServices(IServiceCollection svc) implementation, like:

public void ConfigureServices(IServiceCollection services)
{
    // Your own Serilog logging config
    Logging.ConfigureSerilogLogging(Configuration, _telemetryConfiguration);
    services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
}

By this strategy, by the time the web app starts, App Insights and Serilog is fully configured. It does mean that any logging you'd happen to be trying to do earlier in the startup pipeline would be missed, but, I guess things get a bit trickier if you need that (would maybe need to brute-force configuration outside of the host builder).

Meanwhile, in generic host (at least in 2.2, haven't tried 3.0 yet), I got good results by registering a hosted service as a "logging initialiser" to run before my main service. Something a bit like this (not an exact copy-paste, so please forgive any gremlins in the code):

var builder = new HostBuilder()
    .ConfigureServices((context, services) =>
    {
        services.AddSingleton<ITelemetryInitializer>(new MyTelemInit());
        services.AddHostedService<SerilogInitialiserService>(); // A custom class
    })
    .AddWebJobs( /* config redacted /* ) // Your choice of HostBuilder service may differ
    .ConfigureLogging((context, log) =>
    {
        // Your necessary logging extension may differ.
        log.AddApplicationInsightsWebJobs(aiOptions =>
        {
            aiOptions.InstrumentationKey = aiKey;
            aiOptions.EnableLiveMetrics = false;
        });

        log.AddConsole();
    })
    .UseConsoleLifetime();

internal class SerilogInitialiserService : IHostedService
{
    private TelemetryConfiguration _telemetryConfiguration;
    private IConfiguration _configuration;

    public SerilogInitialiserService(
        TelemetryConfiguration telemetryConfiguration,
        IConfiguration configuration)
    {
        _telemetryConfiguration = telemetryConfiguration;
        _configuration = configuration;
    }

    public Task StartAsync(CancellationToken cancellationToken)
    {
        // Your choice of Serilog logging configuration...
        Logging.ConfigureSerilogLogging(_configuration, _telemetryConfiguration);
        return Task.CompletedTask;
    }

    public Task StopAsync(CancellationToken cancellationToken)
    {
        Log.CloseAndFlush();
        return Task.CompletedTask;
    }
}

[jorgeolive]

Hello,

I've been facing this problem as well. What are the drawbacks of forcing the InstrumentationKey in a generic ITelemetryInitializer class? Just like you would do to define the Cloud context properties.

Thanks,
Jorge.