HybridCache Preserve Activity in Factory Callback

[almarcus]

Description

When using HybridCache and there is a cache miss on GetOrCreateAsync, the factory method is called but the parent Activity is not preserved and the factory method call has an event that is emitted with a different Trace ID

Reproduction Steps

var result = await cache.GetOrCreateAsync(
    "my-key",
    async ct =>
    {
        return await dbContext.MyTable.ToListAsync(ct); // emits its own span with a different trace ID
    },
    cancellationToken: cancellationToken);

Expected behavior

Trace ID of the factory method should be the same as the parent activity. This screenshot shows the trace without the use of HybridCache:

Actual behavior

Trace ID is different than the parent activity

Regression?

Unsure

Known Workarounds

Setting the Activity.Current to the outer Activity.Current within the factory method works, but isn't feasible or reasonable to do everywhere and should work with the default implementation of HybridCache:

var parentActivity = Activity.Current; // get parent activity

await cache.GetOrCreateAsync(
    "my-key",
    async ct =>
    {
        Activity.Current = parentActivity; // set current activity to parent

        return await dbContext.MyTable.ToListAsync(ct);
    });

Configuration

.NET SDK:
 Version:           9.0.200
 Commit:            90e8b202f2
 Workload version:  9.0.200-manifests.a8e064d4
 MSBuild version:   17.13.8+cbc39bea8

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  15.5
 OS Platform: Darwin
 RID:         osx-arm64
 Base Path:   /usr/local/share/dotnet/sdk/9.0.200/

.NET workloads installed:
There are no installed workloads to display.
Configured to use loose manifests when installing new manifests.

Host:
  Version:      10.0.0-preview.4.25258.110
  Architecture: arm64
  Commit:       c22dcd0c7a

.NET SDKs installed:
  8.0.402 [/usr/local/share/dotnet/sdk]
  9.0.100-rc.2.24474.11 [/usr/local/share/dotnet/sdk]
  9.0.100 [/usr/local/share/dotnet/sdk]
  9.0.102 [/usr/local/share/dotnet/sdk]
  9.0.200 [/usr/local/share/dotnet/sdk]
  10.0.100-preview.3.25201.16 [/usr/local/share/dotnet/sdk]
  10.0.100-preview.4.25258.110 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0-rc.2.24474.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 10.0.0-preview.3.25172.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 10.0.0-preview.4.25258.110 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0-rc.2.24473.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 10.0.0-preview.3.25171.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 10.0.0-preview.4.25258.110 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Packages:

   > Microsoft.Extensions.Caching.Hybrid              9.6.0           9.6.0        
   > Microsoft.Extensions.Http.Resilience                   9.5.0           9.5.0        
   > OpenTelemetry.Exporter.OpenTelemetryProtocol           1.12.0          1.12.0       
   > OpenTelemetry.Extensions.Hosting                       1.12.0          1.12.0       
   > OpenTelemetry.Instrumentation.AspNetCore               1.12.0          1.12.0       
   > OpenTelemetry.Instrumentation.EntityFrameworkCore      1.12.0-beta.2   1.12.0-beta.2
   > OpenTelemetry.Instrumentation.Http                     1.12.0          1.12.0       
   > OpenTelemetry.Instrumentation.Runtime                  1.12.0          1.12.0       
   > Serilog.AspNetCore                                     9.0.0           9.0.0        
   > Serilog.Sinks.OpenTelemetry                            4.1.1           4.1.1      

Other information

No response

[KalleOlaviNiemitalo]

This is caused by the same ExecutionContext / AsyncLocal<T> propagation problem as #5568 and #5648.