Description
We're having issues, particularly when our server (IIS / ASP.Net 8.0) is under heavy load, where the creation of the ConnectionMultiplexer becomes a blocking thread for many other threads. Below is a screenshot from a memory dump of our production server showing that this line of code is blocking 1,700+ other threads.
We can seemingly get around this issue by preventing traffic from going to our server, starting our IIS site, requesting a page that interacts with Redis to almost "warm up" the Redis connection and then reenabling traffic. After that, everything seems to work fine, but if we have traffic going to the server out of the gate during the site's startup (such as after a new deployment), the creation of the multiplexer blocks about 50% of the time (sometimes it blocks, sometimes it doesn't... it all has to do with the traffic pattern).
I'm not sure if this is thread starvation or what. We're not using ConfigureAwait(false) since we're not on .Net Framework.
Here's our connection string configuration = {redactedhost:6379,syncTimeout=5000,allowAdmin=False,connectTimeout=5000,ssl=False,abortConnect=False,connectRetry=5}
We create a Singleton Multiplexer through this startup code
public static IServiceCollection AddRedisCacheService(this IServiceCollection services)
{
services.TryAddSingleton<ICache, RedisCache>();
// From AddStackExchangeRedisExtensions
services.TryAddSingleton<IRedisClientFactory, RedisClientFactory>();
services.TryAddSingleton<ISerializer>(new NewtonsoftSerializer(new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.Auto }));
services.TryAddSingleton<IRedisConnectionPoolManager, RedisConnectionPoolManager>();
services.TryAddSingleton((provider) => provider
.GetRequiredService<IRedisClientFactory>()
.GetDefaultRedisClient());
services.TryAddSingleton((provider) => provider
// Block path is through these calls
.GetRequiredService<IRedisClientFactory>()
.GetDefaultRedisClient()
.GetDefaultDatabase());
// Register a 'Configuration Factory' for extensions
services.AddSingleton<IEnumerable<RedisConfiguration>>(sp => [sp.GetRequiredService<RedisConfiguration>()]);
return services;
}