Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using pub/sub GetSubscriber() with RedisCacheConnectionPoolManager #266

Closed
wluke opened this issue May 13, 2020 · 8 comments
Closed

Using pub/sub GetSubscriber() with RedisCacheConnectionPoolManager #266

wluke opened this issue May 13, 2020 · 8 comments

Comments

@wluke
Copy link

wluke commented May 13, 2020

It seems that if a connection drops, the RedisCacheConnectionPoolManager then disposes of the ConnectionMultiplexor (not giving it a chance to reconnect and reattack the subscriptions).

Is this designed to work with the pub/sub and reconnect, and if so how should this be setup?

@imperugo
Copy link
Owner

Hi @wluke
this is an interesting point.
Right now the dispose of the connection is handled by this piece of code

switch (e.FailureType)
{
    case ConnectionFailureType.ConnectionDisposed:
    case ConnectionFailureType.InternalFailure:
    case ConnectionFailureType.SocketClosed:
    case ConnectionFailureType.SocketFailure:
    case ConnectionFailureType.UnableToConnect:
        {
            logger.LogError(e.Exception, "Redis connection error {0}.", e.FailureType);

            this.Invalidate();
            this.invalidateConnectionCallback();
            break;
        }
}

What you are saying is that the connection multiplexer is raising one of the events listened in that block of code, before to retry.
I need to investigate.

Do you have an easy step to reproduce the issue? Unit test or anything?

@imperugo
Copy link
Owner

imperugo commented May 14, 2020

Just another question?
Are you on ASP.NET Core? How do you access to IRedisDatabase? Are you using StackExchange.Redis.Extensions.AspNetCore ?

Let me know
.u

@wluke
Copy link
Author

wluke commented May 14, 2020

Hi @imperugo ,

Exactly! If the ConnectionMultiplexor is used separately if fires the connection failed and then connection restored, and the subscriptions remain.
However, as you indicated, as soon as that connection failed fires the when using the ConnectionPool it disposes the multiplexor.

In order to reproduce I've simply been restarting redis, which triggers the connection failed.

I would suggest being a little less aggressive with the disposing, as the connectionmultiplexor or generally pretty good at reconnecting itself.

I'm not using Core currently.

Thanks!

@imperugo
Copy link
Owner

imperugo commented May 14, 2020

Hi @wluke
I'm able to reproduce the problem, basically I identified 2 possible problems:

  1. Injecting IRedisDatabase directly using StackExchange.Redis.Extensions.AspNetCore (it is registered as singleton, so it doesn't take another available multiplexer)

  2. Do not dispose if there the multiplexer is retrying to re-connect (didn't find a way to understand that)

@imperugo
Copy link
Owner

Just found a way to handle this better (and I've to write a better documentation for this).
Basically when I get a disconnection event I don't dispose the multiplex anymore but I make it unavailable.
This means that you get another connection from the pool in case you need to access to redis.
When the connection comes back thanks to the reconnect policy I bring it again available.

If I get an error during the reconnect, I dispose it.

Does it make sense to you?

I'll send a package update within Sunday.

@wluke
Copy link
Author

wluke commented May 14, 2020

Sounds great!

How do you check for error during reconnect?

@imperugo
Copy link
Owner

imperugo commented May 14, 2020

there is a property into the args called FailureType. If it is equal to None the connection is ok, otherwise Dispose.

imperugo added a commit that referenced this issue May 18, 2020
@imperugo
Copy link
Owner

6.1.7 is out and should fix the problem.

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants