Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Busy loop at disconnect #66

Closed
immerhart opened this issue Apr 2, 2024 · 11 comments
Closed

Busy loop at disconnect #66

immerhart opened this issue Apr 2, 2024 · 11 comments

Comments

@immerhart
Copy link

immerhart commented Apr 2, 2024

Hey, first, thanks for the lib, much appreciated!

I did a connect to redis like this:

let redis_cli = Client::connect("127.0.0.1:6379").await;

which works just fine, if the redis server is running.
But if I kill redis (while still connected), I get error spamming in the console

2024-04-02T00:05:33.200878Z ERROR rustis::network::network_handler: [127.0.0.1:6379] Failed to reconnect: IO("[connection refused] Connection refused (os error 111)")    
2024-04-02T00:05:33.200896Z ERROR rustis::network::network_handler: [127.0.0.1:6379] Failed to reconnect: IO("[connection refused] Connection refused (os error 111)")    
2024-04-02T00:05:33.200914Z ERROR rustis::network::network_handler: [127.0.0.1:6379] Failed to reconnect: IO("[connection refused] Connection refused (os error 111)") 

Is there some way around this?
Not sure how the reconnect logic is for that case, but it seems like a busy loop?

@mcatanzariti
Copy link
Member

Thank for the report.
I will take a look at it.
I think the best thing is to add a reconnection timeout with a decent default value in the rustis config

@immerhart
Copy link
Author

immerhart commented Apr 2, 2024

Thanks for the quick response. I would suggest smth. like:
https://en.wikipedia.org/wiki/Exponential_backoff (Truncated version)
Or do you mean there is some config I forgot to set?

@mcatanzariti
Copy link
Member

I mean it has to be implemented!
And exponential back off is definitely something to consider

@immerhart
Copy link
Author

I mean it has to be implemented! And exponential back off is definitely something to consider

Hey, do you have any plans to add this soon?

@mcatanzariti
Copy link
Member

Hello, I'm working on it!

@immerhart
Copy link
Author

Hello, I'm working on it!

Cool, not yet so firm in rust, but I can at least do some manual testing... :)

@immerhart
Copy link
Author

Thanks, works like a charm!
But the "send while disconnected" (like calling e.g. get, if redis shutdown) behavior has changed, right? In the previous implementation it was returning an error, now it seems to block.
But that is controllable over the command_timeout...

@mcatanzariti
Copy link
Member

If you don't set the retry_on_error flag in the config or in the command, it should not block. Does it?

@immerhart
Copy link
Author

immerhart commented Apr 21, 2024

If you don't set the retry_on_error flag in the config or in the command, it should not block. Does it?

I didn't set that...
With this, it blocks for one second (as I set it with command_timeout) and after that returns the error:

   let mut c : rustis::client::Config = config.into_config().unwrap();
   c.reconnection = rustis::client::ReconnectionConfig::Exponential { 
       max_attempts: 0,
       min_delay: 100, 
       max_delay: 8000, 
       multiplicative_factor: 2, 
       jitter: 50
   };
   c.command_timeout = Duration::from_secs(1);

Without the command_timeout it seems to block/wait, till connection is established again (and the command is successful once established again).

@mcatanzariti
Copy link
Member

Ok, I'm going to take a look at it

@mcatanzariti
Copy link
Member

Should be fixed in 012a123

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants