Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need to implement WAIT , tips ? #1474

Open
bklooste opened this issue May 27, 2020 · 9 comments
Open

Need to implement WAIT , tips ? #1474

bklooste opened this issue May 27, 2020 · 9 comments

Comments

@bklooste
Copy link

Have read the other thread but i need this anyway (not immediately but soon), don't need other blocking operations just WAIT

My options are

  • new ConnectionMultiplexer possibly in a separate domain , i see people have issues but try to disable thread steal and set higher sync time outs. Look at stalling code in stack exchange
  • Try get a connection from the lib and just send the commands.
  • fork stack exchange , make a change there.

Just after some tips / suggestions of the best approach.

@mgravell
Copy link
Collaborator

We know that WAIT is a problem. We have investigated some alternative internal implementations that will enable WAIT and other blocking operations - essentially, switching to a connection pool internally. This also resolves a number of other outstanding problems. The prototype was very successful - but we have not yet merged this work, because: it takes time, and this isn't our day jobs. I have a few other plates that I'm spinning at the moment, but this is very high on my list.

Blocking operations are anathema to the current implementation; there are no supported workarounds; you can hack them in via Execute etc, but: this is not advisory, and no support will be offered to make it work, because it cannot work reliably.

My advice: wait

@bklooste
Copy link
Author

Thank you much appreciated. I presume this will be a major release with a long cycle ?

@mgravell
Copy link
Collaborator

No, I was going to do this as 2.2, and the proof-of-concept is complete - it just needs integrating and testing, but: this isn't my only project and isn't my day job, so: it is tied on my availability. It is something I really want to get in, because it simplifies the IO path hugely, which should resolve a ton a network pain points.

@MikaelElkiaer
Copy link

Does this mean that there are currently no means of ensuring replication?

I am planning to set up master-replicant with sentinels. I suspect I have some data that I need to ensure is the same no matter which node you reach. It seems this is only possible with WAIT.

Is there anything going against an approach using a lua script to perform synchronous operations - i. e. using WAIT.

@mgravell
Copy link
Collaborator

mgravell commented Aug 19, 2021 via email

@MikaelElkiaer
Copy link

MikaelElkiaer commented Aug 20, 2021

That Lua script will block your entire server, and could even get forcibly
terminated. I don't recommend that.

Ok, if that is more or less ruled out, then what else is there left to do?
Even when sticking with DemandMaster, a failover will not be consistent either.
Is there then only going full cluster and manually write all masters, while keeping DemandMaster?

Is this a problem of client and/or server blocking? I'd assume that server blocking always would be possible despite any limitations in the client - what am I missing here?

Edit: I see now that WAIT only knows which writes to confirm based on the current connection, and I guess that in a multiplexer scenario it would be hard to discern which writes to confirm, unless it should be all of them. However, in the context of a transaction (i. e. MULTI), wouldn't the WAIT command only confirm the writes within that transaction?

@MikaelElkiaer
Copy link

@mgravell Sorry for bumping, but I was wondering if you could come with some clarification regarding the last bit of my previous post. Wouldn't it be possible to do WAIT within the context of a MULTI - even in a multiplexer scenario?

@mgravell
Copy link
Collaborator

Honestly, until I get the time to implement pooled connections: I am not, in any scenario, going to advocate anything that introduces a blockage at the server, no matter how transient. I can see many ways it go go badly. Are there scenarios where it might work acceptably under modest load? Possibly, but I'm not going to try and think through what the limits are.

@MikaelElkiaer
Copy link

Honestly, until I get the time to implement pooled connections: I am not, in any scenario, going to advocate anything that introduces a blockage at the server, no matter how transient. I can see many ways it go go badly. Are there scenarios where it might work acceptably under modest load? Possibly, but I'm not going to try and think through what the limits are.

Fair enough. Thanks for the reply, will keep an eye out for connection pooling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants