Redis Cluster support

[byliu]

I can't use lua-resty-redis to connect the redis cluster, right ?
if not, it's there any plan about redis cluster ?
what should i do for that ? I really need the redis cluster support .

[byliu]

maybe i can do something for that :)

[agentzh]

@byliu It's a TODO. Patches welcome :)

[agentzh]

@byliu If you really want to work on that, you're recommended to first discuss your API design here first to avoid any back and forth in the future :) Thanks

[bungle]

I think this is duplicate of #43.

[byliu]

@agentzh @bungle , thank you for your reply,
@h4lflife and @zhduan commited the Redis cluster client long long ago, ref #45 and #60 , it works well in my test, but it has not be merged till now, why?
maybe the redis cluster support should be added to some projects like lua-resty-redis-connector ?

[bungle]

@byliu, I think that no one has reviewed those. Because it is a huge task, and because there seem to be some opinionated things what should and what should not be done in client. You see, the Redis cluster support is mostly a client side thing. The Redis server does just a little bit, but most of the cluster support should be done in cluster client. Compare this to other clustered DBs that practically have just small requirements on clients — e.g. you can use same client (sharing most of the code) to connect both the cluster and the non-cluster.

And there are a lot of things to consider when implementing Redis cluster client, please read this: redis/jedis#990 (comment).

And then read this:
http://redis.io/topics/cluster-spec

You may see that the document above says:

This document is a work in progress as it is continuously synchronized with the actual implementation of Redis

That means that there is no authoritative document that tells us how to implement this. That also means that most cluster client implementations are incompatible with each other. If I was going to implement this, I would try to follow jedis as much as possible to get even some compatibility at least with one other client that has already implemented this to some degree (they also say that it is work-in-progress still).

Now you see the scale of this problem. Would you like to do review on this? Add the documentation? Compare the implementation with other clients, like jedis? Because of magnitude of this work, it is not an easy task. Personally I don't know how well done, or fully implemented, or correct, and documented those two pull requests are, so it is just my gut feeling that they are not there yet (they probably implement the absolute minimum).

I think that you could start lua-resty-redis-cluster project and try to do it as well as you can (maybe take these two pull requests as a base). When it is mature enough, I think that could be included in lua-resty-redis core lib.

I hope that answers the question for "why". The answer for how is different:

1. wait until someone voluntarily creates one
2. try to find someone who does that as a paid work, and find the funding for it (you may announce it on OpenResty mailing list) — how much are you willing to pay for it (I think it could mean one week to one month to get it done correctly depending on who does it)?
3. use current implementations / pull-requests.
4. implement it on your own (the features that you need, or everything)

Btw. I would love to have this as well.

[standsun]

@byliu Base on lua-resty-redis I wrote a lua redis client that support redis cluster, hope can help you. standsun/rcluster.lua

[zhaoxx063]

@standsun the implementation of redis cluster support is very good！

[steve0511]

Here recommends our redis cluster client: https://github.com/steve0511/resty-redis-cluster
Supports most of redis cluster feature/action:

1. slots=>nodes mapping cache
2. Moved redirection/ASK redirection.(https://redis.io/topics/cluster-spec#redirection-and-resharding, must be supported by client. eg. Jedis will renew the slot cache and retry in case of Moved redirection, and send back ask request in case of ASK redirection)
3. re-sharding
4. failover
5. pipeline
6. hashtag
7. read also from slave

[albertXl]

how about this now? is it worked in cluster?

[cen1]

The fact that neither Redis Cluster nor Redis Sentinel are supported is really unfortunate and basically makes this client unusable for serious production workloads. What good is having highly scalable OpenResty instances in something like k8s if they all talk to the single Redis instance? That redis goes down, the whole setup is gone.

Are there any other OpenResty clients which do supported HA datastore which could in theory replace redis? I guess you could hack around using virtual IPs and keepalived failovers for Redis and/or mysql/postgres HA setups but these are old hacks not worthy of modern setups.