Description
We are using RedisCluster
with read_from_replicas=True to offload read operations from the master nodes to replicas. However, we have observed that even with this setting enabled, RedisCluster is still sending some read requests to master nodes instead of strictly using replicas.
Upon checking the source code, we found that the LoadBalancer class is responsible for distributing the requests across both master and replica nodes. This behavior is unexpected as the expectation with read_from_replicas=True
is that all read operations should strictly go to the replicas.
Impact on Our Setup
- We handle heavy read/write operations with 100Cr+ (10 billion) reads and writes daily.
- This issue increases load on master nodes, affecting both write and read latencies.
- Replicas are not being fully utilized as expected.
Expected Behavior
When read_from_replicas=True is set, all read requests should only go to replica nodes, and master nodes should be used strictly for write operations.
Observed Behavior
Reads are distributed between both master and replica nodes, likely due to how the LoadBalancer class assigns nodes.
This issue affects high-scale production workloads where strict read/write separation is necessary for performance optimization.