-
Notifications
You must be signed in to change notification settings - Fork 23.7k
-
Notifications
You must be signed in to change notification settings - Fork 23.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[QUESTION] Redis Cluster with Node Aware or Rack Aware Replication #11306
Comments
We are discussing this issue as well: downstream issue. We don't have any production experience yet, but the following is the design we are discussing. It should be possible to achieve with an explicit "replicaset" topology. Redis Cluster and A "replicaset" consists of a primary and 2 replicas. These 3 nodes are in distinct zones. You then scale by adding more replicasets.
Since failovers only occur within a replicaset, and its members are in distinct zones, we ensure zonal redundancy. In Kubernetes this can be accomplished via one StatefulSet per replicaset, and pod anti-affinity applied to each of them. You'll want to configure your You'll need to take care when setting up the nodes to ensure they are connected in the right way. Hope that helps! |
I will share the notes soon. There are several ways in which this can be solved. |
Currently you will need to write a sidecar/daemon that runs alongside your Redis cluster that has awareness of your physical infrastructure. It would need to monitor the cluster to ensure that:
This will need to run continuously because even if you setup the cluster in a distributed way, after an automatic failover a Redis masters won't move back itself. Perhaps there should be a way Redis cluster could implement this itself? Maybe we should be able to configure nodes with an "anti-affinity" with other nodes, such that Redis will try to failover masters, and migrate replicas with more intelligence? |
@jacob-pro Note that most of these issues would be addressed by Redis Cluster v2: #10875. |
Out of curiosity, if we want to build our own Redis as a Service without using any Cloud Provider, how can we make sure that Master and Slave are not scheduled on the same failure domain? Especially if we are doing this in Kubernetes environment, do others use Operators in Kubernetes or any other solutions ?
Would love to hear from the community on this.
The text was updated successfully, but these errors were encountered: