Skip to content

Commit

Permalink
System design : Partitioning
Browse files Browse the repository at this point in the history
  • Loading branch information
josephjacobmorris committed Dec 11, 2024
1 parent 82b476f commit 3abfd1b
Showing 1 changed file with 16 additions and 1 deletion.
17 changes: 16 additions & 1 deletion SystemDesign/Scalability.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,26 @@ It gives scalability as well as resiliency. It is the preferred way for micro-se

Load Balancer is the software which handles distributing the traffic (incoming requests) across the sever cluster. It usually balances the traffic using consistent hashing algorithms. Nginx is one the most popular load balancer in the industry. Load Bqlancers should be fault tolerant and improves the availability. Consistent Hashing Algorithms accounts for machine failures and is easy to reduce/increase the instances , making it the most preferred load balancing algorithm.

#### Key range partition
data is partitioned based on range of keys.

**Cons**:
Uneven key distribution.

### Static Hash Partitioning
Data is partitioned using a hash function.

**Cons**:
* Not horizontally scalable
* data has to be rehashed when number of node changes


#### Consistent Hashing Algorithm

Normal Hashing Algorithms distribute data evenly but when the number of server changes (as in when server goes down or a new server is added) almost all data needs to be redistributed. In consistent Hashing Algorithm we consider a circular queue and the server are hashed and stored randomly. Then the keys (data) is also hashed and it is stored in
server whih comes next to the key in the clock-wise direction and hence in case the number of servers change only a fraction of data needs to be redistributed. Since the servers are randomly distributed across the circular queue data might not be evenly distributed across them. So to make the distribution more even we add in virtual nodes.

## References

* <https://www.youtube.com/watch?v=UF9Iqmg94tk&ab_channel=ByteByteGo>
* <https://www.youtube.com/watch?v=UF9Iqmg94tk&ab_channel=ByteByteGo>
* https://systemdesign.one/consistent-hashing-explained/

0 comments on commit 3abfd1b

Please sign in to comment.