Skip to content

Commit

Permalink
Fix typo: Change replication to federation (donnemartin#418)
Browse files Browse the repository at this point in the history
  • Loading branch information
kevinliu23 authored Jul 4, 2020
1 parent 6020231 commit 5d4dac6
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion solutions/system_design/web_crawler/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -282,7 +282,7 @@ Some searches are very popular, while others are only executed once. Popular qu

Below are a few other optimizations to the **Crawling Service**:

* To handle the data size and request load, the **Reverse Index Service** and **Document Service** will likely need to make heavy use sharding and replication.
* To handle the data size and request load, the **Reverse Index Service** and **Document Service** will likely need to make heavy use sharding and federation.
* DNS lookup can be a bottleneck, the **Crawler Service** can keep its own DNS lookup that is refreshed periodically
* The **Crawler Service** can improve performance and reduce memory usage by keeping many open connections at a time, referred to as [connection pooling](https://en.wikipedia.org/wiki/Connection_pool)
* Switching to [UDP](https://github.com/donnemartin/system-design-primer#user-datagram-protocol-udp) could also boost performance
Expand Down

0 comments on commit 5d4dac6

Please sign in to comment.