zorroblue / distributed-search-engine Public

Notifications You must be signed in to change notification settings
Fork 4
Star 9

Distributed search engine using dynamic passive replication for load balancing

9 stars 4 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 67 Commits
data		data
protos		protos
.gitignore		.gitignore
CONTRIBUTING.md		CONTRIBUTING.md
README.md		README.md
clean.sh		clean.sh
client.py		client.py
crawler.py		crawler.py
environment.sh		environment.sh
master.py		master.py
masterbackup.py		masterbackup.py
replica.py		replica.py
replicas_list.txt		replicas_list.txt
requirements.txt		requirements.txt
utils.py		utils.py
writeservice.py		writeservice.py

Repository files navigation

Distributed search engine

Milestones:

Get client to receive response from the master
Prepare dummy data and similarity matrix
Get the backup system for master ready
Synchronize writes from crawler process to master and backup
Set up dynamic replica based on place and queries, forward new queries there
Shrink replicas in case they don't get new queries for a long time.
Make new replica in case dynamic replica fails