Tez Spider is a distributed music crawler that crawls 5 different music websites in parallel.
This is the server-side app running on golang's net/http package server.
Checkout the server-side repo for better explaination for how Spiders basically crawl 5 different websites in paralel here
- Golang 1.19
- net/http
- go-colly
- Docker
- Redis
To run the app in your own local machine, first of all, clone this repo to your local machine and make sure you have Docker installed on your machine. Once your machine is statisfied will all the requirements, you can spin-up the application by running the following command below:
$ docker-compose up --buildOnce the docker container is created and app is running, you can start the application by running the command below :
You will then be able to access it at localhost:9090
You can check out the full license here
This project is licensed under the terms of the MIT license.