Amazon crawler that scrapes product information based on an inputted product search string from another retailer to provide price comparison data
The retailer being used to provide price comparison data is: http://www.jomashop.com/
This crawler uses Scrapy and uses Splash to render javascript on the given page. Follow the guides below to set them up in your local environment.
Splash runs in a docker container, see here for installation guide: Docker
For production or large scale crawling multiple splash servers will needed, the cookiecutter template aquarium comes in very handy for this.
Aquarium is a cookiecuter template for hassle-free `Docker Compose`_ + Splash setup. Think of it as a Splash instance with extra features and without common pitfalls.