PetFinder Scrapy Crawler

This repository implements a spider for collecting PetFinder breeds sections.

This source code will be used as base for an medium article about web crawling using Scrapy framework.

Installation

pip install -r requirements.txt

Running petfinder spider

scrapy crawl pet_finder -o pet_finder.json

If you want to filter a specific logging level you can pass the flag -L LOG_LEVEL

scrapy crawl pet_finder -o pet_finder.json -L INFO