Scrapy-RabbitMQ-Publisher allows you to save publish items to RabbitMQ.
Using pip, type in your command-line prompt
pip install scrapy-rabbitmq-publisher
Or clone the repo and inside the scrapy-rabbitmq-publisher directory, type
python setup.py install
Add following code to your scrapy settings.
RABBITMQ_HOST = "localhost"
RABBITMQ_PORT = 5672
RABBITMQ_USER = "guest"
RABBITMQ_PASSWORD = "guest"
RABBITMQ_VIRTUAL_HOST = "/"
RABBITMQ_EXCHANGE = "scrapy"
RABBITMQ_ROUTING_KEY = "item"
RABBITMQ_QUEUE = "item"
ITEM_PIPELINES = {
"scrapy_rabbitmq_publisher.pipelines.RabbitMQItemPublisherPipeline": 1,
}Copyright (c) 2019 Artem Rys - Released under The MIT License.