Using the PropertyWebScraper gem to import listings

PropertyWebBuilder supports importing listing from other sources. One group of sources to import from are third party websites. I've implemented the functionality for this in a gem called property_web_scraper which needs to be imported and mounted.

If you are creating a new website from the heroku project it will just work. You simply need to go to the "import data" section of the admin panel where you will have the option to import from a url. Currently you will have to paste in the url of each property you would like to import. I will add functionality to crawl a website to import several properties at a time when I get some time. If you will find this useful and would like me to prioritise it, get in touch.

This functionality works for a limited number of websites. You can however add support for more websites by following the instructions here:

https://github.com/RealEstateWebTools/property_web_scraper/wiki/Add-support-for-another-portal

If you are not using my heroku project (ie. you are mounting the PropertyWebBuilder engine within your own rails app) then you have to go through some extra steps to enable that functionality:

1. Add property_web_scraper with this line in your gemfile:

gem 'property_web_scraper', github: 'RealEstateWebTools/property_web_scraper'

2. Mount it at /io by adding this to your routes.rb:

mount PropertyWebScraper::Engine => '/io/'

3. Copy the migrations:

rails property_web_scraper:install:migrations

4. Migrate:

rails db:migrate

5. Seed the database

rails property_web_scraper:db:seed

6. Add this line to your config/initializers/assets.rb file:

Rails.application.config.assets.precompile += %w( pwb/themes/default.css )