Skip to content
/ rahnama_dot_com_scraper Public

A Simple Web Automation project to scrap and store real-estate ads in a Sqlite and ElasticSearch

telegram.me/hamshahri_ads
6 stars 2 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

jahan-paisley/rahnama_dot_com_scraper

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
config
config
 
 
data
data
 
 
lib
lib
 
 
output
output
 
 
.gitignore
.gitignore
 
 
.rbenv-gemsets
.rbenv-gemsets
 
 
Gemfile
Gemfile
 
 
Gemfile.lock
Gemfile.lock
 
 
README.md
README.md
 
 
rahnama.thor
rahnama.thor
 
 

Repository files navigation

http://rahnama.com scraper

A Simple Web Automation Script based on Capybara, RSpec and Thor.

How to run:

bundle install
chmod scrap_rahnama.thor a+x #make it executable
./scrap_rahnama.thor start

Output

Output will be save in a sqlite database

TODOS:

  • Save the logo and the filename of bmp file in ads and associate them with advertisers
  • Clean the text to remove the phones from ad body

About

A Simple Web Automation project to scrap and store real-estate ads in a Sqlite and ElasticSearch

telegram.me/hamshahri_ads

Topics

elasticsearch scraper open-data

Resources

Readme
Activity

Stars

6 stars

Watchers

2 watching

Forks

2 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages