Skip to content

Zadigo/google_comments

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Comments scraper

Calling the spider

You can use four different types of endpoints to call one of the following spiders :

  • GooglePlaces
  • GooglePlace
  • SearchLinks
  • SearchBusinesses

Google places

This first enpoint will parse comments for a page with multiple businesses (e.g. Maison Chapon).

Google place

This endpoint will help parse a single Google place url (e.g. 18 Rue de Poissy, Maison Chapon)

You can either call the spider for a single url for a list of urls that you would like to have the comments for. In the latter case, you file needs to be called google_place_urls.csv and be placed in the /media/ folder of the app. The following example shows how to run this specific spider inline:

instance = GooglePlace(output_folder='my_folder_name')
instance.iterate_urls()

Search links

If you need to obtain Google Maps links for businesses, you need to first create a file called search_data.csv in which you need at least a column called data and eventually a column called address and/or zip code. A concatenation of these three values will create a search value that we will be using to automate searching Google Maps links for businesses.

Search business

Unless the above the automater which only returns links, this automater will return business information provided a given search. The mechanism for retrieving the data works in the same way as the above automater.

Calling a spider

To call place or places, do the following:

import subprocess

args = ['python', '-m', 'maps', 'places', 'https://.../google.com/maps/search/...']
subprocess.call(args)

or in a terminal:

python -m maps places https://.../google.com/maps/search/...

About

Scrapper to use Google comments

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published