Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional columns for data. I can see only 8 #23

Open
alandinbedia opened this issue Apr 23, 2020 · 4 comments
Open

Additional columns for data. I can see only 8 #23

alandinbedia opened this issue Apr 23, 2020 · 4 comments

Comments

@alandinbedia
Copy link

Hi, first of all, many thanks for this, loving the tool.
I would find very useful to get access to the columns sold price and year sold for properties for sale.

Is there any way I can do this?
Also, The address doesn't return the house number or the full post code, is there a way around this?

Many thanks

@osmya
Copy link

osmya commented Apr 24, 2020

@alandinbedia @toby-p

I believe that is easily achievable by changing some regex and data params in the init file
https://github.com/toby-p/rightmove_webscraper.py/blob/master/rightmove_webscraper/__init__.py

Regex

244       # Extract postcodes to a separate column:
. . .        pat = r"\b([A-Za-z][A-Za-z]?[0-9][0-9]?[A-Za-z]?)\b"
. . .        results["postcode"] = results["address"].astype(str).str.extract(pat, expand=True)

Additional data from xpath

175      # Create data lists from xpaths:
. . .        price_pcm = tree.xpath(xp_prices)
. . .        titles = tree.xpath(xp_titles)
. . .        addresses = tree.xpath(xp_addresses)

@alandinbedia
Copy link
Author

Great thanks
I will give it a try

@toby-p
Copy link
Owner

toby-p commented Apr 30, 2020

I originally set the regex to only get the first part of the postcode since this was all I could find available in the listings - if the full postcode is available on some listings then it would definitely be better to collect this as well if possible.

@p2327
Copy link

p2327 commented May 14, 2020

@alandinbedia @toby-p update for full postcode PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants