You can run and experiment with the code using free online resources, try executing this notebook by click the "launch" button above.
Or by going to this link
In this project I did following things:
- Used selenium to run the browser to open the steam website
- Navigated to topsellers page, scraped game titles for info like name,price,date of game title using beautiful soup
- Using requests to get page for each tiltle then scraped info like review,rating,description,tags,publisher,developer of game
- Saved all the data to a csv file
- Loading it back using pandas
- Sent the csv file using SMTP
- Added version control using git
- Created a github action to automate whole workflow from opening browser, scraping info, saving csv then sending it to email on scheduled time
To use this repo
- Fork this repo and add these repository secrets to recieve send/recieve emails using these same named keys, refer this on how to add repository secrets
- To run locally,make a virtual environment (recommended) then acitivate the env
- Clone the repo, navigate to it
- Using terminal/cmd run command
pip install -r requirements.txt - Then
python scrapper.py - Either way you can always go ahead and use the jupyter notebook which is very good way to experiment with the code.