Selenium-Webscraping-Example

Description

In this tutorial we will build a web scraping program that will scrape a Github user profile and get the Repository names and the Languages for the pinned repositories.

Resources

We will leverage these Resources

• Selenium
• ChromeDriver - WebDriver for Chrome
• Selenium-Python ReadTheDocs

We will also use the following packages and driver.

*• selenium package — used to automate web browser interaction from Python *• ChromeDriver — provides a platform to launch and perform tasks in specified browser. *• Virtualenv — to create an isolated Python environment for our project. *• Extras: Selenium-Python ReadTheDocs Resource.

Sequence

*• 1st import: Allows you to launch/initialise a browser.

*• 2nd import: Allows you to search for things using specific parameters.

*• 3rd import: Allows you to wait for a page to load.

*• 4th import: Specify what you are looking for on a specific page in order to determine that the webpage has loaded.

*• 5th import: Handling a timeout situation.