Skip to content

Commit 6a00ec6

Browse files
ptyadanaptyadana
committed
Steam scraping project
1 parent 6060205 commit 6a00ec6

16 files changed

+8474
-19455
lines changed

‎06.Project Scraping - Rotten Tomatoes/Rotten_tomatoes_page_2_HTML_Parser.html

Lines changed: 956 additions & 956 deletions
Large diffs are not rendered by default.

‎06.Project Scraping - Rotten Tomatoes/Rotten_tomatoes_page_2_LXML_Parser.html

Lines changed: 954 additions & 954 deletions
Large diffs are not rendered by default.

‎06.Project Scraping - Rotten Tomatoes/Scraper_RottenTomatoes.ipynb

Lines changed: 113 additions & 116 deletions
Large diffs are not rendered by default.

‎06.Project Scraping - Rotten Tomatoes/movies_info.xlsx

0 Bytes
Binary file not shown.

‎08.Scraping Steam Project/New_Trending_Games_Info.csv

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
Title,Price,Tags
2+
Dreamscaper: Prologue,Free,"Action, Indie, RPG, Free to Play"
3+
RESIDENT EVIL 3,$59.99,"Action, Zombies, Horror, Survival Horror"
4+
ONE PIECE: PIRATE WARRIORS 4,$59.99,"Action, Anime, Co-op, Online Co-Op"
5+
Eternal Radiance,$16.19,"Action, Adventure, RPG, Anime"
6+
Deadside,$19.99,"Massively Multiplayer, Action, Adventure, Indie"
7+
Conqueror's Blade,Free to Play,"Strategy, Massively Multiplayer, Action, Simulation"
8+
Borderlands 3,$59.99,"RPG, Action, Online Co-Op, Looter Shooter"
9+
Granblue Fantasy: Versus,$59.99,"Action, Anime, Fighting, 2D Fighter"
10+
Receiver 2,$17.99,"Simulation, Indie, Action, Shooter"
11+
Rakion Chaos Force,Free,"Action, RPG, Free to Play, Strategy"
12+
Mount & Blade II: Bannerlord,$49.99,"Early Access, Medieval, Strategy, Open World"
13+
Half-Life: Alyx,$59.99,"Masterpiece, Action, VR, Adventure"
14+
Last Oasis,$29.99,"Massively Multiplayer, Survival, Action, Adventure"
15+
DOOM Eternal,$59.99,"Action, Masterpiece, Great Soundtrack, FPS"
16+
Disaster Report 4: Summer Memories,$59.99,"Adventure, Action, Survival, VR"

‎08.Scraping Steam Project/Scraper Steam - My Work.ipynb

Lines changed: 1582 additions & 0 deletions
Large diffs are not rendered by default.

‎08.Scraping Steam Project/Section 8 - Scraping Steam - Setup.ipynb

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Extracting data from Steam "
8+
]
9+
},
10+
{
11+
"cell_type": "markdown",
12+
"metadata": {},
13+
"source": [
14+
"## Initial Setup"
15+
]
16+
},
17+
{
18+
"cell_type": "code",
19+
"execution_count": null,
20+
"metadata": {},
21+
"outputs": [],
22+
"source": [
23+
"from bs4 import BeautifulSoup\n",
24+
"import requests"
25+
]
26+
},
27+
{
28+
"cell_type": "markdown",
29+
"metadata": {},
30+
"source": [
31+
"## Connect to Steam webpage"
32+
]
33+
},
34+
{
35+
"cell_type": "code",
36+
"execution_count": null,
37+
"metadata": {},
38+
"outputs": [],
39+
"source": [
40+
"r = requests.get(\"https://store.steampowered.com/tags/en/Action/\")\n",
41+
"r.status_code"
42+
]
43+
},
44+
{
45+
"cell_type": "code",
46+
"execution_count": null,
47+
"metadata": {},
48+
"outputs": [],
49+
"source": [
50+
"html = r.content"
51+
]
52+
},
53+
{
54+
"cell_type": "code",
55+
"execution_count": null,
56+
"metadata": {},
57+
"outputs": [],
58+
"source": [
59+
"soup = BeautifulSoup(html, \"lxml\")"
60+
]
61+
},
62+
{
63+
"cell_type": "code",
64+
"execution_count": null,
65+
"metadata": {},
66+
"outputs": [],
67+
"source": []
68+
},
69+
{
70+
"cell_type": "markdown",
71+
"metadata": {},
72+
"source": [
73+
"## What can we scrape from this webpage?\n",
74+
"## 1) Try extracting the names of the top games from this page.\n",
75+
"## 2) What tags contain the prices? Can you extract the price information?\n",
76+
"## 3) Get all of the header tags on the page\n",
77+
"## 4) Can you get the text from each span tag with class equal to \"top_tag\"?\n",
78+
"## 5) Under the \"Narrow by Tag\" section, there are a collection of tags (e.g. \"Indie\", \"Adventure\", etc.). Write code to return these tags.\n",
79+
"## 6) What else can be scraped from this webpage or others on the site?"
80+
]
81+
},
82+
{
83+
"cell_type": "markdown",
84+
"metadata": {},
85+
"source": [
86+
"## Now is your turn!"
87+
]
88+
},
89+
{
90+
"cell_type": "code",
91+
"execution_count": null,
92+
"metadata": {},
93+
"outputs": [],
94+
"source": []
95+
},
96+
{
97+
"cell_type": "code",
98+
"execution_count": null,
99+
"metadata": {},
100+
"outputs": [],
101+
"source": []
102+
},
103+
{
104+
"cell_type": "code",
105+
"execution_count": null,
106+
"metadata": {},
107+
"outputs": [],
108+
"source": []
109+
},
110+
{
111+
"cell_type": "code",
112+
"execution_count": null,
113+
"metadata": {},
114+
"outputs": [],
115+
"source": []
116+
},
117+
{
118+
"cell_type": "code",
119+
"execution_count": null,
120+
"metadata": {},
121+
"outputs": [],
122+
"source": []
123+
}
124+
],
125+
"metadata": {
126+
"kernelspec": {
127+
"display_name": "Python 3",
128+
"language": "python",
129+
"name": "python3"
130+
},
131+
"language_info": {
132+
"codemirror_mode": {
133+
"name": "ipython",
134+
"version": 3
135+
},
136+
"file_extension": ".py",
137+
"mimetype": "text/x-python",
138+
"name": "python",
139+
"nbconvert_exporter": "python",
140+
"pygments_lexer": "ipython3",
141+
"version": "3.7.3"
142+
}
143+
},
144+
"nbformat": 4,
145+
"nbformat_minor": 2
146+
}

‎08.Scraping Steam Project/Top_Rated_Games.info.csv

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
Title,Price,Tags
2+
Counter-Strike: Global Offensive,Free to Play,"FPS, Shooter, Multiplayer, Competitive"
3+
Tom Clancy's Rainbow Six® Siege,$19.99,"FPS, Hero Shooter, Multiplayer, Tactical"
4+
Warframe,Free to Play,"Looter Shooter, Free to Play, Action, Co-op"
5+
Left 4 Dead 2,$9.99,"Zombies, Co-op, FPS, Multiplayer"
6+
Counter-Strike,$9.99,"Action, FPS, Multiplayer, Shooter"
7+
Borderlands 2,$19.99,"Loot, Shooter, Action, Multiplayer"
8+
Tomb Raider,$19.99,"Adventure, Action, Female Protagonist, Third Person"
9+
PAYDAY 2,$9.99,"Co-op, Action, FPS, Heist"
10+
Counter-Strike: Source,$9.99,"Shooter, Action, FPS, Multiplayer"
11+
Destiny 2,Free To Play,"Free to Play, Looter Shooter, FPS, Multiplayer"
12+
Half-Life 2,$9.99,"FPS, Action, Sci-fi, Classic"
13+
BioShock Infinite,$29.99,"FPS, Story Rich, Action, Singleplayer"
14+
Mount & Blade: Warband,$19.99,"Medieval, RPG, Open World, Strategy"
15+
Risk of Rain 2,$19.99,"Third-Person Shooter, Action Roguelike, Action, Multiplayer"
16+
MONSTER HUNTER: WORLD,$29.99,"Co-op, Multiplayer, Action, Open World"

‎08.Scraping Steam Project/Top_Sellers_Games_info.csv

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
Title,Price,Tags
2+
Counter-Strike: Global Offensive,Free to Play,"FPS, Shooter, Multiplayer, Competitive"
3+
Tom Clancy's Rainbow Six® Siege,$19.99,"FPS, Hero Shooter, Multiplayer, Tactical"
4+
Warframe,Free to Play,"Looter Shooter, Free to Play, Action, Co-op"
5+
Left 4 Dead 2,$9.99,"Zombies, Co-op, FPS, Multiplayer"
6+
Counter-Strike,$9.99,"Action, FPS, Multiplayer, Shooter"
7+
Borderlands 2,$19.99,"Loot, Shooter, Action, Multiplayer"
8+
Tomb Raider,$19.99,"Adventure, Action, Female Protagonist, Third Person"
9+
PAYDAY 2,$9.99,"Co-op, Action, FPS, Heist"
10+
Counter-Strike: Source,$9.99,"Shooter, Action, FPS, Multiplayer"
11+
Destiny 2,Free To Play,"Free to Play, Looter Shooter, FPS, Multiplayer"
12+
Half-Life 2,$9.99,"FPS, Action, Sci-fi, Classic"
13+
BioShock Infinite,$29.99,"FPS, Story Rich, Action, Singleplayer"
14+
Mount & Blade: Warband,$19.99,"Medieval, RPG, Open World, Strategy"
15+
Risk of Rain 2,$19.99,"Third-Person Shooter, Action Roguelike, Action, Multiplayer"
16+
MONSTER HUNTER: WORLD,$29.99,"Co-op, Multiplayer, Action, Open World"

‎08.Scraping Steam Project/Trending_Games_info.csv

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
Title,Price,Tags
2+
Counter-Strike: Global Offensive,Free to Play,"FPS, Shooter, Multiplayer, Competitive"
3+
Tom Clancy's Rainbow Six® Siege,$19.99,"FPS, Hero Shooter, Multiplayer, Tactical"
4+
Warframe,Free to Play,"Looter Shooter, Free to Play, Action, Co-op"
5+
Left 4 Dead 2,$9.99,"Zombies, Co-op, FPS, Multiplayer"
6+
Counter-Strike,$9.99,"Action, FPS, Multiplayer, Shooter"
7+
Borderlands 2,$19.99,"Loot, Shooter, Action, Multiplayer"
8+
Tomb Raider,$19.99,"Adventure, Action, Female Protagonist, Third Person"
9+
PAYDAY 2,$9.99,"Co-op, Action, FPS, Heist"
10+
Counter-Strike: Source,$9.99,"Shooter, Action, FPS, Multiplayer"
11+
Destiny 2,Free To Play,"Free to Play, Looter Shooter, FPS, Multiplayer"
12+
Half-Life 2,$9.99,"FPS, Action, Sci-fi, Classic"
13+
BioShock Infinite,$29.99,"FPS, Story Rich, Action, Singleplayer"
14+
Mount & Blade: Warband,$19.99,"Medieval, RPG, Open World, Strategy"
15+
Risk of Rain 2,$19.99,"Third-Person Shooter, Action Roguelike, Action, Multiplayer"
16+
MONSTER HUNTER: WORLD,$29.99,"Co-op, Multiplayer, Action, Open World"

0 commit comments

Comments
 (0)