- Use Python to develop a Twitter data crawler through Twitter's APIs to collect tweets data.
- Design the SQL server, database schema, tables and entity-relationship model.
- Perform sentiment analysis on tweet data.
We use Scweet which is an 3rd party tool for scraping data from Twitter.
We built ETL process by automating extraction of Twitter data, transform the data for analysis and loading the data to PostgreSQL database. End user can subscribe to receive daily sentiment anlysis visualization on a given topic.
Special thanks to https://github.com/Altimis/Scweet
- Cyrus Chan
- Frankie Lam
- Jimmy Ip
- Tommy To