ecommerce GCP Streaming pipeline ― Cloud Storage, Compute Engine, Pub/Sub, Dataflow, Apache Beam, BigQuery and Tableau; GCP Batch pipeline ― Cloud Storage, Dataproc, PySpark, Cloud Spanner and Tableau

Project for Cloud Computing course (A.Y. 2018/2019)

Hadoop Google DataProc DIO study

Real-time stream processing project using Apache Kafka and Spark Streaming on Google Cloud Dataproc. Includes Python producers/consumers, Spark DStream word count, and full deployment with screenshots.

Process large amount of data and implement complex data analyses using Spark. The dataset has been made available by Google. It includes data about a cluster of 12500 machines, and the activity on this cluster during 29 days.