Meetuplytics

THAT'S A WORK IN PROGRESS: Realtime Meetup "Répondez, s'il vous plaît" (RSVPS) analytics built upon Apache-Beam (streaming processing). Using Cloud Pub/Sub for events ingestion/data circulation, Cloud Functions as a trigger + post-processing and Cloud Firestore as a storage.

About

Visualize trending topics from Meetup in real-time.

Implementation

Sample output

Updating the trending topics in real-time to Firestore

Run the pipeline locally :

Requires apache-beam[gcp] package and Python 2.7

You need to set up your owns Pub/Sub I/O topics and project.

Get Started with Python for Dataflow

$ python pipelines/rsvps-meetups-tpl.py

References:

• Streaming 101: The world beyond batch - Tyler Akidau
• Streaming 102: The world beyond batch - Tyler Akidau
• Triggers in Apache Beam - Kenneth Knowles
• Watermarks - Slava Chernyak
• Beam Documentation