Data Engineering examples for Airflow, Prefect; dbt for BigQuery, Redshift, ClickHouse, Postgres, DuckDB; PySpark for Batch processing; Kafka for Stream processing

A simple demo showing how to use Ably and fastAPI to route messages into Kafka for stream processing

Current 2022 Confluent Keynote Demo covering Stream Designer, Stream Catalog, and Stream Sharing.

Stream smartphone sensor data with FastAPI, Kafka, ksqlDB, and Docker.

Analyze Zeek IDS data with ksqlDB running on Confluent Platform via Docker on your laptop. Or spin up an arbitrary number of AWS hosts, each running Confluent Platform and ksqlDB for use in an instructor-led workshop.

For recreational use. Just a playground of Kafka+Spark+MQTT+KSQLDB+others

Interactive ksqlDB command line client with autocompletion and syntax highlighting written in Python

Free and simple way to interact with ksqlDB using UI

A Python wrapper for the KSQLdb REST API

Pythonic KSQL REST API - Next Gen.

An example of a Kappa architecture solution for transaction fraud detection using Apache Kafka and Python

Kafka Connect and kSQLDB with Oracle

Data streaming with Python and Apache Kafka

Real-time Coinbase market data streaming pipeline with visualizations. Much appreciation to DataTalks.Club Data Engineering Zoom Camp: https://github.com/DataTalksClub/data-engineering-zoomcamp

An app to keep track of Youtube videos and sends the notification to a Telegram bot to inform you if anyone comments on those

Kubernetes demo

Real time fraud analysis using Kafka Streams

Streaming event pipeline around Apache Kafka and its ecosystem, simulating Real-time Data Streaming

This project demonstrates a modern ETL (Extract, Transform, Load) streaming pipeline using various open-source technologies.

KSQLDB and Bokeh for realtime analytics dashboard