Data Engineering examples for Airflow, Prefect; dbt for BigQuery, Redshift, ClickHouse, Postgres, DuckDB; PySpark for Batch processing; Kafka for Stream processing
A simple demo showing how to use Ably and fastAPI to route messages into Kafka for stream processing
Current 2022 Confluent Keynote Demo covering Stream Designer, Stream Catalog, and Stream Sharing.
Free and simple way to interact with ksqlDB using UI
For recreational use. Just a playground of Kafka+Spark+MQTT+KSQLDB+others
Interactive ksqlDB command line client with autocompletion and syntax highlighting written in Python
Pythonic KSQL REST API - Next Gen.
Real-time Coinbase market data streaming pipeline with visualizations. Much appreciation to DataTalks.Club Data Engineering Zoom Camp: https://github.com/DataTalksClub/data-engineering-zoomcamp
Kafka Connect and kSQLDB with Oracle
Real-time crypto market intelligence: Redpanda → ksqlDB → Iceberg → Claude anomaly explainer. Exactly-once, ~US$40/month on GCP.
An example of a Kappa architecture solution for transaction fraud detection using Apache Kafka and Python
This project demonstrates practical strategies for detecting and mitigating Kafka Poison Pills. A malformed or problematic messages that can break consumers.
This project demonstrates a modern ETL (Extract, Transform, Load) streaming pipeline using various open-source technologies.
Real time fraud analysis using Kafka Streams
Kubernetes demo
This project demonstrates a real-time Change Data Capture pipeline. It captures data changes (Inserts, Updates, Deletes) from a MySQL source and streams them into PostgreSQL using Apache Kafka, Debezium, and Avro serialization.
Add a description, image, and links to the ksqldb topic page so that developers can more easily learn about it.
To associate your repository with the ksqldb topic, visit your repo's landing page and select "manage topics."