This repo demonstrates the development of a real-time data pipeline designed to ingest, process, and analyze stock market data. Using cutting-edge tools like Apache Kafka, PostgreSQL, and Python, the pipeline captures stock data in real-time and stores it in a robust data architecture, enabling timely analysis and insights.

Kafka Streams for Low Code Environments

FLaNK AI Weekly covering Apache NiFi, Apache Flink, Apache Kafka, Apache Spark, Apache Iceberg, Apache Ozone, Apache Pulsar, and more...

Demo Flink and Kafka project to show how to react on tracking events in real-time and trigger offer for customer engagement based on campaign configurations. The project also utilizes the Broadcast State Pattern in order to update the rules (campaigns) at runtime without restarting the project, using a dedicated, low-frequency, Kafka topic.

Apache Kafka for python distributed streaming and processing systems

Scottify is the official media services provider of Starfleet 🖖

create a streaming application as a simple real-time fraud detection system backed by Apache Kafka using a Python client.

Demo Scenarios discussed in my Red Hat Developers article series

Demo UI for the flink-real-time-crm project using Spring Boot with Thymeleaf. The Demo UI is able to send messages to and consume from the related Kafka topics.

A Distributed Systems project using kafka, spring boot and docker

Spring - Kafka - FLaNK Stack Air Quality

A reliable subsystem to distribute data across multiple datacenters using multiple languages (C/C++, .NET, JVM enabled languages) over multiple technologies (e.g. Apache Kafka, OpenDDS, etc)

Phone Bridge Project

This project focused on real-time stock market data streaming using Apache Kafka. The project demonstrates how to use Apache Kafka for capturing, processing and streaming stock market data efficiently. It is implemented entirely in Jupyter Notebook and deployed in Amazon EC2 instance and S3; Amazon Athena, Glue and crawler is used to consume data..

Buit Uber Data Architecture from scratch

A streaming pipeline for a data lakehouse ( Kafka / MongoDB / Apache Drill)

Services with nestjs standalone for gRPC, Kafka, BullMQ (BullJs) e RabbitMQ

Gregor is a Python program that produces fake data into an Apache Kafka topic for demo purposes

StreamFlake: Real-Time CDC Pipeline with Kafka and Snowflake