Apache Avro is a data serialization system.

Confluent Schema Registry for Kafka

Apache Kafka, Apache Flink and Confluent Platform examples and demos

Iceberg is a table format for large, slow-moving tabular data

MongoDB Kafka Connector

🔗 A multipurpose Kafka Connect connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka

Uber-project for standard Jackson binary format backends: avro, cbor, ion, protobuf, smile

A Gradle plugin to allow easily performing Java code generation for Apache Avro. It supports JSON schema declaration files, JSON protocol declaration files, and Avro IDL files.

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.

Divolte Collector

**Unofficial / Community** Kafka Connect MongoDB Sink Connector -> integrated 2019 into the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector

eCommerce application is a platform where customers can find products, shop around using a cart, check out the products and initiate payments. It is used for the didactic purpose only, as a support project for my training.

Project goal: Explore Kafka, Kafka Connect, and Kafka Streams. Components: store-api: Inserts/updates MySQL records. Source Connectors: Monitor MySQL changes, push messages to Kafka. Sink Connectors: Listen to Kafka, insert/update Elasticsearch docs. store-streams: Listens to Kafka, processes with Kafka Streams, pushes new messages to Kafka.

Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.

Wrangler Transform: A DMD system for transforming Big Data

Fluent Kafka Streams Test with Java

Practical examples with Apache Kafka®

Experiment with Kafka, Debezium, and ksqlDB. research-service: Performs MySQL record manipulation. Source Connectors: Monitor MySQL changes, push messages to Kafka. Sink Connectors and kafka-research-consumer: Listen to Kafka, insert/update Elasticsearch. ksqlDB-Server: Listens to Kafka, performs joins, and pushes new messages to new Kafka topics.

Tablasco is a JUnit rule for comparing tables and Spark module for comparing large data sets

JDK 11 examples and demo projects.