A simple Spark-powered ETL framework that just works 🍺

Extensible streaming ingestion pipeline on top of Apache Spark

Biopet docs

Pipeline Pattern implementation in Scala

Various data stream/batch process demo with Apache Scala Spark 🚀

Capabilities of StanfordNLP and OpenNLP on Spark

A real-time text classification based on Kafka and Spark.

🍦 Serve doddle-model in a pipeline implemented with Apache Beam

Spark ML Dashboard built to plug-in and tweak the model params to real-time verify classification results on sample test data

A simple in-memory, configuration driven, data processing pipeline for Apache Spark.

Experimental stream processing pipeline with anomaly detection.

a Scala pipeline library

reactive pipelines

Simple Journaling Queue Library

Daily Refresh of COVID-19 data integration resultset (Public s3 Bucket)

Recommendation system for the MovieLens database using Apache Spark

A simple pipeline to transform data within Azure Data Factory using Azure Databricks. Although it is written in Scala the same can be replicated in Python.

Maps reads back to an assembly, extracts mapped reads, and starts a new assembly.

GameTuner BigQuery Loader is application that loads enriched event to BigQuery