A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.

OpenSearch Data Prepper is a component of the OpenSearch project that accepts, filters, transforms, enriches, and routes data at scale.

Apache Spark examples exclusively in Java

Spark in Action, 2e - chapter 9 - Advanced ingestion: finding data sources and building your own

Parallel Streaming Transformation Loader

Biocaddie Data Processing Pipeline. A data ingestion pipeline that collects and transforms original metadata information to a unified metadata model, called DatA Tag Suite (DATS).

Receiving end of new worker to push data across DC boundaries

Microservice to ingest data from Replicate and push it into DAF. Warning: this repo is deprecated.

The sample ingests multiline gzipped files of popular books into postgres.

NLP enhancer plugin for Foundry-ES pipeline management system. The service that enhances elasticsearch functionality with NLP elements.

Performing DataBase Ingester

Welcome to the fascinating intersection of Web3, Artificial Intelligence (AI), Open Data Core (ODC), and Composable Enterprise Fabric - a nexus of modern technologies that are significantly reshaping the enterprise landscape

Ingestion tool for various database logs