Export sales data from Google Sheet to a relational DBSM
The main purpose of this repository is to build the pipeline for training of regression models and predict the compressive strength of concrete to reduce the risk and cost involved in discarding the concrete structures when the concrete cube test fails.
Resource for ETL & Data Ingestion program using Apache Airflow
An application of my Centipede framework to watch 4Chan for any potentially threatening behavior.
This project implements a real-time data pipeline using Apache Airflow, Kafka, Apache Spark, and Delta Lake. It supports both batch (Coldpath) and real-time (Hotpath) data ingestion, processing, and storage. Airflow is used for orchestrating the data workflows.
Our knowledge system systematically ingests, processes, and indexes open-access life science publications. It supports internal research by providing precise question-answering and efficient retrieval from a continuously updated repository of scientific literature
Add a description, image, and links to the dataingestion topic page so that developers can more easily learn about it.
To associate your repository with the dataingestion topic, visit your repo's landing page and select "manage topics."