Demo of running AWS Lambda and AWS Kineses locally using Localstack

Pinterest's experiment analytics data pipeline which runs thousands of experiments per day and crunches billions of datapoints to provide valuable insights to improve the product.

An event-streaming data pipeline from app to data lake ready for modelling

AWS framework to stream, classify, index and store data

Developed an ETL pipeline for real-time ingestion of stock market data from the stock-market-data-manage.onrender.com API. Engineered the system to store data in Parquet format for optimized query processing and incorporated data quality checks to ensure accuracy prior to visualization.

Simple Consent System implementation using AWS services and Python 3.

클라우드(S3) 기반으로 데이터를 변환 압축해 API를 제공하는 데이터 파이프라인

Streaming data pipeline on aws, Tech session repository for hist

Recover AWS Kinesis Firehose errors by running the COPY command to each of the manifests file in the S3 errors folders

A production grade data pipeline that streams real time data of an online retailer website into AWS S3 data lake. The pipeline uses AWS Kinesis to receive real time streaming data and a lambda consumer which puts data into S3 data-lake.

Utility code for AWS Services. Sample use case could be querying data from KDS. This project is open to receive contribution.

AWS Kensis Destionation - Http Endpoint Demo

Big Data Lab | Stream Delivery AWS Kinesis Firehose

Python script that produces logs observed by the Amazon Kinesis Agent

Recovers Kinesis Firehose source records from S3, replaying them into the stream

Real-time Twitter sentiment analysis using AWS services. Captures tweets, analyzes sentiment with Comprehend, and stores results in DynamoDB for visualization and insights into trends.

A streaming data pipeline leveraging managed services from AWS & the serverless architecture

This sample application demonstrates Amazon SNS with AWS X-Ray active tracing

Shipping AWS RDS logs from s3 into a kinesis stream