This repository contains source code for the AWS Database Blog Post Reduce data archiving costs for compliance by automating RDS snapshot exports to Amazon S3
This repository has a collection of utilities for Glue Crawlers. These utilities come in the form of AWS CloudFormation templates or AWS CDK applications.
Automation framework to catalog AWS data sources using Glue
ETL Data pipeline using aws services
Smart City Realtime Data Engineering Project
Terraform configuration that creates several AWS services, uploads data in S3 and starts the Glue Crawler and Glue Job.
It is a project build using ETL(Extract, Transform, Load) pipeline using Spotify API on AWS.
An end-to-end data pipeline built with AWS S3, Glue, Crawler, Athena, Tableau visulization
AWS Athena, Glue Database, Glue Crawler and S3 buckets deployment through AWS GUI console.
Creating an audit table for a DynamoDB table using CloudTrail, Kinesis Data Stream, Lambda, S3, Glue and Athena and CloudFormation
Analyzed a multicategory e-commerce store using big data techniques on a Kaggle dataset with the help of AWS EC2, AWS S3, PySpark, AWS Glue ETL, AWS Athena, AWS CloudFormation, AWS Lambda and Power BI!
An end-to-end solution for managing and analyzing YouTube video data from Kaggle, leveraging AWS services and visualized through Quicksight and Tableau
AWS Athena, Glue Database, Glue Crawler and S3 buckets deployment through CloudFormation stack on AWS console.
This project automates an ETL pipeline using AWS Glue, S3, Athena, and Step Functions to transform raw Airbnb data. It cleanses, enriches, and organizes the data into separate raw and transformed databases, enabling efficient querying and analysis via Athena, with automated notifications through SNS.
Unlocking Spotify insights with an AWS data pipeline: S3 data lake -> Glue ETL-> Athena queries -> QuickSight Dashboard
In this project I have used the Trending YouTube Video Statistics data from Kaggle to analyze and prepare it for usage.
This project automates the extraction, transformation, and loading (ETL) of Reddit data into a Redshift data warehouse using Airflow. Key technologies include Celery, PostgreSQL, S3, Glue, Athena, and Redshift, providing a complete data pipeline solution.
An end-to-end solution for managing and analyzing YouTube video data from Kaggle, leveraging AWS services and visualized through Quicksight and Tableau
Unveiling job market trends with Scrapy and AWS
Add a description, image, and links to the aws-glue-crawler topic page so that developers can more easily learn about it.
To associate your repository with the aws-glue-crawler topic, visit your repo's landing page and select "manage topics."