Skip to content

Collect client behavior data from around the world via HTTP(S) requests and store them in AWS S3 bucket.

License

Notifications You must be signed in to change notification settings

cowcoa/aws-data-collection

Repository files navigation

AWS Data Records Collection

alt text

Deployment Steps:

  1. Before you deploy this project, please check and adjust config variables in
    ./bootstrapping/config.sh
  2. Execute basic script to create S3 bucket, VPC, Subnet, IGW, ECR repository and other necessary AWS resources.
    ./bootstrapping/deploy_basic_template.sh create
  3. Execute image pipeline script to build a Fluent Bit/Nginx pre-installed/configed AMI.
    ./image_pipeline/deploy_image_pipeline.sh create
  4. Execute data ingest script to create Application Load Balancer, Kinesis Data Stream, Kinesis Firehose and other AWS resources.
    ./data_ingest/deploy_data_ingest.sh create
  5. Execute docker image script to build Lambda container image and push the image to ECR repository.
    ./data_consumer/lambda/deploy_image.sh
  6. Execute data consumer script to create Lambda function based on pre-builded image in step 4.
    ./data_consumer/deploy_data_consumer.sh create

About

Collect client behavior data from around the world via HTTP(S) requests and store them in AWS S3 bucket.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published