Skip to content

Commit 04e3785

Browse files
author
Aboushady
committed
Adding a new Sample State Machine to Integrate Amazon AppFlow with AWS Glue
1 parent 376dc1f commit 04e3785

File tree

4 files changed

+495
-0
lines changed

4 files changed

+495
-0
lines changed
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Combining Amazon AppFlow with AWS Step Functions to maximize application integration benefits
2+
This is a sample application that uses [AWS Step Functions](https://aws.amazon.com/step-functions/) to integrate [Amazon AppFlow](https://aws.amazon.com/appflow/) and [AWS Glue](https://aws.amazon.com/glue/) catalog without the need of writing code. It automatically uses [Amazon EventBridge](https://aws.amazon.com/eventbridge/) to trigger AWS Step Functions every time a new Amazon AppFlow flow finished running.
3+
4+
Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
5+
6+
## Overview
7+
An everyday use case of Amazon AppFlow is creating a customer-360 by integrating marketing, customer support, and sales data. For example, analyze the revenue impact of different marketing channels by synchronizing the revenue data from Salesforce with marketing data from Adobe Marketo.
8+
The Amazon AppFlow flow pushes the data to [Amazon Simple Storage Service](https://aws.amazon.com/s3/) (S3) to help access the data using standard SQL language with [Amazon Athena](https://aws.amazon.com/athena/) or using [Amazon QuickSight](https://aws.amazon.com/quicksight/) for visualizations. In this context AWS Step Functions is used to automate a workflow to help catalog the AppFlow flow data using Amazon Glue everytime a new flow runs.
9+
10+
## State Machine Overview
11+
![StateMachine](./images/stepfunctions_graph.png)
12+
13+
### Prerequisites
14+
15+
You will need below items to successfully test and deploy:
16+
- [AWS CLI](https://aws.amazon.com/cli/)
17+
- [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)
18+
19+
20+
### How to deploy
21+
- Use `sam deploy -guided` to start the deployment guide
22+
* Enter a stack name
23+
* Enter the desired AWS Region
24+
* Enter required parameters such as the AWS Glue Database Name
25+
* Allow SAM CLI to create IAM roles with the required permissions.
26+
- Note the outputs from the SAM deployment process. These contain the resource names and/or ARNs
27+
28+
29+
### Testing
30+
- Create a new Amazon AppFlow flow that uses S3 as a destination
31+
- After the Amazon AppFlow flow finishes running, check the AWS Step Functions console for execution logs
32+
33+
34+
### Cleanup
35+
- Run `sam delete --stack-name <stack-name>` to delete the stack
36+
- Confirm deletion
243 KB
Loading

0 commit comments

Comments
 (0)