Skip to content

Commit

Permalink
Merge pull request #40 from axiomhq/INP-317-improve-docs
Browse files Browse the repository at this point in the history
docs: add explanations for each stack
  • Loading branch information
dasfmi authored Feb 16, 2024
2 parents a60c2cd + baf2488 commit ccd6c62
Show file tree
Hide file tree
Showing 3 changed files with 18 additions and 11 deletions.
21 changes: 14 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,28 @@
# Axiom CloudWatch Lambda [![CI](https://github.com/axiomhq/axiom-cloudwatch-lambda/actions/workflows/ci.yaml/badge.svg)](https://github.com/axiomhq/axiom-cloudwatch-lambda/actions/workflows/ci.yaml)

Easy to use AWS CloudFormation template to deploy a CloudWatch Log Group subscription filter and A lambda to push the
Easy to use AWS CloudFormation template to deploy a CloudWatch Log Group subscription filter and a Lambda to push the
logs from your CloudWatch to [Axiom](https://axiom.co).


Axiom’s CloudWatch Lambda is deployed using three distinct CloudFormation stacks. These stacks are responsible for the following:

1. Axiom Ingester: This stack creates a Lambda function that ingests logs from CloudWatch and sends them to Axiom.
2. Backfiller: This stack runs once to create subscription filters on the ingest Lambda, for all existing CloudWatch log groups.
3. LogsSubscriber: This stack creates a Lambda function that listens for new log groups and creates subscription filters for them. This way you don't have to create subscription filters manually for new log groups.

## Guide

1. Create an account at [Axiom Cloud](https://cloud.axiom.co)
1. Create an account at [Axiom](https://app.axiom.co)
2. Create a dataset and an API token with ingest permission for that dataset
3. Launch the stack: [![Launch Stack](https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?#/stacks/new?stackName=CloudWatch-Axiom&templateURL=https://axiom-cloudformation-stacks.s3.amazonaws.com/axiom-cloudwatch-lambda-cloudformation-stack.yaml)
4. Subscribe to more LogGroups: [![Launch Stack](https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?#/stacks/new?stackName=CloudWatch-Backfiller-Axiom&templateURL=https://axiom-cloudformation-stacks.s3.amazonaws.com/axiom-cloudwatch-backfiller-lambda-cloudformation-stack.yaml)
5. Automatically Subscribe to new LogGroups: [![Launch Stack](https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?#/stacks/new?stackName=Axiom-CloudWatch-LogsSubscriber&templateURL=https://axiom-cloudformation-stacks.s3.amazonaws.com/axiom-cloudwatch-logs-subscriber-cloudformation-stack.yaml)
4. Automatically subscribe to all existing log groups: [![Launch Stack](https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?#/stacks/new?stackName=CloudWatch-Backfiller-Axiom&templateURL=https://axiom-cloudformation-stacks.s3.amazonaws.com/axiom-cloudwatch-backfiller-lambda-cloudformation-stack.yaml)
5. Automatically Subscribe to new log groups: [![Launch Stack](https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?#/stacks/new?stackName=Axiom-CloudWatch-LogsSubscriber&templateURL=https://axiom-cloudformation-stacks.s3.amazonaws.com/axiom-cloudwatch-logs-subscriber-cloudformation-stack.yaml)


# Logs Subscriber architecture

- Creates an S3 bucket for Cloudtrail
- Creates a Trail to capture creation of new LogGroups
- Creates a Trail to capture creation of new log groups
- Creates an Event Rule to pass those creation events to event bus
- EventBridge sends an event to a Lambda function when a new LogGroup is created
- Lambda function creates a subscription filter for the new LogGroup
- EventBridge sends an event to a Lambda function when a new log group is created
- Lambda function creates a subscription filter for the new log group
4 changes: 2 additions & 2 deletions axiom-cloudwatch-lambda-cloudformation-stack.template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ Parameters:
AllowedPattern: "^(xaat-|xait-).*"
AxiomURL:
Type: String
Default: "https://cloud.axiom.co"
Default: "https://api.axiom.co"
AllowedPattern: ".+" # required
Description: The URL of Axiom endpoint (without trailing /). Defaults to "https://cloud.axiom.co".
Description: The URL of Axiom endpoint (without trailing /). Defaults to "https://api.axiom.co".
AxiomDataset:
Type: String
Description: The Name of the Dataset in Axiom to push events to.
Expand Down
4 changes: 2 additions & 2 deletions handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@


# push events to axiom
axiom_url = os.getenv("AXIOM_URL", "https://cloud.axiom.co").strip("/")
axiom_url = os.getenv("AXIOM_URL", "https://api.axiom.co").strip("/")
axiom_token = os.getenv("AXIOM_TOKEN")
axiom_dataset = os.getenv("AXIOM_DATASET")
disable_json = os.getenv("DISABLE_JSON", "false") == "true"
Expand Down Expand Up @@ -67,7 +67,7 @@ def push_events_to_axiom(events: list):
if len(events) == 0:
return

url = f"{axiom_url}/api/v1/datasets/{axiom_dataset}/ingest"
url = f"{axiom_url}/v1/datasets/{axiom_dataset}/ingest"
data = json.dumps(events)
req = urllib.request.Request(
url,
Expand Down

0 comments on commit ccd6c62

Please sign in to comment.