A comprehensive serverless AWS solution for tracking and auditing IAM, STS, and AWS Console signin activities across all regions. Features real-time collection via free CloudTrail event history, advanced analytics with S3 + Athena, and automated security alerting. Built with AWS SAM, Lambda, DynamoDB, and optional Parquet analytics.
IAM Activity Tracker provides comprehensive monitoring of authentication and authorization activities across your AWS account, helping you:
- Track who created, modified, or deleted IAM resources
- Monitor role assumptions and credential usage
- Detect suspicious authentication patterns and failed login attempts
- Monitor AWS Console login activity including root account usage
- Build compliance reports and audit trails
- Maintain event data beyond CloudTrail's 90-day limit
- Real-time security alerts for suspicious activities
- Multi-Region Support: Automatically tracks IAM events (us-east-1), STS events (all regions), Console signin events (global), and SSO/Identity Center events
- Complete Coverage: Monitors IAM, STS, signin.amazonaws.com, and sso.amazonaws.com event sources
- SSO Tracking: Full support for AWS SSO/Identity Center administrative events (permission sets, account assignments, policy attachments)
- Incremental Processing: Only processes new events since last run with checkpoint management
- Initial Collection: First run collects up to 90 days of historical events from CloudTrail
- Parallel Processing: Uses multi-threading for fast regional queries (up to 32 concurrent threads)
- Real-time Storage: DynamoDB with GSI for immediate user and action queries
- Smart Filtering: Automatically filters out AWS service-linked role noise while preserving security-relevant events
- Serverless: No infrastructure to manage, fully event-driven architecture
- Security Alerts: Real-time SNS notifications for root usage, failed auth, privilege escalation, SSO admin actions, and more
- S3 Data Lake: Optional daily export to S3 in optimized Parquet format
- Athena Integration: Query years of data with standard SQL
- Pre-built Queries: 15 ready-to-use security and compliance queries (including 6 SSO-specific queries)
- Dual Output Format: All queries support both table format (curated fields, no truncation) and JSON format (complete data)
- Cost Optimization: S3 lifecycle policies automatically archive old data
- Query Tools: Python CLI for running analytics programmatically with flexible output options
- Cost-Effective: Operates within AWS free tier for most organizations
- Customizable Schedule: Configurable collection and export frequencies
- Comprehensive Monitoring: CloudWatch alarms for all functions
- Easy Deployment: Single-command SAM deployment
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ EventBridge │────▶│ Tracker Lambda │────▶│ DynamoDB │
│ (Hourly) │ │(Multi-threaded) │ │ (Events) │
└─────────────────┘ └──────────────────┘ └─────────────────┘
│ │
▼ ▼
┌──────────────────┐ ┌─────────────────┐
│ CloudTrail Event │ │Security Alerts │
│ History API │ │ (SNS) │
│ (Free 90 days) │ └─────────────────┘
└──────────────────┘ │
▼
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ EventBridge │────▶│ Export Lambda │ │ DynamoDB │
│ (Daily) │ │ (Parquet) │ │ (Control) │
└─────────────────┘ └──────────────────┘ └─────────────────┘
│
▼
┌──────────────────┐ ┌─────────────────┐
│ S3 Bucket │────▶│ Athena + Glue │
│ (Data Lake) │ │ (Analytics) │
└──────────────────┘ └─────────────────┘
│ │
▼ ▼
┌──────────────────┐ ┌─────────────────┐
│ Lifecycle Rules │ │ Query Tools │
│(CostOptimization)│ │ (Python) │
└──────────────────┘ └─────────────────┘
- Collection: Tracker Lambda queries CloudTrail every hour across all regions
- Storage: Events stored in DynamoDB for real-time queries
- Export: Export Lambda converts daily data to Parquet format in S3
- Analytics: Glue crawlers discover partitions, Athena enables SQL queries
- Optimization: S3 lifecycle rules archive old data to reduce costs
- AWS Account with appropriate permissions
- AWS CLI installed and configured
- SAM CLI installed (Installation Guide)
- Python 3.13 runtime support in your AWS region
- AWS SDK Pandas Layer for Python 3.13 (automatically configured in template)
To deploy and operate the IAM Activity Tracker, your AWS user/role needs the following permissions:
- CloudFormation: Full access to create, update, and delete stacks
- Lambda: Create, update functions and execution roles
- DynamoDB: Create tables, indexes, and manage data
- S3: Create buckets and manage objects (if analytics enabled)
- EventBridge: Create and manage rules
- CloudWatch: Create log groups and alarms
- IAM: Create and manage service roles and policies
- Glue: Create databases, tables, and crawlers (if analytics enabled)
For deployment, you can use the AWS managed policy PowerUserAccess or create a custom policy with these specific permissions.
git clone https://github.com/TocConsulting/iam-activity-tracker
cd iam-activity-trackermake deployThe deployment will:
- Build the Lambda function
- Create an S3 bucket for deployment artifacts (if needed)
- Deploy the CloudFormation stack
- Set up all required resources
- Automatically offer to initialize the system (recommended)
Immediate Initialization After deployment, the system will ask if you want to initialize immediately. This will:
- Collect up to 90 days of historical CloudTrail events (1-5 minutes)
- Export data to S3 in Parquet format (if analytics enabled)
- Set up Athena tables automatically
- Run Glue crawler to discover partitions
Without initialization, you would need to wait 25+ hours before analytics are ready!
make helpThis shows all available commands including deployment, queries, and utilities.
export AWS_REGION=us-east-1
export AWS_PROFILE=production
make deploy
# Choose 'Y' when prompted for initializationThis provides the best user experience with immediate access to analytics.
make deploy
# Choose 'n' when prompted for initializationUse this if you want to wait for scheduled collection (1 hour + 24 hours).
make initRun this anytime after deployment to collect historical data and set up analytics.
By default, the tracker filters out AWS service-linked role events to reduce noise and focus on security-relevant activities.
Events are filtered based on CloudTrail's userIdentity.type field:
{
"userIdentity": {
"type": "AWSService",
"invokedBy": "elasticloadbalancing.amazonaws.com"
}
}AWS services generate thousands of routine operational events that create noise in security monitoring:
- Volume: Can be 80-90% of total IAM/STS events
- Operational: Not indicative of user or application security activities
- Cost: Increases DynamoDB storage and processing costs
- Alert fatigue: Makes it harder to spot actual security incidents
The tracker supports filtering out specific roles that generate excessive noise, particularly useful for Cloud Security Posture Management (CSPM) tools and security scanners.
Filter out roles used by:
- CSPM Tools: PrismaCloud, Wiz, Orca, Dome9, CloudHealth
- Security Scanners: Qualys, Rapid7, Tenable
- Compliance Tools: AWS Config rules, custom compliance checkers
- Monitoring Tools: DataDog, New Relic, Splunk collectors
Set the FilteredRoles parameter during deployment or update:
# During deployment
sam deploy --parameter-overrides FilteredRoles="PrismaCloudRole,WizSecurityRole,*Scanner*"
# Or export before deployment
export FILTERED_ROLES="PrismaCloud*,Wiz*,OrcaSecurityRole,*CSPM*"
make deployThe filter supports multiple pattern types:
- Exact role names:
SecurityAuditRole - Wildcards:
*SecurityScanner*,CSPM-*,*-audit-role - Full ARNs:
arn:aws:iam::123456789012:role/PrismaCloudRole - Multiple patterns: Comma-separated list
When a role matches your filter patterns:
- AssumeRole events for that role are not stored
- All subsequent actions by that assumed role session are filtered
- Events are counted but not stored in DynamoDB
- Filter metrics are included in Lambda execution logs
# Filter common CSPM tools
FilteredRoles="PrismaCloud*,WizSecurityRole,OrcaSecurityRole,Dome9-Connect"
# Filter by pattern
FilteredRoles="*SecurityScanner*,*CSPM*,*Compliance*"
# Filter specific ARNs
FilteredRoles="arn:aws:iam::123456789012:role/SecurityAudit,arn:aws:iam::123456789012:role/CloudHealth"
# Mixed patterns
FilteredRoles="PrismaCloud*,*Scanner*,arn:aws:iam::123456789012:role/SpecificRole"The Lambda response includes filtering statistics:
{
"total_events_processed": 1500,
"total_events_filtered": 8500, // Events filtered out
"execution_time_seconds": 45.2
}This helps you understand the impact of filtering and validate your patterns are working correctly.
| Variable | Default | Description |
|---|---|---|
STACK_NAME |
iam-activity-tracker | CloudFormation stack name |
AWS_REGION |
us-east-1 | Deployment region |
LOG_LEVEL |
INFO | Logging verbosity (DEBUG/INFO/WARNING/ERROR) |
| Variable | Default | Description |
|---|---|---|
SCHEDULE_EXPRESSION |
rate(1 hour) | How often to collect events |
MAX_WORKERS |
16 | Max concurrent threads for region processing |
PROCESS_IAM_EVENTS |
true | Track IAM events (us-east-1) |
PROCESS_STS_EVENTS |
true | Track STS events (all regions) |
PROCESS_SIGNIN_EVENTS |
true | Track AWS Console signin events (us-east-1) |
PROCESS_SSO_EVENTS |
true | Track AWS SSO/Identity Center events |
SSO_REGION |
us-east-1 | Primary region for SSO events |
FILTER_AWS_SERVICE_EVENTS |
true | Filter out AWS service-linked role events |
FILTERED_ROLES |
'' | Comma-separated list of role names/patterns to filter (e.g., PrismaCloud*,WizRole,*SecurityScanner*). Supports wildcards (*) |
| Variable | Default | Description |
|---|---|---|
ENABLE_ANALYTICS |
true | Enable S3 export and Athena integration |
EXPORT_SCHEDULE_EXPRESSION |
rate(1 day) | How often to export to S3 |
EXPORT_DAYS_BACK |
1 | Number of days to export per run |
| Variable | Default | Description |
|---|---|---|
ENABLE_SECURITY_ALERTS |
true | Enable security alerts via SNS |
ALERTS_EMAIL_ADDRESS |
'' | Email address for security alerts (set during deployment or leave empty to skip) |
OFF_HOURS_START |
22 | Start hour for off-hours alerts (24-hour format) |
OFF_HOURS_END |
6 | End hour for off-hours alerts (24-hour format) |
rate(1 hour)- Every hour (recommended for active environments)rate(6 hours)- Every 6 hours (cost-conscious option)rate(12 hours)- Twice daily (minimal active monitoring)rate(1 day)- Once daily (archive-focused)
rate(6 hours)- Every 6 hours (near real-time analytics)rate(12 hours)- Twice dailyrate(1 day)- Daily (recommended for most use cases)rate(7 days)- Weekly (cost-optimized for large datasets)
For small organizations (<100 users, <50 service roles):
- DynamoDB: $0 (within 25GB free storage, 25 RCU/WCU)
- Lambda: $0 (within 1M invocations, 400,000 GB-seconds)
- CloudTrail: $0 (using free 90-day event history API)
- CloudWatch: $0 (within free tier limits)
- Total: $0/month
- Core components: $0 (free tier)
- S3 storage: ~$0.50-2/month (depends on data retention)
- Athena queries: ~$0.10-1/month (depends on query frequency)
- Total: $0.60-3/month
- DynamoDB: $2-5/month (beyond free tier)
- Lambda: $0-2/month (increased execution time for exports)
- S3 storage: $2-8/month (more data, lifecycle optimization)
- Athena: $1-5/month (more frequent analytics)
- Total: $5-20/month
- DynamoDB: $10-25/month
- Lambda: $2-10/month
- S3 storage: $5-20/month (with lifecycle policies)
- Athena: $5-15/month
- Glue: $1-5/month (crawler executions)
- Total: $23-75/month
Note: Costs do not include SNS alerts (minimal, ~$0.10 per 1000 emails)
-
S3 Lifecycle Policies: Automatically move data to cheaper storage classes
- Standard (30 days) → Standard-IA (90 days) → Glacier (365 days) → Deep Archive
- Can reduce storage costs by 80-95% for older data
-
Parquet Format: 70-90% smaller than JSON, reducing storage and query costs
-
Partitioned Data: Athena only scans relevant partitions, reducing query costs
-
Conditional Deployment: Disable analytics if only real-time monitoring is needed
Perfect for recent data (last 30-90 days) and operational monitoring.
- Navigate to DynamoDB in AWS Console
- Select the events table (e.g.,
iam-activity-tracker-events) - Use Query or Scan with filters
# Query events by user
aws dynamodb query \
--table-name iam-activity-tracker-events \
--index-name user_name-index \
--key-condition-expression "user_name = :username" \
--expression-attribute-values '{":username":{"S":"alice.johnson"}}'
# Query events by action
aws dynamodb query \
--table-name iam-activity-tracker-events \
--index-name event_name-index \
--key-condition-expression "event_name = :eventname" \
--expression-attribute-values '{":eventname":{"S":"CreateUser"}}'Perfect for historical analysis, compliance reporting, and complex queries.
Option 1: Automatic (Recommended) The system will automatically set up Athena during deployment initialization.
Option 2: Manual Setup If you skipped initialization, run this after the first export:
make setup-athenaOption 3: Initialize Later If you deployed without initialization, you can run it anytime:
make init# List all available queries
make list-queries
# Run specific security analysis (table format - default)
make run-query Q=failed_auth # Failed authentication attempts
make run-query Q=root_usage # Root account activity
make run-query Q=off_hours # After-hours access
make run-query Q=active_users # Most active users
make run-query Q=permission_changes # IAM permission modifications
make run-query Q=role_assumptions # Role usage patterns
make run-query Q=daily_summary # Compliance reporting
# SSO/Identity Center queries
make run-query Q=sso_permission_sets # SSO permission set changes
make run-query Q=sso_account_assignments # Account access grants
make run-query Q=sso_admin_policies # Admin policy attachments
make run-query Q=sso_admin_users # SSO administrators
make run-query Q=sso_activity_summary # SSO usage overview
# JSON format for complete data (ALL queries support both formats!)
make run-query Q=failed_auth FORMAT=json # Any IAM query
make run-query Q=root_usage FORMAT=json
make run-query Q=sso_account_assignments FORMAT=json # Any SSO query
make run-query Q=sso_admin_policies FORMAT=json
make run-query Q=active_users FORMAT=json # Any analytics queryALL queries support two output formats:
- Table Format (default): Displays curated fields optimized to prevent truncation. Perfect for quick analysis and reporting.
- JSON Format: Returns complete data with all available fields. Ideal for detailed investigations or data export.
# Examples - Every query works with both formats
make run-query Q=user_lookup # Table format (default)
make run-query Q=user_lookup FORMAT=table # Explicit table format
make run-query Q=user_lookup FORMAT=json # JSON format with all fields
make run-query Q=sso_admin_policies # Table format (default)
make run-query Q=sso_admin_policies FORMAT=json # Complete dataFor users who prefer AWS Athena console directly, all queries are available as ready-to-use SQL:
# View all queries in SQL format
cat queries/analytics_queries.sql
# Copy/paste any query directly into Athena console
# Queries use default table: iam_activity_tracker_database.iam_eventsThe queries/analytics_queries.sql file contains all 15 queries in standard SQL format, synchronized with the Python implementation. Perfect for:
- Custom modifications: Edit queries for specific requirements
- Learning: Understand the SQL behind each security analysis
- Integration: Use with other tools beyond the CLI
Example Query Output:
-- Recent failed authentication attempts
SELECT
substr(event_time, 1, 10) as event_date,
user_name,
source_ip,
event_name,
error_code,
COUNT(*) as failure_count
FROM iam_activity_tracker_database.iam_events
WHERE
substr(event_time, 1, 10) >= cast(current_date - INTERVAL '7' DAY as varchar)
AND error_code IS NOT NULL AND error_code != ''
AND event_name IN ('ConsoleLogin', 'AssumeRole', 'GetSessionToken')
GROUP BY 1, 2, 3, 4, 5
ORDER BY failure_count DESC;
-- Permission changes by user
SELECT
user_name,
event_name,
COUNT(*) as change_count,
MIN(substr(event_time, 1, 10)) as first_change,
MAX(substr(event_time, 1, 10)) as last_change
FROM iam_activity_tracker_database.iam_events
WHERE
substr(event_time, 1, 10) >= cast(current_date - INTERVAL '30' DAY as varchar)
AND event_name IN (
'AttachUserPolicy', 'DetachUserPolicy',
'AttachGroupPolicy', 'DetachGroupPolicy',
'AttachRolePolicy', 'DetachRolePolicy',
'CreateUser', 'DeleteUser',
'CreateRole', 'DeleteRole',
'PutUserPolicy', 'DeleteUserPolicy'
)
GROUP BY user_name, event_name
ORDER BY change_count DESC;
-- Role assumption patterns
SELECT
JSON_EXTRACT_SCALAR(request_parameters, '$.roleArn') as role_arn,
user_name,
COUNT(*) as assumption_count,
COUNT(CASE WHEN error_code IS NOT NULL AND error_code != '' THEN 1 END) as failed_count
FROM iam_activity_tracker_database.iam_events
WHERE
substr(event_time, 1, 10) >= cast(current_date - INTERVAL '30' DAY as varchar)
AND event_name = 'AssumeRole'
AND JSON_EXTRACT_SCALAR(request_parameters, '$.roleArn') IS NOT NULL
GROUP BY 1, 2
ORDER BY assumption_count DESC
LIMIT 20;The tool provides rich, formatted output for all queries with execution metrics:
| Feature | DynamoDB | Athena |
|---|---|---|
| Best for | Real-time, recent data | Historical analysis, complex queries |
| Time range | All data (no TTL by default) | All exported data |
| Query language | DynamoDB Query/Scan | Standard SQL |
| Performance | Milliseconds | Seconds to minutes |
| Cost | Free tier friendly | Pay per TB scanned |
| Complexity | Simple filters | Complex joins, aggregations |
The solution includes CloudWatch alarms for:
- Tracker Lambda: Errors and high execution duration (>4 minutes)
- Export Lambda: Errors and high execution duration (>13 minutes)
# View tracker Lambda logs with automatic formatting
make logs
# View specific log group
make logs | grep ERROR # Filter for errors
make logs | grep CRITICAL # Filter for critical alerts# Check stack status and outputs
make status
# Test a specific query
make run-query Q=failed_authBy default, events are kept indefinitely. To enable automatic deletion:
- Edit
template.yaml - Add TTL configuration to IAMEventsTable
- Redeploy the stack
# Pull latest changes
git pull
# Update deployment
make update-
"Access Denied" errors
- Ensure Lambda has CloudTrail read permissions in all regions
- Verify CloudTrail is enabled (check AWS Console → CloudTrail)
- Check IAM policy attached to tracker Lambda role
-
High tracker Lambda duration
- Reduce MAX_WORKERS if hitting API rate limits
- Consider increasing schedule interval to reduce frequency
- Check CloudWatch metrics for throttling
-
Missing events
- Verify PROCESS_IAM_EVENTS and PROCESS_STS_EVENTS are true
- Check CloudWatch Logs for region-specific errors
- Ensure all required AWS regions are accessible
-
No data in S3 bucket
- Check export Lambda logs for errors
- Verify ENABLE_ANALYTICS is set to true
- Ensure DynamoDB has data before export runs
- Check S3 bucket permissions
-
Athena queries return no results
- Run Glue crawler to discover new partitions:
aws glue start-crawler --name iam-activity-tracker-crawler
- Verify S3 data location in Athena table definition
- Check partition projection configuration
- Run Glue crawler to discover new partitions:
-
High Athena costs
- Use LIMIT in queries during testing
- Add date filters to reduce data scanned
- Check query execution plans
- Consider partition pruning
-
Export Lambda timeout
- Reduce EXPORT_DAYS_BACK to process fewer days per run
- Increase Lambda memory (currently 2048MB)
- Check if DynamoDB has throttling issues
-
Slow query performance
- Use appropriate indexes in DynamoDB
- Add partition filters in Athena queries
- Consider using columnar projection in Athena
-
Cost optimization
- Enable S3 lifecycle policies (automatically configured)
- Reduce query frequency if not needed real-time
- Use Athena workgroups to set query limits
To enable verbose logging, update the CloudFormation stack parameter:
- Set
LogLevelparameter toDEBUGduring deployment
# Check DynamoDB table
aws dynamodb describe-table --table-name iam-activity-tracker-events
# Check S3 bucket (if analytics enabled)
aws s3 ls s3://iam-activity-tracker-analytics-ACCOUNT_ID/iam-events/ --recursive | head -5
# Test a query
make run-query Q=daily_summary- Lambda function has read-only access to CloudTrail
- DynamoDB tables are encrypted at rest
- No credentials or sensitive data in logs
- Follows principle of least privilege
To remove all resources:
make destroyWarning: This will delete all collected event data!
Important: If analytics is enabled, manually empty the S3 buckets before running destroy:
aws s3 rm s3://iam-activity-tracker-analytics-ACCOUNT_ID/ --recursive
aws s3 rm s3://iam-activity-tracker-athena-results-ACCOUNT_ID/ --recursiveContributions are welcome! Please:
- Fork the repository
- Create a feature branch
- Submit a pull request
This project is licensed under the MIT License - see the LICENSE file for details.
For issues or questions:
- Check the troubleshooting section
- Review CloudWatch Logs
- Open an issue in the repository
user_lookup- User activity patternsfailed_auth- Failed authentication attemptsroot_usage- Root account activityoff_hours- After-hours access (10 PM - 6 AM)active_users- Most active userspermission_changes- IAM policy modificationsrole_assumptions- Role usage patternsdaily_summary- Daily activity summarieshourly_activity- Peak usage analysis
sso_permission_sets- Track SSO permission set creation and updatessso_account_assignments- Monitor who is getting access to which AWS accountssso_admin_policies- Track dangerous administrative policy attachmentssso_applications- Monitor third-party application integrationssso_admin_users- Identify users making SSO configuration changessso_activity_summary- SSO usage patterns and error rates
- Root account activity (login/failed)
- IAM user creation
- Admin policy attachments
- Dangerous inline policies
- Access key creation
- Role trust policy issues
- Access key updates
- MFA device changes
iam-activity-tracker/
├── assets/ # Screenshots and documentation
│ ├── Failed_Auth.png
│ └── List_Queries.png
├── functions/ # Lambda function code
│ ├── tracker/ # Real-time event collection
│ │ ├── handler.py # Main tracker Lambda
│ │ ├── cloudtrail_processor.py
│ │ ├── dynamodb_operations.py
│ │ ├── security_alerts.py # SNS alerting logic
│ │ └── requirements.txt
│ └── exporter/ # S3 analytics export
│ ├── export_handler.py # Export Lambda
│ ├── parquet_processor.py
│ ├── s3_operations.py
│ ├── dynamodb_operations.py
│ └── requirements.txt
├── queries/ # Analytics tools
│ ├── athena_utilities.py # Athena integration
│ ├── query_runner.py # CLI query tool
│ ├── analytics_queries.sql # Pre-built SQL queries
│ ├── setup.sh # Python environment setup
│ └── requirements.txt
├── scripts/ # Operational scripts
│ ├── deploy.sh # Deployment script
│ ├── destroy.sh # Cleanup script
│ ├── logs.sh # View Lambda logs
│ ├── run-query.sh # Query execution wrapper
│ ├── setup-athena.sh # Athena table setup
│ ├── status.sh # Stack status check
│ ├── test-alerts.sh # Test alert system
│ └── validate.sh # Template validation
├── template.yaml # SAM deployment template
├── Makefile # Build automation
├── Architecture.md # Detailed system design
└── README.md # This file
Built for the AWS security community