[CoreyLeath-code]

Summary

Adds a Terraform-managed CloudWatch observability layer on top of the AWS Lambda event-processing extension.

This is a stacked PR based on feat/aws-lambda-event-pipeline so the serverless implementation and observability changes remain independently reviewable.

What changed

• Add a CloudWatch dashboard for both Lambda functions and the processed-events SQS/DLQ path.
• Track Lambda invocations, errors, p95 duration, throttles, and concurrent executions.
• Track SQS visible backlog, oldest-message age, and DLQ depth.
• Add a CloudWatch Logs metric filter for record-level notification failures that can be hidden from the native Lambda Errors metric when using SQS partial-batch responses.
• Add operational alarms for Lambda errors, throttling, high p95 duration, queue lag, DLQ activity, and record-level notification failures.
• Add a dedicated operational-alerts SNS topic separate from product-facing ride notifications.
• Add an optional alarm_email Terraform variable; AWS subscription confirmation is still required before email delivery begins.
• Add configurable alarm thresholds for error count, Lambda p95 duration, and SQS oldest-message age.
• Expose dashboard name and operational alert topic ARN as Terraform outputs.
• Add a CloudWatch operations guide with alarm triage guidance and production-boundary notes.
• Update the AWS architecture documentation, README architecture, quick start, reliability section, evidence table, Q&A, and engineering roadmap.
• Update the changelog.

Design choices

• Reuse native AWS/Lambda and AWS/SQS metrics instead of duplicating standard telemetry with custom metrics.
• Add a custom log-derived metric only for record-level SQS notification failures.
• Separate operator alerting from rider/driver notification fan-out.
• Treat default thresholds as development/reference values rather than production SLOs.
• Keep Terraform statically valid without requiring AWS credentials or a live MSK cluster.

Validation

Expected checks:

• Terraform fmt -check -recursive
• Terraform init -backend=false
• Terraform validate
• existing Lambda tests
• existing Black/Ruff/mypy/pytest/benchmark CI
• broker/load-balancer integration suites
• hygiene/security/release verification

Production boundary

This PR adds deployable monitoring and alerting resources, not evidence of a live production SRE program. A production deployment still needs measured SLIs/SLOs, paging severity and escalation policy, tracing, idempotent DLQ replay, PII-safe logging controls, cost budgets, and operational ownership.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: c6213e3b-38fe-4e09-9189-414df27fecab

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d0df3deaf3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Isolate the failure metric per deployment

When multiple stacks with different name_prefix values run in the same AWS account and region, every metric filter publishes into the identical namespace/name pair with no dimensions. CloudWatch therefore aggregates all environments into one time series, so a record failure in a development stack can trigger the production stack's alarm and appear on its dashboard. Include the deployment prefix in the metric identity or add a per-stack dimension, and use the same identity in the alarm and dashboard.

Useful? React with 👍 / 👎.