A production-ready webhook processing service built with NestJS that demonstrates enterprise-grade event-driven architecture patterns. This service handles external webhooks from multiple providers with advanced security, reliability, and observability features.
graph LR
A[External Webhook] --> B[Security Layer]
B -->|HMAC Verified| C{Idempotency Check}
C -->|New Event| D[PostgreSQL Event Store]
C -->|Duplicate| E[200 OK Response]
D --> F[BullMQ / Redis]
F --> G[Worker Process]
G --> H[Final Actions: Email/Log]
subgraph Observability
G -.-> I[Jaeger Tracing]
G -.-> J[Winston Logs]
end
| Component | Implementation | Industry Benefit |
|---|---|---|
| Reliability | BullMQ + Exponential Backoff | Ensures critical transaction data is never lost |
| Cybersecurity | HMAC Signature Verification | Prevents spoofing attacks on financial & system webhooks |
| Observability | OpenTelemetry + Jaeger | Drastically reduces MTTR (Mean Time To Repair) in production |
| Data Integrity | PostgreSQL Idempotency Store | Prevents duplicate processing and financial inconsistencies |
- Webhook Controllers: Provider-specific endpoints with validation
- Security Layer: HMAC signature validation and input sanitization
- Event Store: Immutable event storage with status tracking
- Queue System: BullMQ for reliable asynchronous processing
- Worker Processes: Dedicated job processors with retry logic
- Observability Stack: OpenTelemetry tracing and Winston logging
- Language: TypeScript 5.0+
- Runtime: Node.js 18+
- Framework: NestJS 10+ (Dependency Injection, Modules)
- Database: PostgreSQL 15+ with Prisma ORM
- Queue: BullMQ 5+ with Redis 7+
- Observability: OpenTelemetry 1.20+ + Jaeger
- Logging: Winston 3.11+ (JSON structured logs)
- Containerization: Docker 24+ with multi-stage builds
- Health Checks: @nestjs/terminus
- Testing: Jest 29+ with Supertest
- Signature Validation: HMAC verification for each provider
- Provider Support: Stripe, PayPal, GitHub webhook formats
- Security Headers: Proper handling of provider-specific headers
- Event Deduplication: Database-backed idempotency keys
- Retry Mechanism: Configurable retry attempts with exponential backoff
- Dead Letter Queue: Failed events moved to separate queue for analysis
- Status Tracking: Complete event lifecycle monitoring
- OpenTelemetry Tracing: Distributed tracing with Jaeger
- Structured Logging: JSON-formatted logs with Winston
- Health Checks: Database and memory monitoring
- Metrics: Processing success/failure rates
- Queue-Based: Non-blocking webhook acceptance
- Worker Processes: Dedicated job processors
- Backoff Strategy: Smart retry timing
- Concurrency Control: Configurable worker limits
POST /webhooks/:provider
Content-Type: application/json
X-Stripe-Signature: t=1234567890,v1=signature...Supported Providers:
stripe- Stripe payment webhookspaypal- PayPal transaction webhooksgithub- GitHub repository webhooks
GET /healthReturns system health status including database connectivity and memory usage.
# Database
DATABASE_URL="postgresql://user:pass@localhost:5432/event_integration"
# Redis
REDIS_HOST="localhost"
REDIS_PORT="6379"
# Webhook Secrets
STRIPE_WEBHOOK_SECRET="whsec_..."
PAYPAL_WEBHOOK_SECRET="..."
GITHUB_WEBHOOK_SECRET="..."
# OpenTelemetry
OTEL_SERVICE_NAME="event-driven-integration-service"
JAEGER_ENDPOINT="http://localhost:14268/api/traces"- Max Retries: 3 attempts per event
- Backoff: Exponential delay (2^attempt * 1000ms)
- Dead Letter: Events moved after max retries
- Cleanup: Completed jobs removed after 10, failed after 5
- Node.js 18+
- PostgreSQL
- Redis
- Docker (optional)
-
Clone and install
git clone https://github.com/PkLavc/event-driven-integration-service.git && cd event-driven-integration-service npm install
-
Database setup
npm run prisma:migrate npm run prisma:generate
-
Start infrastructure
docker-compose up -d
-
Run the service
npm run start:dev
docker-compose up --buildcurl -X POST http://localhost:3001/webhooks/stripe \
-H "Content-Type: application/json" \
-H "Stripe-Signature: t=1234567890,v1=test_signature" \
-d '{
"id": "evt_test_webhook",
"type": "payment.succeeded",
"data": {
"amount": 1000,
"currency": "usd"
}
}'curl http://localhost:3001/healthAccess tracing dashboard at http://localhost:16686
logs/combined.log- All log levelslogs/error.log- Error logs only
BullMQ provides dashboard at Redis connection.
Codebase follows the Twelve-Factor App methodology, treating logs as event streams. JSON formatting ensures compatibility with modern log aggregators like ELK or Datadog.
- Events stored immutably in database for audit trail
- Status transitions tracked for debugging and monitoring
- Complete webhook processing history maintained
- Failed providers can be temporarily disabled
- Automatic recovery attempts
- Multi-step event processing
- Compensation actions for failures
- Validation Errors: 400 Bad Request with details
- Signature Invalid: 400 Bad Request
- Idempotency Conflict: 200 OK (already processed)
- Processing Failures: Retried with backoff
- System Errors: 500 with correlation ID
- Connection Pooling: Prisma handles database connections
- Redis Clustering: Horizontal scaling support
- Worker Scaling: Multiple worker instances
- Event Batching: Future enhancement for high volume
- Signature Verification: HMAC-SHA256 validation
- Input Sanitization: JSON schema validation
- Rate Limiting: Configurable per provider
- Audit Logging: All events logged with metadata
- PostgreSQL 15+
- Redis 7+
- Jaeger Collector
- Load Balancer
- Horizontal pod scaling
- Database read replicas
- Redis cluster
- Message queue partitioning
- Prometheus metrics
- Grafana dashboards
- Alert Manager
- ELK stack for logs
Patrick - Computer Engineer To view other projects and portfolio details, visit: https://pklavc.github.io/projects.html
Built to showcase enterprise-grade event processing capabilities.