🛒 Modular Mart (E-Commerce Microservices)

🧠 System Overview

Modular Mart is a cloud-native, microservices-based e-commerce platform designed with a focus on domain isolation, event-driven choreography, and deep observability. It has evolved from a single-vendor store into a full multi-vendor marketplace.

• API Gateway: The single entry point using Kong, handling routing, rate limiting, and auth verification. A NestJS-based gateway exists as a legacy alternative.
• User Service: Manages identity and profiles, synced with Clerk.
• Catalog Service: Manages product inventory, categories, and seller product approvals.
• Order Service: Core domain managing orders, seller-specific splits, and the Transactional Outbox-based saga.
• Payment Service: Handles Stripe payments, webhooks, and asynchronous payment status updates.
• Notification Service: A robust multi-channel notification engine (Email, SMS, Push, In-App) with real-time SSE updates.
• Web Frontend: A modern Next.js storefront with dedicated Customer and Seller dashboards.
• Docs: A Next.js-based documentation site providing developer guides and API references.

---

🏗 System Architecture

The system follows the Database-per-Service and Microservice Chassis patterns, fully integrated with the LGTM observability stack.

graph TB
    subgraph Client_Layer [Client Layer]
        Web[Next.js Storefront / Dashboards]
        Docs[Next.js Docs Site]
    end

    subgraph Entry_Layer [Entry Layer]
        Gateway[Kong API Gateway]
        Auth[Clerk Auth]
    end

    subgraph Logic_Layer [Service Layer]
        UserSvc[User Service]
        CatalogSvc[Catalog Service]
        OrderSvc[Order Service]
        PaymentSvc[Payment Service]
        NotifSvc[Notification Service]
    end

    subgraph Data_Layer [Isolated Data Layer]
        UserDB[(User PostgreSQL)]
        CatalogDB[(Catalog PostgreSQL)]
        OrderDB[(Order PostgreSQL)]
        PaymentDB[(Payment PostgreSQL)]
        NotifDB[(Notification PostgreSQL)]
    end

    subgraph Cache [Shared Redis Cache]
        Redis[(Upstash Redis)]
    end

    subgraph Messaging [Event-Driven Choreography]
        RMQ[RabbitMQ Event Bus]
        Outbox[Transactional Outbox]
    end

    subgraph Observability [Full LGTM Stack]
        Grafana[Grafana Dashboards]
        Loki[Loki Logging]
        Prom[Prometheus Metrics]
        Jaeger[Jaeger Tracing]
    end

    Web --> Gateway
    Docs --> Gateway
    Gateway --> Auth
    Gateway --> UserSvc
    Gateway --> CatalogSvc
    Gateway --> OrderSvc
    Gateway --> PaymentSvc
    Gateway --> NotifSvc

    UserSvc --> UserDB
    CatalogSvc --> CatalogDB
    OrderSvc --> OrderDB
    PaymentSvc --> PaymentDB
    NotifSvc --> NotifDB

    UserSvc -- Cache --> Redis
    CatalogSvc -- Cache --> Redis
    NotifSvc -- Cache --> Redis

    OrderSvc -- Events --> RMQ
    CatalogSvc -- Events --> RMQ
    PaymentSvc -- Events --> RMQ
    UserSvc -- Events --> RMQ
    RMQ -- Consume --> NotifSvc
    
    Logic_Layer -.-> Loki
    Logic_Layer -.-> Prom
    Logic_Layer -.-> Jaeger
    Loki & Prom & Jaeger --> Grafana

Loading

---

🔄 Core Architectural Patterns

1. Choreographed Saga & Outbox Pattern (Checkout)

Checkout is handled through an asynchronous Choreographed Saga combined with the Transactional Outbox Pattern. This guarantees that database changes and event publishing are atomic.

Order States: PENDING_STOCK → PAYMENT_PENDING → PAID → SHIPPED → DELIVERED

sequenceDiagram
    participant U as User
    participant Gateway as Kong API Gateway
    participant Order as Order Service
    participant Catalog as Catalog Service
    participant Payment as Payment Service
    participant RMQ as RabbitMQ

    U->>Gateway: POST /orders
    Gateway->>Order: Create Order (PENDING_STOCK)
    Order->>Order: DB: Save Order + STOCK_RESERVE_REQUESTED (Outbox)
    Order-->>U: Return Success (OrderId)

    Note over Order,RMQ: Outbox Processor publishes event
    Order->>RMQ: Publish STOCK_RESERVE_REQUESTED
    RMQ->>Catalog: Consume Event
    
    Catalog->>Catalog: DB: Reserve Stock (Pessimistic Lock + Idempotency)
    Catalog->>RMQ: Publish STOCK_RESERVED
    
    RMQ->>Order: Consume Event
    Order->>Order: DB: Update Status (PAYMENT_PENDING) + ORDER_CREATED (Outbox)
    Order->>RMQ: Publish ORDER_CREATED
    
    Note over Payment,U: Payment occurs via Stripe Webhook
    Payment->>RMQ: Publish PAYMENT_SUCCEEDED
    RMQ->>Order: Consume Event (Idempotent)
    Order->>Order: DB: Final Status (PAID)

Loading

2. Microservice Chassis

All services inherit standard behavior from the packages/ directory:

• Tracing: Every request is tagged with an X-Request-ID (Correlation ID) that persists across HTTP and RabbitMQ boundaries.
• Logging: Structured JSON logging via nestjs-pino for centralized aggregation.
• Idempotency: All event consumers use a processed_messages table to ensure Exactly-Once processing semantics.
• Health: Standardized /health/live and /health/ready endpoints.

3. Full-Stack Observability (LGTM)

We implement the full LGTM stack for deep system visibility:

• Loki: Log aggregation with container-level metadata.
• Grafana: Unified dashboards combining logs, metrics, and traces.
• Prometheus: Metrics scraping for performance and health monitoring.
• Jaeger: Distributed tracing to visualize request flow across the entire microservice web.

4. Cache-Aside with Single-Flight

A shared cache-aside layer backed by a single Upstash Redis instance (ioredis) with single-flight stampede protection:

• CacheService.getOrSet(): Checks Redis first; on miss, acquires a distributed SET NX PX lock so only one request recomputes while others wait.
• Key namespacing: catalog:*, user:*, notif:* prefixes prevent collisions across services.
• TTL strategy: Catalog 5–30 min, User 5–10 min, Notifications 30s–5 min — tuned per read pattern.
• Invalidation: Pattern-based SCAN + pipeline DEL for list keys; targeted DEL for specific entities. Notification invalidation happens before SSE pushes.
• Observability: 6 Prometheus metrics (cache_hits_total, cache_misses_total, cache_lock_acquired_total, cache_lock_contention_total, cache_loader_duration_seconds, cache_invalidation_total), Redis ping in /health/ready, and auto-instrumented OpenTelemetry spans.

---

🛡️ Resiliency & Reliability

Modular Mart is designed for production-grade stability:

• Dead Letter Queues (DLQ): Failed messages are automatically routed to DLQs after retries are exhausted to prevent pipeline stalls.
• Exponential Backoff: Transient failures (like database hiccups) are handled via a custom retry decorator with increasing delays (1s, 2s, 4s).
• Compensating Transactions: The Checkout Saga automatically "rolls back" stock reservations if a downstream payment fails.
• Idempotent Consumers: Every event handler uses a processed_messages guard to guarantee Exactly-Once side effects.
• Pessimistic Locking: Prevents inventory overselling by locking product rows during the reservation window.

---

🧪 Testing & Reliability

Every distributed-systems claim in this README is backed by automated tests.

What's covered

Area	What's tested	Service
Saga state machine	Happy-path transitions (PENDING_STOCK → PAYMENT_PENDING → PAID), compensations (→ CANCELLED / STOCK_FAILED)	order-service
Transactional outbox	Event publishing, messageId correlation, error recovery, partial failure continuation	order-service
Stripe webhook leg	Payment recording, idempotency, edge cases (missing metadata, unknown user)	payment-service
Stock reservation	Pessimistic locking, overselling prevention, cache invalidation	catalog-service
Circuit breaker	CLOSED/OPEN/HALF_OPEN transitions, failure thresholds, per-service isolation	@repo/common
Exponential retry	Backoff sequencing, transient vs. permanent classification	@repo/common

Running tests

pnpm test          # Run all unit tests (Turborepo, cached)
pnpm test:cov      # With coverage reports
cd apps/order-service && pnpm test   # Single service

Testing philosophy

• Unit tests simulate the full saga flow with mocked TypeORM/RabbitMQ dependencies — no external infra required.
• AAA pattern (Arrange, Act, Assert) with explicit mocks for repositories, event buses, and external APIs.
• Idempotency is tested, not just claimed — duplicate payment/webhook events are verified to be no-ops.
• CI gates every merge: lint → typecheck → test → build on every PR.

---

📁 Engineering & Project Structure

Managed via Turborepo, the codebase is optimized for sharing types and logic without tight coupling.

e-commerce-microservices/
├── apps/
│   ├── api-gateway/            # Legacy NestJS-based gateway
│   ├── catalog-service/        # Products, Inventory, & Categories
│   ├── docs/                   # Developer documentation site
│   ├── kong-gateway/           # Primary API Gateway
│   ├── order-service/          # Order lifecycle & Saga coordination
│   ├── payment-service/        # Stripe integration & webhooks
│   ├── notification-service/   # Multi-channel (SSE, Email, SMS)
│   ├── user-service/           # Identity & Clerk sync
│   └── web/                    # Storefront & Dashboards (Customer/Seller/Admin)
├── packages/
│   ├── auth/                   # Shared Clerk guards & Roles logic
│   ├── common/                 # Chassis (Logging, Tracing, Health, Metrics)
│   ├── contracts/              # Shared DTOs & Event Patterns (Source of Truth)
│   ├── database/               # Shared TypeORM config
│   ├── eslint-config/          # Shared ESLint configuration
│   ├── shared-types/           # Shared TypeScript types and interfaces
│   ├── typescript-config/      # Shared tsconfig.json files
│   └── ui/                     # Design System (Tailwind + Shadcn)

---

🛠 Tech Stack & Tools

• Backend: NestJS, TypeORM, PostgreSQL, RabbitMQ
• Caching: Upstash Redis (ioredis) with cache-aside + single-flight
• Frontend: Next.js 14 (App Router), Tailwind CSS, Shadcn UI
• API Gateway: Kong
• Observability: Loki, Prometheus, Grafana, Jaeger
• Identity: Clerk (Auth-as-a-Service)
• Payments: Stripe (Elements + Webhooks)
• Infrastructure: Docker, Turborepo, Render