Skip to content

About

Local-first event pipeline for ordered device signals: UDP/TCP ingress (NestJS), Kafka, Spring Boot processor, PostgreSQL, live React dashboard

Topics

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

SignalBridge

CI

SignalBridge is a local-first event pipeline for ordered device signals. It accepts one canonical JSON event over UDP or framed TCP, validates before publication, normalizes sequence order in a Spring processor, persists the result in PostgreSQL, and exposes a compact live dashboard.

This is a portfolio reference implementation. It contains no benchmark claims, customer data, production traffic, or invented operational results.

Architecture

UDP datagram ─┐
              ├─ NestJS gateway ── signal.raw.v1 ── Spring processor ── PostgreSQL
TCP frame ────┘         │                                  │
                        └─ signal.processed.v1 ◄───────────┘
                                   │
                         REST snapshot + Socket.IO
                                   │
                              React dashboard

Components

  • Gateway — Node.js 20, NestJS, KafkaJS, raw UDP/TCP listeners, validation, bounded buffers, REST, Socket.IO.
  • Processor — Java 21, Spring Boot, Spring Kafka, JPA, Flyway, PostgreSQL. Sequence watermark updates and event inserts share one transaction.
  • Dashboard — React 19 and Vite. It renders only validated processed events and has explicit loading, empty, error, and connection states.
  • Simulator — deterministic UDP/TCP client for local protocol checks, duplicate delivery, and late sequence scenarios.
  • Compose — pinned local Kafka KRaft, PostgreSQL, gateway, processor, and dashboard images. Kafka and PostgreSQL have no host port publication.

Quick start

Requirements: Docker Compose v2, Node.js 20.18+, Java 21, Python 3.12.

cp .env.example .env          # use `copy` in cmd.exe
npm install
docker compose up --build -d
python -m pip install -r scripts/requirements.txt   # Python 3.12
python scripts/smoke.py

Open http://localhost:3000. The gateway API is available at http://localhost:3001/healthz for local diagnostics. The smoke client checks both transports, duplicate idempotency, late ordering, REST state, and a live Socket.IO event.

Stop and remove local data:

docker compose down -v

The Compose fallback password is a local-only development value. Replace it through POSTGRES_PASSWORD for any environment outside this demo; do not commit .env.

Event contract

The canonical event is UTF-8 JSON with a maximum serialized size of 8 KiB:

{
  "schema_version": 1,
  "event_id": "evt_01J...",
  "source_id": "sim-alpha",
  "sequence": 42,
  "occurred_at": "2026-09-01T12:00:00Z",
  "signal_type": "reading",
  "value": 73.5,
  "unit": "ms",
  "labels": {"region": "local"}
}

Supported signal types: heartbeat, reading, alert. Unknown fields, invalid timestamps, negative sequences, oversized labels, and non-finite values are rejected before Kafka publication.

TCP uses a 4-byte unsigned big-endian length prefix followed by exactly one JSON payload. The ACK is newline-delimited and contains only accepted and event_id; malformed input receives a bounded reason code and the connection closes. UDP has no response channel.

Processed events add order_status (in_order or late) and processed_at. Equal or lower sequence values remain persisted as late events; the source watermark advances only for a higher sequence. event_id is the persistence key.

HTTP and live API

  • GET /healthz — process health.
  • GET /readyz — 503 until the gateway producer and processed-event consumer are ready.
  • GET /api/v1/signals?limit=50&source_id=... — bounded normalized snapshot.
  • GET /api/v1/summary — observed processed, in-order, late counts, and latest processing timestamp.
  • Socket.IO namespace /ws/signals, event signal.processed.v1 — normalized events after Kafka consumption.
  • Processor readiness: /actuator/health/readiness.

Project layout

gateway/      NestJS gateway: UDP/TCP ingress, validation, Kafka, REST, Socket.IO
processor/    Spring Boot processor: sequence normalization, JPA, Flyway
dashboard/    React + Vite live dashboard
contracts/    JSON Schema for the canonical event, plus fixtures
scripts/      Smoke client and secret scan
docs/         Design spec, plan and workflow notes

Testing

Gateway:

npm --workspace gateway test -- --run
npm --workspace gateway run typecheck
npm --workspace gateway run lint
npm --workspace gateway run build

Dashboard:

npm --workspace dashboard test -- --run
npm --workspace dashboard run typecheck
npm --workspace dashboard run lint
npm --workspace dashboard run build

Processor:

bash processor/gradlew -p processor check
bash processor/gradlew -p processor integrationTest

On Windows, use processor/gradlew.bat -p processor instead of bash processor/gradlew -p processor.

The integration flow starts real Kafka and PostgreSQL Testcontainers and therefore needs Docker. Operational commands and secret scanning are documented in AGENTS.md; the evidence-led automation workflow is in docs/ai-workflow.md.

Security boundaries

  • UDP and TCP payloads are validated before side effects; TCP length is checked before body allocation.
  • Kafka keys use source_id; payload bodies are not logged.
  • Processor DLQ records contain reason, event ID, and source ID only.
  • Gateway snapshot memory and WebSocket connections are bounded.
  • Compose publishes only loopback gateway/dashboard ports; containers drop capabilities and use non-root application users.
  • CORS is an explicit configured allowlist for HTTP and Socket.IO.
  • No authentication or authorization surface is implied by this local demo.

About

Local-first event pipeline for ordered device signals: UDP/TCP ingress (NestJS), Kafka, Spring Boot processor, PostgreSQL, live React dashboard

Topics

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages