Node Pulse Admiral

A comprehensive agent fleet management dashboard for monitoring Node Pulse agents across your infrastructure.

Why Push-Based Architecture?

Node Pulse uses a push-based approach where agents actively send metrics to the dashboard, unlike traditional pull-based systems (e.g., Prometheus) that scrape metrics from targets. This provides significant advantages:

Key Benefits

1. Firewall-Friendly: Agents can push metrics through firewalls, NAT, and network restrictions without requiring inbound ports to be exposed. This makes it ideal for:

   • Agents behind corporate firewalls
   • Servers with strict security policies
   • Cloud instances without public IPs
   • Edge devices with dynamic IPs

2. Built-in Reliability: Each agent has a local buffer that stores metrics when the dashboard is unreachable, ensuring:

   • No data loss during network outages or dashboard maintenance
   • Automatic retry with exponential backoff
   • Up to 48 hours of buffered metrics (configurable)

3. Simplified Network Configuration: No need to:

   • Open inbound firewall rules on monitored servers
   • Configure service discovery mechanisms
   • Maintain allowlists of scraper IPs
   • Set up VPN tunnels for monitoring access

4. Real-time Data: Metrics arrive as soon as they're collected (5-second default interval), providing:

   • Immediate visibility into system state
   • Faster incident detection and response
   • No scrape interval delays

5. Scalability: The dashboard scales independently from the number of agents:

   • Valkey Streams buffer incoming metrics during traffic spikes
   • Multiple digest workers process metrics in parallel
   • No need to manage scrape scheduling and intervals

Architecture

This project uses Docker Compose to orchestrate multiple services:

• Submarines (Go-Gin): High-performance metrics ingestion pipeline (ingest, digest, status, sshws services)
• Flagship (Laravel 12): Web dashboard and management UI with Inertia.js + React frontend
• PostgreSQL 18: Main database with admiral schema
• Valkey: Redis-compatible in-memory data store for message streams, caching, and sessions
• Caddy: Modern reverse proxy and web server with automatic HTTPS

Roadmap

Roadmap

Prerequisites

• Docker and Docker Compose (Docker Desktop or standalone)
• Git

Quick Start

1. Clone the repository (if not already done):

   git clone <repository-url>
   cd node-pulse-stack/dashboard

2. Copy the environment file:

   cp .env.example .env

3. Update the .env file with your configuration:

   • Change POSTGRES_PASSWORD to a secure password
   • Change VALKEY_PASSWORD to a secure password
   • Update JWT_SECRET with a strong random value

4. Start all services:

   docker compose up -d

5. Check service status:

   docker compose ps

6. View logs:

   docker compose logs -f

Service URLs

Once all services are running:

• Flagship (Admin Dashboard): http://localhost (via Caddy)
• Submarines Ingest: http://localhost:8080
• Submarines Status: http://localhost:8082
• PostgreSQL: localhost:5432
• Valkey: localhost:6379
• Vite Dev Server (development): http://localhost:5173

Database Schema

The PostgreSQL database uses a single admiral schema for all application data (shared by Submarines and Flagship):

• servers: Server/agent registry
• metrics: Time-series metrics data
• alerts: Alert records
• alert_rules: Alert rule configurations
• users: User accounts (Laravel Fortify authentication)
• sessions: User sessions
• ssh_sessions: SSH session audit logs
• private_keys: SSH private keys for server access
• settings: Application settings

API Endpoints

Metrics Ingestion

Agents send metrics to:

POST http://your-domain/metrics

Example payload (matches Node Pulse agent format):

{
  "timestamp": "2025-10-13T14:30:00Z",
  "server_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "hostname": "server-01",
  "system_info": {
    "hostname": "server-01",
    "kernel": "Linux",
    "kernel_version": "5.15.0-89-generic",
    "distro": "Ubuntu",
    "distro_version": "22.04.3 LTS",
    "architecture": "amd64",
    "cpu_cores": 8
  },
  "cpu": {
    "usage_percent": 45.2
  },
  "memory": {
    "used_mb": 2048,
    "total_mb": 8192,
    "usage_percent": 25.0
  },
  "network": {
    "upload_bytes": 1024000,
    "download_bytes": 2048000
  },
  "uptime": {
    "days": 15.5
  }
}

Dashboard API

• GET /api/servers - List all servers
• GET /api/servers/:id/metrics - Get metrics for a specific server

Configuring Node Pulse Agents

Update your Node Pulse agent configuration (/etc/node-pulse/nodepulse.yml):

server:
  endpoint: "http://your-dashboard-domain/metrics"
  timeout: 3s

agent:
  server_id: "00000000-0000-0000-0000-000000000000" # Auto-generated if not set
  interval: 5s

buffer:
  enabled: true
  path: "/var/lib/node-pulse/buffer"
  retention_hours: 48

Development

Submarines (Go-Gin Backend)

cd submarines
go mod download

# Run ingest server (receives agent metrics)
go run cmd/ingest/main.go

# Run digest worker (consumes from Valkey Stream, writes to PostgreSQL)
go run cmd/digest/main.go

Flagship (Laravel Web Dashboard)

cd flagship
composer install
npm install

# Run development server (all services)
composer dev

# Or run individually
php artisan serve              # Laravel web server
npm run dev                    # Vite dev server
php artisan queue:listen       # Queue worker
php artisan pail               # Log viewer

# Other commands
php artisan migrate            # Run migrations
php artisan test               # Run tests

Laravel + Inertia.js Stack

Flagship uses Laravel 12 with Inertia.js for a modern SPA experience:

• Backend: Laravel for API, authentication, and business logic
• Frontend: React 19 with TypeScript
• Routing: Server-side routing via Inertia.js (no client-side router needed)
• UI Components: Radix UI + Tailwind CSS
• Authentication: Laravel Fortify with CAPTCHA support

Creating New Pages

1. Create a controller in flagship/app/Http/Controllers/:

<?php
namespace App\Http\Controllers;

use Inertia\Inertia;

class ExampleController extends Controller
{
    public function index()
    {
        return Inertia::render('example', [
            'data' => [...],
        ]);
    }
}

2. Create a React component in flagship/resources/js/pages/:

// resources/js/pages/example.tsx
export default function Example({ data }) {
  return <div>Your page content</div>;
}

3. Add a route in flagship/routes/web.php:

Route::get('/example', [ExampleController::class, 'index']);

Accessing pgweb

pgweb provides a web-based interface to browse and query the PostgreSQL database:

1. Open http://localhost:8081
2. Connection is pre-configured via DATABASE_URL environment variable
3. Navigate between schemas using the schema selector

Stopping Services

# Stop all services
docker compose down

# Stop and remove volumes (WARNING: This deletes all data)
docker compose down -v

Updating Services

# Rebuild and restart a specific service
docker compose up -d --build submarines-ingest
docker compose up -d --build submarines-digest
docker compose up -d --build flagship

# Rebuild all services
docker compose up -d --build

Troubleshooting

Services won't start

Check logs for specific services:

docker compose logs submarines-ingest
docker compose logs submarines-digest
docker compose logs flagship
docker compose logs postgres
docker compose logs valkey

Database connection issues

1. Ensure PostgreSQL is healthy:

   docker compose ps postgres

2. Check database logs:

   docker compose logs postgres

3. Verify connection from backend:

   docker compose exec submarines sh
   # Inside container:
   # Try connecting to postgres

Valkey connection issues

docker compose logs valkey
docker compose exec valkey valkey-cli ping

Flagship (Laravel) issues

1. Check Laravel application logs:

   docker compose logs flagship

2. Access Laravel container:

   docker compose exec flagship bash
   php artisan about  # Show Laravel environment info

3. Check frontend build:

   npm run build  # Production build
   npm run dev    # Development with hot reload

Frontend won't load

1. Check if Vite dev server is running:

   docker compose logs flagship | grep vite

2. Ensure Submarines API is accessible:

   curl http://localhost:8080/health

Production Deployment

For production:

1. Update all secrets in .env
2. Use production target in Dockerfiles
3. Set GIN_MODE=release for Submarines
4. Set APP_ENV=production and APP_DEBUG=false for Flagship
5. Run php artisan optimize for Laravel optimization
6. Build frontend assets with npm run build
7. Configure proper domains in .env (ADMIN_DOMAIN, INGEST_DOMAIN, etc.)
8. Use Caddyfile.prod for automatic HTTPS via Let's Encrypt
9. Set up proper backup strategy for PostgreSQL
10. Configure monitoring and alerting
11. Scale digest workers based on Valkey Stream lag
12. Configure Laravel authentication and authorization
13. Set up proper session and cache drivers

License

Apache 2.0

Support

For issues and questions, please open an issue on GitHub.