Game Debug MCP

Give your AI evidence, not another screenshot.

Game Debug MCP is an open, engine-neutral visual and performance debugger for AI-assisted game development. It turns saved frame buffers, IDs, traces, and capture receipts into deterministic measurements; walks those observations in causal order; identifies the earliest divergence it can actually prove; and tells the agent which smallest capture would reduce the remaining uncertainty.

It works with any AI through either a read-only Model Context Protocol server or a JSON CLI. The model proposes and explains; the tool measures and refuses unsupported claims.

v0.1 is an evidence analyzer and capture planner. It does not control an editor, launch a game, execute a GPU workload, or claim that pixels look good. Engine and graphics-debugger capture adapters are the next layer, not a hidden promise in this release.

Why this exists

An AI can look at a beauty screenshot and make a plausible guess. Rendering defects usually need a better question:

• Did geometry disappear before shading, or did final color become black later?
• Did the material assignment change, or only its albedo input?
• Is a temporal artifact first visible in motion vectors, depth, history validity, or final color?
• Is a 3 ms improvement measured on the same hardware and workload—or merely two incomparable traces?
• Was a frame submitted, completed, read back, reviewed, or only requested?

Game Debug MCP represents those boundaries explicitly. A diagnosis is a chain of measurements, not a confident paragraph with no receipt.

flowchart LR
  A[Game or capture adapter] -->|PNG, NPY, trace JSON, receipts| B[Sealed frame bundle]
  B --> C[Deterministic analyzers]
  C --> D[First-divergence workflow]
  D --> E[MCP-compatible AI host]
  D --> F[JSON CLI or CI]
  D --> G[Self-contained HTML report]
  D -->|missing evidence| H[Smallest next-capture plan]

Loading

What ships in v0.1

• A zero-runtime-dependency Node.js 20+ analysis core.
• A 13-tool, read-only MCP server over standard input/output.
• A JSON CLI for models and automation that do not use MCP.
• PNG decoding with chunk CRC validation and NumPy .npy decoding.
• Exact SHA-256 artifact seals plus a canonical manifest integrity seal.
• Color, scalar, mask, categorical-ID, normal, and vector-buffer analysis.
• Pixel differences, MAE, RMSE, color-only PSNR and tiled SSIM, first differing coordinate, difference bounds, ID transitions, normal angular error, masks, and heatmaps. Non-finite mismatches invalidate aggregate error metrics instead of producing a false zero.
• Causal workflows for blank frames, missing geometry, wrong materials, flat lighting, wrong shadows, temporal defects, and performance investigations.
• Frame-time distributions, budget breaches, top GPU-pass summaries, and identity-gated trace comparisons.
• Synthetic fixtures for a material-input regression and missing geometry.
• A self-contained HTML evidence report with baseline, candidate, heatmap, causal walk, structured measurements, and an explicit pending human-review state.

The core runs locally and makes no network request.

Quick start

Clone and check the source:

git clone https://github.com/theisegoria/game-debug-mcp.git
cd game-debug-mcp
npm install --ignore-scripts
npm run check

Generate the deterministic demo in a disposable directory:

node bin/game-debug.mjs demo /tmp/game-debug-demo

Ask where the wrong-material case first diverges:

node bin/game-debug.mjs diagnose \
  baseline-material-shift \
  candidate-material-shift \
  wrong_material \
  --project /tmp/game-debug-demo

The important part of the result is:

{
  "first_divergence": {
    "semantic": "albedo",
    "workflow_position": 3,
    "pixel": {
      "x": 32,
      "y": 14
    }
  },
  "confidence": "bounded_first_divergence",
  "next_observation": null
}

Generate a reviewable report:

node bin/game-debug.mjs report \
  baseline-material-shift \
  candidate-material-shift \
  wrong_material \
  --out /tmp/material-report.html \
  --project /tmp/game-debug-demo

The demo is synthetic. It validates the product workflow without starting an engine or GPU job.

Connect an AI host

Every MCP-compatible host has its own configuration surface. The underlying command is:

node /absolute/path/to/game-debug-mcp/bin/game-debug-mcp.mjs \
  --project /absolute/path/to/your-game

A common MCP configuration shape is:

{
  "mcpServers": {
    "game-debug": {
      "command": "node",
      "args": [
        "/absolute/path/to/game-debug-mcp/bin/game-debug-mcp.mjs",
        "--project",
        "/absolute/path/to/your-game"
      ]
    }
  }
}

The project root is fixed when the server starts. Individual MCP calls cannot supply a path or command. A sensible first agent instruction is:

Start with get_project_status. Treat saved artifact analysis, GPU submission, GPU completion, pixel readback, performance, and human visual approval as separate proof axes. Use plan_capture when a causal observation is missing.

For an agent without MCP, run the CLI and consume its JSON output. The measurement contract is the same.

The 13 MCP tools

Tool	Purpose
get_project_status	Count bundles, suites, declared proof axes, and safety properties.
get_debug_catalog	Discover standard semantics, workflows, and proof axes.
list_bundles	Find evidence by stable suite, set, and case identifiers.
get_bundle	Read a manifest without implying that hashes were revalidated.
validate_bundle	Re-hash every artifact and verify the manifest seal.
list_buffers	See which causal observations exist for a frame.
inspect_buffer	Measure distributions, invalid values, occupancy, IDs, normals, and blankness.
compare_buffers	Compare identity-compatible buffers with optional masks and ID selections.
diagnose_visual	Walk one symptom-specific causal chain and bound the first divergence.
triage_suite	Pair baseline/candidate cases and summarize a whole suite.
analyze_trace	Measure frame-time distributions, budget breaches, and top GPU passes.
compare_traces	Refuse mismatched traces or report a compatible median delta.
plan_capture	Request the smallest ordered evidence set for a symptom.

All 13 carry MCP read-only, non-destructive, idempotent, closed-world annotations. The catalog and its parity assertion are generated from the same runtime contract.

Evidence layout

Initialize a project:

node /path/to/game-debug-mcp/bin/game-debug.mjs init /path/to/your-game

This creates:

your-game/
└── .game-debug/
    ├── config.json
    └── evidence/
        └── candidate-town-night/
            ├── manifest.json
            ├── buffers/
            │   ├── beauty.png
            │   ├── coverage.npy
            │   ├── material_id.npy
            │   └── albedo.png
            └── trace.json

A minimal manifest looks like this before ingestion adds hashes and bundle_seal:

{
  "schema": "org.gamedebug.frame_bundle.v1",
  "bundle_id": "candidate-town-night",
  "suite_id": "lighting-regression",
  "set_id": "candidate",
  "case_id": "town-night",
  "identity": {
    "source_revision": "change-under-test",
    "workload_id": "town-night-script-v2",
    "frame_index": 480,
    "backend": "your-backend",
    "hardware_id": "your-device-profile",
    "width": 1920,
    "height": 1080,
    "render_scale": 1,
    "settings_hash": "quality-profile-v4",
    "camera_hash": "camera-pose-17"
  },
  "buffers": [
    { "semantic": "beauty", "path": "buffers/beauty.png", "color_space": "srgb" },
    { "semantic": "coverage", "path": "buffers/coverage.npy" },
    { "semantic": "material_id", "path": "buffers/material_id.npy" },
    { "semantic": "albedo", "path": "buffers/albedo.png", "color_space": "srgb" }
  ],
  "evidence": {
    "gpu_submission": { "status": "unproven" },
    "gpu_completion": { "status": "unproven" },
    "pixel_readback": { "status": "unproven" },
    "performance": { "status": "unproven" },
    "human_review": { "status": "unproven" }
  }
}

Prepare that manifest and its relative artifacts outside the project, then ingest it explicitly through the CLI:

node bin/game-debug.mjs ingest /path/to/export/manifest.json --project /path/to/your-game

Ingestion copies regular files into a fresh bundle, computes every artifact hash, and seals the canonical manifest. It refuses to replace an existing bundle.

See the evidence model for the full contract and the JSON schemas for machine-readable structure.

Standard semantic buffers

The built-in catalog includes:

beauty                 coverage              object_id
material_id            albedo                normal
roughness              metalness             ao
depth                  motion                direct_light
indirect_light         shadow_visibility     history_validity
overdraw               lod                   residency

Adapters can add custom.<name> with an explicit kind. Stable meaning matters more than engine vocabulary: document the units, coordinate space, encoding, valid range, and identity rules.

PNG is useful for inspectable color and encoded debug views. NPY preserves floating-point values and large categorical IDs without visualization loss. A captured beauty image and an analytical buffer can coexist in the same bundle. Color comparisons require the same explicit color_space on both artifacts; v0.1 measures the declared encoded sample space and does not silently convert between sRGB, linear, HDR, or custom spaces.

How first-divergence diagnosis works

Each symptom maps to an ordered causal workflow. For wrong_material, v0.1 checks:

material_id → residency → albedo → normal → roughness → ao → beauty

For each available observation, the analyzer:

1. verifies bundle identity and artifact hashes as required by the call;
2. decodes the buffer with explicit dimensions and channels, then joins its dimensions to the containing manifest identity;
3. computes deterministic statistics and invariant findings;
4. compares the baseline and candidate at the same semantic boundary;
5. records the first coordinate beyond the selected threshold; and
6. returns the earliest divergent semantic in the workflow.

If an earlier semantic is absent, the result says that divergence was observed but not bounded. If the workflow has no divergent saved artifact, it says so. It never fills a missing buffer with a guess.

What makes it different

Common AI game-development tool	Game Debug MCP
Controls an editor, creates objects, changes scenes, or runs commands.	Analyzes immutable evidence and plans the next observation.
Gives the model another screenshot to interpret.	Gives it exact pixels, IDs, distributions, identities, and hashes.
Starts from the visible symptom.	Walks upstream intermediate state to find the first observed divergence.
Reports a pass/fail flag.	Returns counters, coordinates, error magnitudes, bounds, and missing evidence.
Treats a capture as proof that rendering worked.	Separates submission, completion, readback, performance, and human approval.
Is tied to one engine or one model vendor.	Uses engine-neutral semantics, MCP, and a JSON CLI.
Needs broad filesystem or execution authority.	Keeps the MCP surface path-free, command-free, and read-only.

This is complementary to editor-control MCPs, not a replacement for them. Let an editor agent make the change; let Game Debug MCP test whether the evidence moved at the expected boundary.

It is also designed to compose with established capture and inspection tools rather than reimplement them. Potential adapters can translate data from RenderDoc, Open Image Debugger, Perfetto, GFXReconstruct, Metal programmatic capture, PIX programmatic capture, or Nsight Graphics CLI capture into one evidence contract. Those adapters are roadmap work; no such integration is claimed in v0.1.

Safety and trust

The MCP server:

• is read-only;
• is bound to one startup project root;
• exposes identifiers rather than paths;
• rejects path traversal and symbolic-link artifacts;
• caps file bytes, decoded elements, decoded bytes, simultaneously compared bytes, unique IDs, bundle count, protocol message size, and preview size;
• validates PNG CRCs, payload dimensions, SHA-256 digests, and manifest seals; and
• never starts an engine, executable, debugger, editor, or GPU workload.

Integrity is not authenticity. A bundle seal proves that the bytes match the manifest now; it does not prove who produced them, that a GPU completed them, or that a human approved them. See SECURITY.md and docs/EVIDENCE_MODEL.md.

The default hard envelope is 64 MiB per artifact, 64 MiB per decoded tensor, 128 MiB of decoded tensors in one comparison, and 16 MiB per trace JSON file. Trace rows and identifier lengths have separate structural limits. Project configuration can lower but cannot raise them.

Architecture

The package deliberately has three layers:

1. Capture adapters export engine-specific state into the public evidence schema. None ship in v0.1.
2. The deterministic core loads, validates, measures, compares, diagnoses, and reports. It knows semantics, not engines or models.
3. Thin interfaces expose the same core through MCP and the JSON CLI.

That boundary keeps an adapter bug from becoming permission to execute arbitrary work, and keeps a model-specific integration from owning the diagnosis logic. Read docs/ARCHITECTURE.md and docs/ADAPTERS.md before adding a new integration.

Development

JSON-producing CLI commands accept --compact for single-line output. Options and positional arity are strict, so a misspelled threshold or set name fails rather than silently selecting a default.

npm run format:check
npm test
npm run smoke
npm run scan:private
npm run check

npm run smoke starts only the local MCP process against synthetic fixtures. It does not start a game or graphics API.

Contributions should include a falsifying fixture, not just a happy path. See CONTRIBUTING.md.

Roadmap

The next useful work is adapter breadth and stronger image formats, not more agent prose:

• a documented adapter SDK and conformance suite;
• OpenEXR through an optional, separately licensed decoder boundary;
• translators for common frame-capture and trace tools;
• engine templates for exporting standard semantics;
• suite history and baseline promotion with explicit human approval;
• signed producer receipts and remote read-only transport hardening; and
• perceptual metrics that remain deterministic and locally reproducible.

See docs/ROADMAP.md for release gates. Current claims stop at what v0.1 tests.

License

Apache-2.0. See LICENSE.