feat: Rust coordinator — coordinator-rs workspace (plan Milestones 1–4, pilot scope)

[Gajesh2007]

Summary

Implements the Rust coordinator per docs/architecture/rust-coordinator-plan.md (added in this PR, including the latency amendments: speculative prefill, prepare-stage hedging, binary hot-path frames, calibrated TTFT, single-CTE money transactions). Greenfield: everything lives under coordinator-rs/; no existing coordinator/provider/UI code is touched.

• crates/protocol — JSON v1 wire types pinned by 43 golden vectors generated from the real Go coordinator/protocol package (omitempty semantics locked for the Swift strict decoder); protocol-v2 frames (prepare/prepared/start/started/abort/cancel/signed terminal/ACK/model lifecycle) with job/attempt/lease + session/coordinator epoch + nonce + digest fencing; binary encrypted-payload frame codec (zero-copy decode); NaCl Box byte-verified against Go (incl. precomputed shared keys), sealed-sender envelope, P-256 attestation + canonical-status verification, v2 terminal digest canonicalization.
• crates/core — pure domain: request-lifecycle reducer (funding CAS, one prepare hedge + one sequential alternate pre-start only, full §13 cancellation ladder, sent_unknown hold), fleet admission/scoring with clamped windowed-median TTFT calibration, health machine, permit book, hedge budget, revision-fenced model presence, settlement provenance + conserving splits + checkpoint-capped billing. Nine proptest suites cover every §9 invariant.
• migrations/ — additive rust_coord schema (8 migrations): jobs with version CAS + frozen terms + an at-most-one-started-attempt partial unique index, terminal receipts with conflict flags, financial_operations idempotency keys, insert-only fee_allocations, external inbox/intents, outbox, coordinator ownership epoch. Plus a test-only legacy baseline traced from store/postgres.go and a money-flow SQL smoke.
• crates/server — Axum chat completions (SSE + sealed transport), models, encryption-key; RequestTask driving the core reducer (v2 two-phase prepare/fund/start + hedging, v1 compatibility mapping that works against today's Swift providers); single-task FleetActor (reliable lane > coalesced heartbeats, epoch supersede); two-lane session writer with on-wire acks and a reserved terminal-class event slot; blocking-pool trust verifier (SE signatures incl. v2 terminals — forged terminals security-fence the provider); single-CTE reserve, advisory-lock-serialized spend caps, settle with fee rows + legacy projections, release, review parking; recovery sweepers (incl. expired running jobs → review_pending); ownership guard with black-hole-detecting keeper; phased-shutdown supervisor; bootstrap shared by the binary and tests.

Verification: 263 tests green (unit + property + Postgres-backed integration + a 4-scenario full-stack e2e with real ephemeral Postgres and fake providers doing real crypto over real WebSockets, asserting actual DB money rows and zero leaked permits); clippy zero warnings; release build clean; binary smoke (migrate → serve → readyz → SIGTERM drain → ownership release). Dual review (Bugbot + independent design review) found 8 issues — all fixed with fails-without-fix regressions in the final commit.

Behavior: before / after

flowchart LR
  subgraph Before["Before (Go coordinator, unchanged by this PR)"]
    A1[Consumer request] --> B1[Preflight scan + separate committing reserve]
    B1 --> C1[Dispatch ladder: up to 64 attempts,<br/>120s queue, start-stage speculation]
    C1 --> D1[inference_accepted before real capacity]
    D1 --> E1[Settlement fan-out goroutines<br/>process-local holds]
  end
  subgraph After["After (coordinator-rs, isolated pilot path)"]
    A2[Consumer request] --> B2[Durable single-CTE reserve<br/>idempotent op keys]
    B2 --> C2[One FleetActor::admit<br/>permit + frozen quote]
    C2 --> D2[v2: prepare -> prepared lease -> fund/freeze -> start<br/>one hedge + one alternate pre-start only]
    D2 --> E2[Signed terminal -> one settlement txn<br/>fee rows + legacy projections -> ACK after commit]
    C2 -.no capacity.-> F2[Fast 429 + Retry-After<br/>no internal queue]
  end

Loading

Code: before / after

flowchart LR
  subgraph Before["Before"]
    G1[coordinator/ Go control plane<br/>PendingRequest: 4 channels + 3 locks<br/>Registry god object, 157-method Store]
  end
  subgraph After["After (new, coexisting)"]
    H2[crates/protocol<br/>wire + crypto, pure]
    H3[crates/core<br/>reducers + admission + money math, pure]
    H4[crates/server<br/>Axum / RequestTask / FleetActor /<br/>ProviderSession / SQLx ledger / recovery]
    H5[migrations/ additive rust_coord schema]
    H2 --> H4
    H3 --> H4
    H5 --> H4
  end
  G1 ---|untouched; Go remains prod| G1

Loading

Scope and non-goals

Plan Milestones 1–4 code scope (isolated-pilot surface, §23.2). Deliberately excluded, per the plan's sequencing: Go Milestone-0 safety fixes (separate fast-fix work), the dual-stack Swift provider that speaks protocol v2 on the provider side, Stripe/Privy/MDM/enrollment/self-route surfaces, and any production cutover machinery. Known gaps are listed in the final commit message (unacked pre-start rejection terminals, fixed hedge timeout pending per-model percentiles, provider auth via account API keys pending a device-token surface, v1 bills the frozen prompt estimate).

Test plan

• cargo test --workspace — 263 green (incl. Postgres-backed ledger/recovery/e2e suites)
• cargo clippy --workspace --all-targets — zero warnings
• Golden-vector compatibility vs Go protocol package (coordinator-rs/fixtures/)
• Money-flow SQL smoke vs baseline + migrations
• Binary smoke: migrate → serve → readyz → graceful SIGTERM
• CI on this PR (no Rust lane exists yet — follow-up: add a coordinator-rs job to ci.yml)
• Isolated pilot per plan §23 (separate env, dedicated providers) — future milestone

Made with Cursor

---

^{Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
d-inference	Ready	Preview	Jul 10, 2026 10:02pm
d-inference-console-ui-dev	Ready	Preview	Jul 10, 2026 10:02pm
d-inference-landing	Ready	Preview	Jul 10, 2026 10:02pm

[github-actions]

This PR introduces a substantial new Rust coordinator codebase (coordinator-rs/) that is entirely outside the current threat model's affected_files coverage and introduces new or parallel trust-boundary surface that must be modelled before any production deployment.

---

Trust Boundaries Touched

None of the changed files match any existing affected_files pattern. However, based on file paths and module names, the following existing trust boundaries are structurally replicated or extended by this new codebase:

Existing TB	New surface in coordinator-rs/
TB-001 (consumer → coordinator)	Request machine (request/machine/), admission, effects, errors
TB-002 (coordinator → provider WebSocket)	Fleet admission gates (fleet/admission/), scoring, permits, health
TB-006 (admin → coordinator)	Admission decisions, calibration, model presence
TB-008 (coordinator → payments)	money.rs, request effects

---

New Attack Surface Not Covered by Existing Threats

1. Parallel coordinator implementation — trust model undefined

coordinator-rs/ appears to be a Rust rewrite (or parallel implementation) of the Go coordinator. The threat model assumes a single trusted coordinator running on EigenCloud AMD SEV-SNP. If coordinator-rs runs alongside or as a replacement, every trust assumption in the model (TB-001 through TB-009) needs to be re-evaluated for the new implementation. Critical questions that must be answered before this reaches production:

• Does coordinator-rs enforce the same attestation chain (TB-009)? The fleet admission gates (gates.rs) and decision.rs appear to encode provider routing policy, but there is no evidence of SE-signature verification, MDM/MDA verification, or challenge-response logic in the diffed files.
• Does admit.rs in the request machine enforce the same 11-flag private-text routing gate as coordinator/registry/registry.go:169–197? The Go gate is the single chokepoint for consumer prompt privacy (A-001). A Rust reimplementation that misses any flag is a regression against T-009 / SEC-017.
• Does scoring.rs / permits.rs replicate the binary-hash policy and weight-hash enforcement? If scoring admits providers without checking CodeAttested (T-034) or without the weight-hash gate (SEC-007 / T-027), the Rust path is fail-open by default.

2. money.rs — payment logic without visible idempotency controls

coordinator-rs/crates/core/src/money.rs introduces monetary types alongside request/effects.rs. The Go coordinator has a known non-atomic Stripe idempotency bug (SEC-012 / T-030). If this Rust implementation touches balance mutation, the same race condition must be explicitly addressed with a DB-level unique constraint. No such constraint is visible in the diffed files.

3. provider_error.rs and request/errors.rs — error granularity leak

The Go coordinator's device auth flow leaks code lifecycle via distinct 404 vs 410 responses (TB-001 current limitations). If provider_error.rs / errors.rs propagates similarly granular error variants to unauthenticated or consumer-facing surfaces, the same enumeration risk applies.

4. hedge.rs — speculative / hedged dispatch

fleet/hedge.rs suggests hedged request dispatch (sending to multiple providers speculatively). This has direct implications for A-001 (consumer prompts): a hedged request may fan out an encrypted prompt to more than one provider simultaneously. The threat model does not currently address fan-out routing; if the second provider receives the same consumer X25519-encrypted payload, and that provider is at a lower trust level than the primary, privacy guarantees are weakened.

5. No threat model entries for coordinator-rs/

All existing threats reference Go coordinator files under coordinator/. None reference coordinator-rs/. This means:

• Automated threat-model coverage checks (like the one that triggered this review) will never flag regressions in the Rust codebase.
• The affected_files patterns in the threat model YAML must be extended to cover coordinator-rs/crates/core/src/fleet/admission/, request/machine/, money.rs, and equivalents before this component is used in production.

---

Recommendations

1. Before merging to a production-routable path: confirm whether coordinator-rs is purely internal library code, a shadow/canary deployment, or a replacement. If it can route real consumer requests or admit real providers, a threat model amendment PR is required first.
2. Audit fleet/admission/gates.rs and request/machine/admit.rs against the Go registry's private-text routing gate (11 flags, registry.go:169–197) and the CodeAttested gate (T-034). Any missing flag is a privacy regression.
3. Audit money.rs + request/effects.rs for idempotency controls on balance mutation (T-030 / SEC-012).
4. Audit fleet/hedge.rs for fan-out trust-level enforcement — hedged requests must only go to providers meeting the same trust gate as the primary.
5. Extend threat model affected_files to cover coordinator-rs/ paths so future PRs touching this codebase receive automated coverage.

---

🔐 Threat model: docs/threat-model.yaml · Updates on each push to this PR

[Gajesh2007]

Two follow-up commits pushed:

Networking audit (a27d8064) — verified the HTTP/SSE/WebSocket stack at the socket level against the production Caddy topology; five real fixes: TCP_NODELAY + armed header-read timeout via a dedicated accept loop (bare axum::serve sets neither — Nagle alone would have blown the 2ms relay budget on the Linux host), a /health alias (prod Caddyfile probes /health; only /healthz existed — behind real Caddy every request would have been shed), WS max_frame_size 16→32 MiB + bounded close handshake, concurrency shed before body buffering, and X-Accel-Buffering/no-store on SSE. Raw-TCP latency test shows per-chunk delivery with no batching (mean gap 14ms at a 10ms send cadence); backpressure verified bounded end-to-end (provider cancel at 1.17 MiB with a stalled reader).

Modularization (8dd46787) — move-only restructure per coordinator-rs/docs/STYLE.md, distilled from the otter-sec/anchor workspace conventions (thin mod.rs, one concern per file, private siblings, types/error placement). Largest source file went from 1608 lines to 393; nothing above the 500 hard cap; all external paths preserved via re-export shims; 122 integration tests exact-name parity.

Workspace after both: 271 tests green, clippy zero warnings, fmt clean.