feat(cluster-protocol): bootstrap v1 and prove initialize/get

[tomdps]

Parent

• [EPIC] Open Engine Cluster Protocol v1: programmatic interface and legacy Zeroshot bridge #643

Execution

AFK. This is the first tracer bullet and can start immediately after its body is verified.

What to build

Establish the smallest real Open Engine Cluster Protocol v1 path: a Rust client negotiates openengine.cluster/v1 with a backend-neutral server and reads the canonical empty cluster state. The same behavior must work in process and through NDJSON stdio, proving the wire and backend seams before later graph or lifecycle semantics are added.

The slice introduces the Cargo workspace and the four foundation crates (openengine-cluster-protocol, openengine-cluster-server, openengine-cluster-client, and openengine-cluster-testkit), a pinned Rust toolchain, generated-artifact drift checking, and the async ClusterBackend/client transport boundaries. Only initialize and empty get are functional in this issue; other methods may be typed but must return an explicit unsupported-domain response rather than pretend to work.

JSON-RPC uses named params, rejects batches, keeps stdout protocol-only in stdio mode, and reports diagnostics on stderr. Empty state has spec:null, phase:"empty", no generation/run ID, and atCursor:null.

Acceptance criteria

• Rust client and server negotiate exactly openengine.cluster/v1; an incompatible version returns UNSUPPORTED_PROTOCOL_VERSION in error.data.code.
• initialize and get return the same canonical empty status over in-process and NDJSON stdio transports.
• Batch JSON-RPC, positional params, malformed envelopes, and unknown methods are rejected deterministically with correct JSON-RPC numeric classes.
• Stdio stdout contains only complete one-line protocol frames; diagnostics go to stderr.
• Generated JSON Schema/OpenRPC skeletons and golden envelopes are committed and CI fails on drift.
• The existing npm entrypoint and CLI behavior remain unchanged.
• Rust format, clippy with warnings denied, workspace tests, existing Node lint/tests, and opcore check --changed pass.
• Required CI's existing check job runs the Rust gates; Rust is not placed only in an optional side job.
• AGENTS.md documents the new protocol crate routing and source-of-truth convention; CLAUDE.md is untouched.

Verification

cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
npm run lint
npm test
opcore check --changed

Blocked by

None - can start immediately.

[tomdps]

JIT implementation plan

Planning baseline: freshly fetched origin/dev@74e365d4c1116e2d841a978705e7574549002d29. The main checkout is dirty and must not be used for implementation; the ship run must create an isolated worktree from origin/dev via --pr-base dev.

Tracer-bullet outcome

At the end of this issue, the repository has a real Rust protocol seam—not placeholder directories. A Rust client must call one backend-neutral dispatcher through both an in-process transport and an NDJSON stdio transport, negotiate openengine.cluster/v1, and receive the same canonical empty get snapshot.

No graph AST, admission state machine, WebSocket, persistence, TypeScript client, or legacy Node integration belongs in this slice.

Repository changes

1. Add a root Cargo workspace using resolver 2 with exactly these initial members:
   • crates/openengine-cluster-protocol
   • crates/openengine-cluster-server
   • crates/openengine-cluster-client
   • crates/openengine-cluster-testkit
2. Add rust-toolchain.toml pinned to the current stable channel with rustfmt and clippy; keep the workspace minimum Rust version explicit and consistent across crates.
3. Keep canonical wire/domain types in openengine-cluster-protocol:
   • protocol/version and profile constants;
   • JSON-RPC request/response/error envelope types with string or integer request IDs;
   • named InitializeParams, InitializeResult, GetParams, GetResult;
   • ServerCapabilities, ClusterStatus, Phase::Empty, and opaque cursor/generation/run-ID newtypes needed by the response shape;
   • domain error data where stable string codes live under JSON-RPC error.data.code.
4. Keep request dispatch and embedding authority in openengine-cluster-server:
   • async ClusterBackend with only the behavior actually implemented in this slice (initialize and get);
   • ConnectionContext supplied by the embedding and never deserialized from request params;
   • one dispatcher used by every transport;
   • deterministic rejection of top-level batches, positional params, malformed requests, unsupported versions, and unknown methods.
5. Keep transport-facing client behavior in openengine-cluster-client:
   • transport-neutral async request trait;
   • typed ClusterClient<T> methods for initialize and get;
   • in-process transport that crosses the same serialized JSON-RPC boundary/dispatcher as stdio rather than bypassing validation.
6. Keep fixtures and a deterministic empty backend in openengine-cluster-testkit. It advertises no production graph profile, worker protocol, logs, or attach capability.
7. Add an NDJSON stdio server binary/library path:
   • read one complete UTF-8 JSON value per input line;
   • write exactly one single-line JSON response per request to stdout;
   • send all diagnostics to stderr;
   • reject a top-level JSON array with -32600 rather than treating it as a batch;
   • do not add Content-Length framing or provider-ACP coupling.
8. Generate deterministic tooling artifacts from the Rust types into protocol/openengine-cluster/v1/:
   • JSON Schema for the implemented envelopes/methods;
   • an OpenRPC document containing only initialize and get for this slice;
   • golden request/response transcripts for successful negotiation, empty get, incompatible version, invalid params, unknown method, malformed request, and rejected batch.
     Provide a generator --write mode and a CI --check mode that compares bytes without rewriting.
9. Update package scripts with non-mutating Rust/protocol checks, but do not change main, bin, provider-helper exports, or npm package runtime behavior.
10. Extend the existing required .github/workflows/ci.yml check job to install the pinned Rust toolchain and run protocol generation drift, cargo fmt --check, clippy with warnings denied, and workspace tests. Do not create a non-required Rust-only job as the sole gate.
11. Update only AGENTS.md with a concise protocol routing/source-of-truth section. Do not edit CLAUDE.md.

Locked wire behavior

• initialize.protocolVersion accepts exactly openengine.cluster/v1.
• Successful initialize returns the same version, empty capabilities, and status {phase:"empty", observedGeneration:null, currentRunId:null, atCursor:null} (field names may use the Rust serde casing required by the wire, but golden JSON fixes the external form).
• Empty get returns spec:null, the same empty status, and atCursor:null.
• Named params are required; positional params return -32602.
• Unsupported protocol version uses a JSON-RPC application error with error.data.code = "UNSUPPORTED_PROTOCOL_VERSION".
• Unknown methods return -32601; invalid envelope/batch returns -32600; malformed JSON returns -32700; internal failures use -32603 plus structured domain data.
• Stdio stdout is protocol-only. Tests must fail if diagnostics or banners appear there.

Test-first sequence

1. Commit golden JSON-RPC behavior tests for the empty backend and invalid envelopes.
2. Make protocol serde/schema types satisfy the golden vectors.
3. Make dispatcher and in-process client pass the same vectors.
4. Add a spawned stdio black-box test that drives stdin, captures stdout/stderr, and proves protocol parity and stdout purity.
5. Add deterministic generator drift tests.
6. Add CI/package-script coverage and run the repository's complete required checks.

Required proof before completion

cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
npm run lint
npm test
opcore check --changed

Also invoke the stdio binary once as a user would, sending initialize and get NDJSON lines and recording the resulting stdout/stderr evidence. Passing unit tests alone is insufficient.

Scope guard

Reject scope expansion into graph modeling, schema subtyping, plan/apply, journal/ledger storage, watch subscriptions, lifecycle mutation, WebSocket, TypeScript, ACP/A2A adapters, current Node orchestrator integration, or publishing. Those have dedicated dependent issues.

[tomdps]

Ship attempt receipt — stale-base rejection

Cluster misty-core-73 and PR #656 were stopped/closed before merge because the runner created its worktree from stale local commit b83a2cf rather than fresh origin/dev@74e365d4.

Evidence:

origin/dev: 74e365d4c1116e2d841a978705e7574549002d29
PR head:    06c0bd62264e13d1bc87056b67ff3d31b55f745c
merge-base: b83a2cf3a40ee144315f36a3e0f3c8d7b4260e30
merge-tree: conflict in AGENTS.md

The run produced and validated a substantial implementation, but accepting hand-reconciled history would violate this issue's fresh-dev requirement and contaminate the dependency chain. Auto-merge was disabled, the PR was closed, and the cluster was gracefully stopped.

Next attempt: launch from a dedicated clean checkout at the then-current origin/dev, verify the created worktree's merge base immediately, and only then allow implementation to proceed. This is launch/process remediation, not a change to the issue's API scope or JIT plan.

[tomdps]

Ship attempt receipt — runner base bug reproduced

Second launch copper-aegis-72 was stopped immediately, before implementation, because the global Zeroshot 6.3.0 runner again created its ship worktree from stale local dev@b83a2cf.

This occurred even though the caller ran from a clean detached launcher checkout at origin/dev@74e365d4, proving the runner resolves the main worktree's local dev branch rather than the caller's verified base:

launcher HEAD:   74e365d4
generated HEAD:  b83a2cf
generated merge-base with origin/dev: b83a2cf

No PR was opened and no implementation from this attempt will be used. Next remediation is to run the current 6.6.0 package with an explicitly supported model, then verify the generated worktree merge base before agents proceed.

[tomdps]

Launch receipt: contained stale-base attempt (Zeroshot 6.6 Sol overlay)

The pinned Zeroshot 6.6.0 runtime passed its integrity/model/template preflight and resolved all worker levels to gpt-5.6-sol with xhigh. It was launched from a clean detached checkout at current origin/dev.

The generated ship worktree nevertheless failed the mandatory base invariant before implementation began:

cluster: stellar-drift-89
launcher HEAD: 74e365d4c1116e2d841a978705e7574549002d29
origin/dev: 74e365d4c1116e2d841a978705e7574549002d29
generated worktree HEAD: b83a2cf3a40ee144315f36a3e0f3c8d7b4260e30
generated merge-base with origin/dev: b83a2cf3a40ee144315f36a3e0f3c8d7b4260e30

The cluster was stopped immediately, before allowing implementation or PR creation. This is launch infrastructure behavior caused by the shared repository's stale local dev, not an implementation failure.

Next attempt: launch the same preflighted runtime from a separate fresh clone whose local dev tracks current origin/dev, then re-check the generated worktree base before allowing work to continue.

[tomdps]

Valid ship launch receipt

Issue #644 is now running in Zeroshot ship mode after the JIT plan above.

cluster: agile-condor-87
runtime: @the-open-engine/zeroshot 6.6.0 (pinned Sol overlay)
workers: gpt-5.6-sol, reasoning xhigh
launcher: separate clean clone on dev
launcher HEAD: 74e365d4c1116e2d841a978705e7574549002d29
origin/dev: 74e365d4c1116e2d841a978705e7574549002d29
generated worktree HEAD: 74e365d4c1116e2d841a978705e7574549002d29
generated merge-base with origin/dev: 74e365d4c1116e2d841a978705e7574549002d29
mode: --ship --pr-base dev -d

The runtime integrity/model/template preflight passed before launch. The generated worktree base invariant also passed before implementation was allowed to continue. Progress will be monitored on a five-minute cadence; dependent issues remain gated on merge and post-merge verification.

[tomdps]

Completion receipt

Implemented and merged by #657.

merged PR: #657
dev merge commit: 8b213476e02f550056214eb6f5d2ab09a56713df
post-merge CI run: 29411568860
required check: success

Post-merge dev proof passed generated Cluster Protocol artifact drift, Rust formatting, clippy with warnings denied, workspace tests, security audit, Node lint/type/provider-helper checks, unit/E2E/integration suites, and Opcore-backed implementation validation. The final implementation also includes regression coverage for formatter-stable generated artifacts, exact protocol-version schema constraints, and JSON-RPC unknown-method error precedence.

This unlocks #645 for JIT planning. No dependent implementation has started before this merge and post-merge verification.