Enforce single-authority writes and idempotency across multi-origin deployments #85

Goal

Make multi-origin Zed deployment safe for mutable operations by keeping exactly one admitted write authority per mutable route family and by making retries deterministic under response loss/transport ambiguity.

Depends on zed-pkg/zed-interfaces#149 for explicit origin/write-authority declarations and on compatibility admission before routing.

Core invariants

• Read preference never implies write authority.
• Each mutable route family has exactly one admitted write origin for a given configuration generation.
• A missing, incompatible, or unhealthy write authority makes the operation unavailable; the edge must not silently send the write to a read replica/secondary origin.
• No active-active publication/yank/org mutation is introduced merely because multiple origins are online.
• Cloudflare must not retry an ambiguous write against a different origin.

Idempotency contract

For every admitted mutable request, derive or require a bounded idempotency identity that binds the logical operation and the exact authority-relevant inputs. The server-side authority should distinguish:

1. first execution;
2. exact retry of the same logical operation;
3. conflicting reuse of an idempotency identity with changed payload/authority;
4. definitely-not-committed transport failure where a same-authority retry is safe;
5. ambiguous outcome where commit may have happened and reconciliation is required.

Where the existing Zed API already has operation/request ids, converge on those rather than adding a second unrelated identity.

Initial mutable route families

Exercise at least:

• package publish/upload metadata commit;
• yank/restore/retirement mutation;
• namespace/org ownership mutation;
• organization membership/role mutation where routed through this plane;
• any edge-enabled publication exception must retain its separately reviewed contract and must not become an implicit second normal write authority.

Edge behavior

• Select write authority from the admitted OriginSetV1 generation before sending the body.
• Confirm current compatibility fingerprint and route capability before mutation.
• Preserve request body integrity across forwarding/retry boundaries.
• Do not follow arbitrary redirects for writes.
• Same-authority retry must reuse the same idempotency identity and caller deadline.
• On ambiguous commit outcome, return a typed retry/reconciliation response rather than attempting another origin.
• Changes to the configured write authority must have a generation boundary so an in-flight retry cannot jump authorities silently.

Conformance / failure injection

Test:

• primary write succeeds;
• transport fails before request reaches authority;
• response lost after commit;
• authority becomes unhealthy before body send;
• authority becomes unhealthy after body send;
• config generation changes between first attempt and retry;
• same idempotency id + same payload replays safely;
• same idempotency id + changed payload fails;
• secondary/read-only origin is healthy while write authority is down -> write still fails closed;
• publish/yank/org mutation cannot execute twice across two origins.

Observability

Record secret-free origin id, route family, config generation, idempotency/operation hash or redacted identifier, commit classification, elapsed time, and terminal result. Never log request bearer/session/provider credentials or package private-source secrets.

Non-goals

• no consensus protocol between normal HTTP origins;
• no multi-primary database claim;
• no write hedging;
• no fallback to GitHub/npm/Cargo for registry mutations.