Skip to content

Commit 98f76d5

Browse files
docs(workflow): document queue and database ownership by service (#213)
## Summary ### Why? Issue #211 (follow-up from PR #205) asks for a single place that records the submitqueue topology at a high level: which service owns its data and how the two services communicate. The workflow RFC already covers the cross-queue flow, so ownership belongs alongside it. ### What? Append an "Ownership by service" section to doc/rfc/submitqueue/workflow.md, described at a conceptual level rather than enumerating individual tables and topics: - Gateway — RPC entry point and owner of the request log; the only service that reads or writes that record. - Orchestrator — runs the pipeline and owns its working state (requests, batches, builds); the only service that writes it. - Messaging queue — the shared, pluggable infrastructure the two services communicate through, kept in its own database separate from application data. A closing "Request-log ownership invariant" section captures the rule: the orchestrator only emits log events, the gateway is the sole consumer and the only writer of the request log. Documentation only; no code, schema, or proto changes. ## Test Plan - ✅ `make lint` (clean tree) ## Issue Closes #211 ## Issues ## Stack 1. #205 1. @ #213 1. #214 --------- Co-authored-by: Oz <oz-agent@warp.dev>
1 parent e1ea9c4 commit 98f76d5

1 file changed

Lines changed: 24 additions & 2 deletions

File tree

doc/rfc/submitqueue/workflow.md

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,8 @@ The pipeline has two cycles: `speculate → build → buildsignal → speculate`
1414
│ LandRequest
1515
1616
┌──────────────────────┐ ┌──────────────────────────────────┐
17-
│ log (terminal sink) │◄───│ start │
18-
Append RequestLog │ │ Persist Request, emit Started │
17+
gateway: log │◄───│ start │
18+
Persist request log │ │ Persist Request, emit Started │
1919
└──────────────────────┘ └────────────────┬─────────────────┘
2020
▲ │ RequestID
2121
│ ▼
@@ -90,3 +90,25 @@ The DLQ controllers do not re-attempt the failed work. They decode the payload t
9090
DLQ consumers are wired with `errs.AlwaysRetryableProcessor` and a very high `Retry.MaxAttempts`, with their own DLQ disabled. That combination makes reconciliation effectively non-droppable: any failure is forced retryable rather than escalating to a second-level dead-letter that nobody consumes. The trade-off is that a genuinely unprocessable DLQ message — typically a malformed payload — must be removed by an operator.
9191

9292
See `submitqueue/orchestrator/controller/dlq/README.md` for the design constraints (simplest possible implementation, reconcile-only, no recovery) and the per-topic controller mapping.
93+
94+
## Ownership by service
95+
96+
Each service owns its own data; the gateway and orchestrator never touch each other's, and the only thing they share is the messaging queue.
97+
98+
### Gateway
99+
100+
The gateway is the RPC entry point and the owner of the request log. It accepts requests, hands them to the orchestrator over the queue, and owns the record of what happened to each request — the only service that reads or writes the request log. It writes that record both directly, as requests arrive, and by consuming the log events the orchestrator emits.
101+
102+
### Orchestrator
103+
104+
The orchestrator runs the pipeline that advances a request from acceptance to a terminal state. It owns the working state of that pipeline — requests, batches, builds, and their bookkeeping — and is the only service that writes it. It drives a request through a series of internal stages, re-entering speculation as CI results arrive and as batches advance.
105+
106+
### Shared: the messaging queue
107+
108+
The two services communicate only through the messaging queue. It is pluggable infrastructure kept in its own database, separate from either service's application data: the gateway publishes incoming requests for the orchestrator to consume, and the orchestrator publishes log events for the gateway to consume.
109+
110+
## Request-log ownership invariant
111+
112+
The request log has exactly one owner: the **gateway**. The orchestrator only emits log events onto the queue; it never persists them. The gateway is the sole consumer of those events and the only writer of the request log.
113+
114+
This keeps all request-log writes in one service: the orchestrator stays a pure pipeline that emits events, and the gateway owns the request log end to end.

0 commit comments

Comments
 (0)