Skip to content

Event feed connector: the run loop and tier-2 driver (2/3) - #705

Open
jeremy wants to merge 21 commits into
event-feed-foundationsfrom
event-feed-go-connector
Open

Event feed connector: the run loop and tier-2 driver (2/3)#705
jeremy wants to merge 21 commits into
event-feed-foundationsfrom
event-feed-go-connector

Conversation

@jeremy

@jeremy jeremy commented Aug 12, 2026

Copy link
Copy Markdown
Member

This PR has been split and force-pushed. It now carries the state machine
only, and is stacked on #777 (foundations). #778 (conformance corrections)
stacks on this. The pre-split head is preserved at tag pre-split/705-head;
the exact commit this PR pointed at before the force-push is
pre-split/705-remote-head (d379f2e11). All 63 review threads are intact,
but line anchors on foundation files now resolve against #777.

Why: eight bot rounds here did not converge (12→3→5→2→2→1→3 threads, with late
findings in files no earlier round had touched), and a review pass then found a
P1 credential defect all eight missed because it composes two files across a
package boundary. That defect is fixed in #777.

The Go reference implementation of the SPEC.md §23 Event Feed connector — BC3's
account-wide event feed over Action Cable push plus polling catch-up — and the
tier-2 conformance driver. All 22 fixtures pass (23 with #778's addition).

Layer 2 only: the connector reaches the wire through TicketMinter /
PollSource seams and one sanctioned cable dial (AGENTS.md Hard Rule 2). The
Layer-1 adapters are deferred to G1b, so the package is experimental and a
consumer supplies the seams today.

What is here

connector.go (New, the options, Events, Close), loop.go (states,
transitions, timers, live buffer), catchup.go (the poll walk, its page
boundary, the drain), recovery.go (the 400/409/410 matrix), and the tier-2
driver with its fixture model, harness and self-tests.

The foundations — seams, wire types, transports, filters, checkpoint identity,
the file store, dedupe, backoff, clock, cable codec, feedtest/ — are #777.

Rebase note

The 29 commits were re-cut, not rebased. Rebasing onto #777 was attempted
and abandoned: 8 of them touch only foundation files and would replay empty, and
21 of the remaining 22 mix both halves, so every one conflicts against the four
fixes #777 applies. Resolving 22 interleaved conflicts by hand is a worse
guarantee than re-cutting, which is byte-exact by construction — the foundation
paths come from #777's tip, these 17 from the pre-split head, verified with an
empty git diff against both.

The five fixes on top

Blocker 2 — a poll page carrying no position is malformed

The walk took page.Position on trust, and an empty one silently skipped
history in two different ways.

Position-resume: acceptPosition("") sets l.position = "", and
entryCursor selects on l.position != "" — so it does not preserve the old
cursor, it falls through to a bare present entry. The feed resumes at the
server's head with everything between skipped, reporting nothing.

Present-class is worse. held uses "" as its sentinel for "the final entry
was not present-class", so an empty position is not saved as empty — it
collapses into the sentinel, the held != "" guard skips acceptPosition and
saveCheckpoint outright, and caught_up announces anyway. The position is
discarded, with the drain's deliveries already handed to the consumer.

Refused before delivery, counter resets, and every mutation. That placement is
what the mutation check exercises: moving the guard after the delivery loop
fails three of four subtests on delivered ids = [101], want [].

Blocker 6 — order the one durable effect against Close, and narrow the promise

Close's doc claimed "no seam call and no delivery can begin after Close has
returned". That is not true and cannot be made true: the run goroutine checks its
context at each dispatch point and then acts, so a Close landing in between
cannot stop the call from starting — only from starting on a live context.
Closing that window means holding a lock across arbitrary host code, trading a
benign race for a deadlock reachable from any callback.

Narrowed to what holds. One effect is not self-limiting and is now ordered: a
checkpoint save. Cancellation cannot stop it — a CheckpointStore may ignore
ctx, and the built-in one documents that it does, so the cancelled context
Close publishes is precisely the signal that store is specified to ignore. A
A durableGate narrows it. The failure is sequential, not adversarial —
Close, then a second connector over the same store, then a late save landing on
a lineage the new run already loaded.

Stated precisely, because an earlier version of this paragraph claimed more than
the gate delivers: the guarantee is that no save commences after Close
returns, where commencing means claiming the gate — an act that is atomic
against Close and takes no host code with it. Close does not wait, and must
not: holding the lock across CheckpointStore.Save self-deadlocks a store whose
Save calls Close (documented callable from anywhere), and a store that merely
stalls would block every Close for as long as it stalls.

So a save that claimed the gate and was then descheduled can still land after a
replacement connector loaded — the window is narrowed from [decision, write] to
[claim, write], not closed. That residual moves a checkpoint backward against
§23's "checkpoints only move forward"; its cost is bounded replay, not skipped
events. Closing it needs Close to wait (rejected above) or a fencing token on
CheckpointStore — a six-SDK contract change. Tracked in #784.

Released before the observer callbacks, because Close-from-a-callback is
documented as supported and holding the gate across host code deadlocks
Observer.Checkpoint against the very save that fired it — verified by mutation.

Also: a cancelled checkpoint load is no longer Terminal(checkpoint_load).

Blocker 7 — observers see origins only

#777's redactor applied to Observer.Gap and both CatchUpStarted sites. An
accepted 410 latches the server's resume URL as reconnect state, so the
reconnect announces its walk carrying it — redacting Gap alone would have left
the identical URL leaving through a different callback one reconnect later.

Observer.Disconnected is redacted, and this paragraph used to say the
opposite. The original reasoning — an error is opaque text, and stripping a
credential out of arbitrary text means modelling the credential, which §23's
"opaque bearer" contract forbids — argued for leaving it alone. A later round
found a ticket reaching the callback through the raw seam read error, which no
seam obligation can repair from the connector's side, so both arguments now go
through closed vocabularies: observableDisconnectReason maps every
unrecognized peer reason to "other", and observableSocketError reduces every
cause to one the connector owns, degrading anything unrecognized to
errSocketFailed.

"Reduces" is load-bearing and was the last hole: matching a sentinel is not the
same as being one, so a seam returning fmt.Errorf("read %s: %w", cableURL, context.Canceled) matched a recognized arm while its text carried the ticket.
Every arm now returns the connector's own value rather than its argument;
errors.Is still matches, and the wrapper does not survive.

The seam obligation remains on Dial, ReadFrame and WriteFrame — it is what
keeps a preserved typed error safe — but the connector no longer depends on it
being honored.

#763 — staleness arms at socket open

Observer.Connected fired between the socket opening and the window that
measures silence on it. Whatever a host's callback spent was time the window
never counted. Observed from inside the callback, which is the only place the
ordering is visible.

#760 — an occupied deferral slot no longer blinds the drain's fatal scan

drainScan returned the moment it found the slot occupied, on the reasoning
that everything queued "arrived behind this one". That is a claim about arrival
order, where §23's carve-out is a claim about which verdict governs. With
any non-fatal outcome parked ahead of it the scan looked at nothing — and the
budget it runs under is pumpDepth+1, sized in drain's own comment to reach
every frame the pump had already read. An occupied slot spent none of it.

The planned fix was a bounded deferral queue carved out of pumpDepth; that
turned out to be unnecessary.
Only one deferral is ever dispatched —
pumpExited, dispatchDisconnect and the invalid-frame teardown all end the
cycle — so a queue would be a buffer sized for a delivery that cannot happen.
The scan keeps the first outcome and discards the rest as it passes them, which
removes the capacity question entirely: no share of pumpDepth, no channel
resize, no change to the published memory bound, no fixture sweep.

The new subtest needs both halves of the two around it: deferring during the
entry poll occupies the slot, and queuing the fatal frame mid-drain puts it
where only the scan can find it. Queued earlier, the ownership cut consumes it
first — which is how the first draft passed against un-fixed code.

On #758/#759: I could not reproduce the missing-wake-source hang. Every path
leaves a wake. The overshoot is real (close to two staleness windows) and
provably cannot exceed two, so the bound is documented in place rather than
patched, and carried to bc3 as an open §23 contract question.

Verification

Pristine worktree, one pass, clean tree before and after:

  • go build / go vet / -race -count=1 / -count=5 — pass
  • make go-lint 0 issues; gosec on the CI-pinned v2.23.0 (hash-verified) 0 issues
  • full make checkexit 0
  • 22/22 fixtures
  • TestWalkFailureBetweenPages both subtests — the invariant that killed the
    reviewers' one-liner survives
  • staleness soak 8 × 500 under -race: 0 failures, 0 data races, re-earned
    because this PR rewrites the files that wait

Kill-matrix correction

Row 15's claim was inherited from the family README and is wrong; #778
corrects it. Tier 2 cannot prove zero egress to a foreign redirect target,
because the driver is the seam and manufactures the verdict. That is a
Layer-1 property, tracked for G1b.

Copilot AI balanced review requested due to automatic review settings August 12, 2026 02:40
@github-actions github-actions Bot added the go label Aug 12, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds the experimental Go Event Feed reference connector, deterministic conformance infrastructure, WebSocket transport, and checkpoint persistence.

Changes:

  • Implements the push/poll state machine, recovery, deduplication, and checkpointing.
  • Adds real and fake transports plus tier-2/tier-3 conformance tests.
  • Documents the experimental API and architecture exception.

Tip

If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

Reviewed changes

Copilot reviewed 54 out of 55 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
AGENTS.md Registers Event Feed infrastructure.
CONTRIBUTING.md Documents conformance verification.
go/README.md Adds Event Feed usage guidance.
go/go.mod Adds WebSocket dependency.
go/go.sum Locks WebSocket dependency.
eventfeed/backoff.go Implements retry timing.
eventfeed/backoff_test.go Tests retry timing.
eventfeed/cable.go Implements cable framing.
eventfeed/cable_test.go Tests cable framing.
eventfeed/catchup.go Implements catch-up and streaming.
eventfeed/catchup_test.go Tests catch-up behavior.
eventfeed/checkpoint.go Defines checkpoint contracts.
eventfeed/clock.go Implements production timers.
eventfeed/clock_test.go Tests production timers.
eventfeed/connector.go Defines the public connector.
eventfeed/connector_test.go Tests connector construction.
eventfeed/continuation.go Validates continuation URLs.
eventfeed/dedupe.go Implements delivered-ID deduplication.
eventfeed/dedupe_test.go Tests deduplication.
eventfeed/digest.go Implements filter digests.
eventfeed/digest_test.go Tests shared digest vectors.
eventfeed/doc.go Documents the package contract.
eventfeed/errors.go Defines terminal errors.
eventfeed/errors_test.go Tests error taxonomy.
eventfeed/event.go Defines feed events.
eventfeed/event_test.go Tests event decoding.
eventfeed/export_test.go Exposes test-only hooks.
eventfeed/filestore.go Implements file checkpoints.
eventfeed/filestore_test.go Tests file checkpoints.
eventfeed/filters.go Defines filter validation.
eventfeed/filters_test.go Tests filter validation.
eventfeed/loop.go Implements connector lifecycle.
eventfeed/loop_test.go Tests lifecycle behavior.
eventfeed/reconnect_test.go Tests reconnect and staleness.
eventfeed/recovery.go Implements poll recovery.
eventfeed/recovery_test.go Tests recovery paths.
eventfeed/scenario_conformance_test.go Replays conformance fixtures.
eventfeed/scenario_fixture_test.go Decodes fixture contracts.
eventfeed/scenario_harness_test.go Provides scenario harnessing.
eventfeed/scenario_selftest_test.go Tests driver strictness.
eventfeed/seams.go Defines connector seams.
eventfeed/transport.go Enforces cable URL policy.
eventfeed/transport_contract_test.go Defines transport contract tests.
eventfeed/transport_test.go Tests cable URL policy.
eventfeed/websocket_transport.go Implements WebSocket transport.
eventfeed/websocket_transport_test.go Tests real WebSocket behavior.
eventfeed/feedtest/clock.go Adds deterministic virtual time.
eventfeed/feedtest/clock_test.go Tests virtual time.
eventfeed/feedtest/minter.go Adds scripted ticket minting.
eventfeed/feedtest/minter_test.go Tests scripted minting.
eventfeed/feedtest/polls.go Adds scripted polling.
eventfeed/feedtest/polls_test.go Tests scripted polling.
eventfeed/feedtest/store.go Adds scripted checkpoints.
eventfeed/feedtest/transport.go Adds scripted cable transport.
eventfeed/feedtest/transport_test.go Tests scripted transport.
Suppressed comments (1)

go/pkg/basecamp/eventfeed/filestore.go:230

  • The rename is atomic but not durable without syncing the staged file and parent directory. After a system crash, the first checkpoint file can disappear; the next Load then reports Missing and starts at the present, which can skip history rather than merely replay from an older position. Since Save and the package advertise durable checkpointing, sync the file before rename and the directory after rename, or stop claiming crash durability and avoid treating disappearance as a safe present entry.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread go/pkg/basecamp/eventfeed/catchup.go Outdated
Comment thread go/pkg/basecamp/eventfeed/recovery.go
Comment thread go/pkg/basecamp/eventfeed/loop.go
Comment thread go/pkg/basecamp/eventfeed/feedtest/clock.go
Comment thread go/pkg/basecamp/eventfeed/filestore.go Outdated
Comment thread go/pkg/basecamp/eventfeed/connector.go Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ceb8398f4c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread go/pkg/basecamp/eventfeed/catchup.go Outdated
Comment thread go/pkg/basecamp/eventfeed/catchup.go
Comment thread go/pkg/basecamp/eventfeed/cable.go Outdated
Comment thread go/pkg/basecamp/eventfeed/catchup.go Outdated
Comment thread go/pkg/basecamp/eventfeed/loop.go
Comment thread go/pkg/basecamp/eventfeed/connector.go
Comment thread go/pkg/basecamp/eventfeed/connector.go Outdated
Comment thread go/pkg/basecamp/eventfeed/cable.go Outdated
Comment thread go/pkg/basecamp/eventfeed/recovery.go
Comment thread go/pkg/basecamp/eventfeed/checkpoint.go
Copilot AI review requested due to automatic review settings August 12, 2026 05:24

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 54 out of 55 changed files in this pull request and generated no new comments.

Suppressed comments (4)

go/pkg/basecamp/eventfeed/websocket_transport.go:73

  • The offered subprotocol is not verified after the handshake. coder/websocket v1.8.15 accepts a 101 response with an empty Sec-WebSocket-Protocol, so this can return a connection even though actioncable-v1-json was never negotiated, contrary to the CableTransport contract. Check conn.Subprotocol() and reject/close a missing selection; add a server case that intentionally selects none.
    go/pkg/basecamp/eventfeed/catchup.go:302
  • This parks an already-observed overflow until the poll returns. SPEC §23 requires semantic signals at the first consumer-context opportunity after the condition arises (with “before the next save” only as the outer bound), so a stalled poll can postpone the handler forever even though this goroutine has received the dropping frame. Dispatch the overflow immediately here; an Accept disposition can continue awaiting the poll, while Terminate should cancel the attempt/poll.
			} else if over {
				// The buffer, not the socket: the call is unaffected and is
				// awaited to completion, and the drop's disposition runs
				// before this page's position moves anything durable.
				l.deferred = &deferredFrame{item: item, overflow: true}
				if l.hooks.frameDeferred != nil {
					l.hooks.frameDeferred(true)
				}
				r := <-done
				return r.page, false, r.err

go/pkg/basecamp/eventfeed/websocket_transport.go:73

  • checkCableURL explicitly accepts case-insensitive schemes (the new unit test includes WSS://), but this passes that original spelling to coder/websocket. In v1.8.15 its handshake switch recognizes only lowercase ws/wss, so a URL accepted by policy fails as a transient dial and is retried indefinitely. Normalize only the scheme before dialing, while leaving the ticket-bearing remainder unchanged.

This issue also appears on line 70 of the same file.
go/pkg/basecamp/eventfeed/websocket_transport.go:183

  • This synchronous graceful close can block teardown for several seconds: coder/websocket v1.8.15 waits up to 5 seconds to write the close frame and another 5 seconds for the peer response. Because dispose calls this before cancelling the attempt, caller cancellation, Connector.Close, terminal outcomes, and reconnects can all stall on a live peer that ignores the handshake. Use a bounded teardown strategy that preserves the required close frame without letting the library's full handshake timeout delay the universal Closed edge.

Copilot AI review requested due to automatic review settings August 12, 2026 05:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 54 out of 55 changed files in this pull request and generated 3 comments.

Suppressed comments (1)

go/pkg/basecamp/eventfeed/catchup.go:504

  • A live event admitted by drainScan is stranded when the buffer was empty at the start of this iteration: batch stays empty, so this returns even though the scan just repopulated l.buffer. Streaming never drains that buffer, delaying the event until a later repair walk and allowing caught_up (and a held save) to happen first. Continue the loop when the scan added events.
		batch := l.buffer.take()
		if out, done := l.drainScan(at, &budget); done {
			return out, true
		}
		if len(batch) == 0 {
			return cycleOutcome{}, false

Comment thread go/pkg/basecamp/eventfeed/catchup.go Outdated
Comment thread go/pkg/basecamp/eventfeed/filestore.go
Comment thread go/pkg/basecamp/eventfeed/catchup.go

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6eb58e6010

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread go/pkg/basecamp/eventfeed/catchup.go Outdated
Comment thread go/pkg/basecamp/eventfeed/catchup.go Outdated
Comment thread go/pkg/basecamp/eventfeed/cable.go Outdated
Comment thread go/pkg/basecamp/eventfeed/websocket_transport.go Outdated
Comment thread go/pkg/basecamp/eventfeed/loop.go Outdated
Comment thread go/pkg/basecamp/eventfeed/transport.go Outdated
Comment thread go/pkg/basecamp/eventfeed/catchup.go Outdated
@jeremy

jeremy commented Aug 12, 2026

Copy link
Copy Markdown
Member Author

Round-2 note on the suppressed comments, since three of them are load-bearing and one changed the design.

catchup.go:302 (overflow parked until the poll returns) — taken, and it reframed the visible finding above it. Rather than add a third dispatch site I deleted the overflow deferral outright; admitDuringPoll now dispatches at the drop's instant. Details in the thread. Fixture 22's own description already called drop-time dispatch normative and its post-poll rendezvous "scheduling slack only", so the implementation now matches what the merged fixture says it should do.

catchup.go:504 (live event stranded by drainScan) — real, and a defect introduced by my own round-1 change: drain returned on len(batch) == 0 even when the scan had just repopulated the buffer, so the event waited for a later repair walk while caught_up and the held save went first. Now gated on l.buffer.empty() too; red proof showed [event 41 save pos-1] where [event 41 event 42 save pos-1] was required.

websocket_transport.go:73 ×2 and :183 — all three taken and in progress: the negotiated subprotocol is never verified after the handshake, a WSS://-spelled URL passes policy but the library's dial switch is lowercase-only (so it fails transient and retries forever), and the graceful close can block teardown on a peer that ignores the handshake. That last one has to keep the close frame that nine fixtures pin — an earlier round caught the opposite bug — so it is getting a bound rather than a reordering.

Thanks for putting the sharp ones in the suppressed block; they've been the most useful part of both rounds.

Copilot AI review requested due to automatic review settings August 12, 2026 06:11
@jeremy

jeremy commented Aug 12, 2026

Copy link
Copy Markdown
Member Author

Closing out the remaining suppressed comments from round 2 (02fec5f). Two fixed, one declined with evidence.

Unverified subprotocol — real, fixed. Confirmed in the library rather than assumed: coder/websocket@v1.8.15/dial.go:270-283, verifySubprotocol returns nil when Sec-WebSocket-Protocol is absent, and the server side (accept.go, selectSubprotocol) sets no header at all when it selects nothing — so a 101 that negotiated nothing yielded a live connection. A mismatched selection the library does reject, leaving the empty case as the only reachable one, exactly as you said. Now conn.Subprotocol() must match actioncable-v1-json after the handshake; on mismatch the socket is torn down and the dial fails DialPolicy. Policy rather than transient is a deliberate call: a fresh mint returns a URL pointing at the same server, which will select the same nothing, so retrying forever against a server that cannot speak the protocol is the wrong shape — the redirect refusal already lands there for the same structural reason. Test includes the server-selects-none case you asked for.

Unbounded graceful close — real, fixed. close.go:99-128,157-228: Close writes the close frame under a hardcoded 5s context, then waits another 5s for the peer, then waitGoroutines. The phases can't be bounded separately (closeHandshake is unexported, no exported close-frame writer), and CloseNow is not an escape hatch once Close is in flight — casClosing has already flipped, so it just waits too. Bound is 1s, run off-caller: only the write is contractual (§23 needs the peer to see the frame, which is why dispose closes before cancelling — this is a bound, not the reordering an earlier round rejected), and that write is a control frame to an open socket bounded by the kernel send buffer, not by the peer. All 12 fixtures carrying expectClientClose still pass. Red proof: Close blocked past 3s against a peer that never answers; green returns at exactly the 1s budget, so it passes via the timeout path rather than a lucky response.

Case-insensitive scheme — not a defect, declined. net/url.Parse lowercases the scheme before anything downstream sees it ($(go env GOROOT)/src/net/url/url.go:454), and coder/websocket's dial switch runs on u.Scheme from its own url.Parse, so a WSS:// spelling arrives there already wss. Verified empirically, not just by reading: the new test dials a WS://-spelled loopback URL and passed against un-fixed code, with the ticket-bearing remainder byte-identical. No normalization added; the test stays as a regression pin binding checkCableURL's deliberate case-insensitivity to what actually dials.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 54 out of 55 changed files in this pull request and generated no new comments.

Suppressed comments (3)

go/pkg/basecamp/eventfeed/catchup.go:158

  • A socket outcome deferred while this poll was in flight is skipped when the poll itself fails. For terminal poll branches, recoverPoll calls disposeAttempt, which clears l.deferred; for retryable failures, the deferred frame can remain undispatched through arbitrarily many retries. In particular, an invalid_event_stream_command observed during CatchingUp can be replaced by poll_failed/authorization_failed or delayed indefinitely, despite SPEC §23 requiring protocol-fatal to terminate from every socket-open state. Dispatch the already-observed socket outcome here before applying poll recovery; a failed poll has no successful page boundary left to finish.
		if p.err != nil {
			step, out, done := l.recoverPoll(at, cursor, p.err)

go/pkg/basecamp/eventfeed/transport.go:40

  • Checking u.Host does not ensure that the URL has a hostname. For example, wss://:443/cable has Host == ":443" but an empty Hostname(), so it passes the policy check and is classified as a transient dial failure instead of terminal invalid_cable_url, causing repeated re-mints/dials for a structurally unusable URL.
    go/pkg/basecamp/eventfeed/cable.go:250
  • The tier-2 push-event schema requires all nine keys, including presence-bearing visible_to_clients, and SPEC §23 treats a correlated message missing a required event key as an invalid frame. Omitting this field from the presence checks accepts both an absent value and JSON null, exposing a nil value on a push event instead of taking the socket-failure recovery edge.
		{"creator_id", p.CreatorID != nil},
		{"recording_id", p.RecordingID != nil},

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 02fec5ff1f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread go/pkg/basecamp/eventfeed/connector.go Outdated
Comment thread go/pkg/basecamp/eventfeed/catchup.go
Comment thread go/pkg/basecamp/eventfeed/cable.go Outdated
Comment thread go/pkg/basecamp/eventfeed/catchup.go Outdated
Copilot AI review requested due to automatic review settings August 12, 2026 06:41

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 54 out of 55 changed files in this pull request and generated no new comments.

Suppressed comments (1)

go/pkg/basecamp/eventfeed/catchup.go:160

  • A deferred socket outcome is skipped when the in-flight poll returns an error. recoverPoll may retry, increment authorization failures, or terminate, and disposal then clears l.deferred; this can even swallow an already-observed invalid_event_stream_command instead of producing protocol_fatal. Since no page succeeded, dispatch the deferred outcome before classifying the poll error (the finish-page ordering only applies to successful pages).
		if p.err != nil {
			step, out, done := l.recoverPoll(at, cursor, p.err)
			if done {
				return out, "", true

@jeremy

jeremy commented Aug 12, 2026

Copy link
Copy Markdown
Member Author

Holding two findings open deliberately: the in-flight-poll mechanism has drawn four rounds

Two round-4 findings — the suspendable bound in awaitSupersededPoll (thread-adjacent) and the deferred socket outcome swallowed on the poll-error path (suppressed, catchup.go:160) — are not being fixed in this round. Both are correct. I am not writing the next patch on that mechanism until its shape is settled, because the pattern is now the finding.

The ledger on one mechanism, in order:

  1. Round 1 (Codex P1): a stalled PollSource holds the consumer's goroutine when the socket dies → added awaitSupersededPoll, bounding the wait by the staleness window.
  2. Round 2 (Copilot + Codex, P1): a deferred overflow dispatches after the save, and is dropped entirely on failed-poll paths → removed the overflow deferral; admitDuringPoll dispatches at the drop's instant.
  3. Round 3 (Codex): the fatal-frame scan's budget was sized by the live-buffer capacity, so a fatal could hide behind one ping → rebounded on the pump queue's own depth.
  4. Round 4 (Codex, this round): the staleness bound from step 1 is suspendable by the very pump backpressure that creates the problem, so a misbehaving peer can keep a compliant stalled poll alive indefinitely. Plus (suppressed): the socket deferral is cleared by disposal when the poll returns an error, which can swallow an already-observed invalid_event_stream_command instead of producing protocol_fatal.

Each fix was principled and each was verified, but four rounds of edge-findings on one structure is evidence about the structure. What they all orbit: the state machine both awaits a poll seam call and services the frame queue during it, parking one out-of-band frame in a single l.deferred slot, with a bound borrowed from a timer whose evaluation the queue itself can suspend.

The concurrency is not optional — fixtures 01 and 19 require a live frame admitted after confirm but before the entry page is served, and transition 21 requires the in-flight page to be finished before the dying socket is observed. So this is not a mechanism that can simply be deleted, which is precisely why it wants a decision rather than a fifth patch. The candidate shapes:

  • (a) Patch in place — give the superseded-poll wait its own deadline from the injected clock (unsuspendable), and dispatch a deferred socket outcome before recoverPoll classifies a poll error, since the finish-page ordering only applies to pages that succeeded. Small, local, and both findings close.
  • (b) One event loop — stop blocking on the poll at all: make the poll result just another case in the same select that reads frames, so there is no deferral slot, no superseded wait, and no borrowed bound. Larger, restructures the core of catchup.go, and dissolves this whole class.

I lean (a) plus a note, because the deferral that remains is the one SPEC explicitly sanctions (transition 21's deferred consumption) and (b) risks the delivery-ordering guarantees that nine fixtures and the save-ordering invariant pin. But it is a judgment call about this PR's core, so it goes to a human rather than to me.

The other three round-4 findings (synchronous cancellation on Close, visible_to_clients required on push frames, and the drain batch escaping the live-buffer ceiling) are independent of this and are being fixed now.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a9103dd83a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread go/pkg/basecamp/eventfeed/websocket_transport.go
Comment thread go/pkg/basecamp/eventfeed/transport.go
Comment thread go/pkg/basecamp/eventfeed/loop.go Outdated
Comment thread go/pkg/basecamp/eventfeed/loop.go Outdated
Comment thread go/pkg/basecamp/eventfeed/catchup.go
Comment thread go/pkg/basecamp/eventfeed/loop.go
Copilot AI review requested due to automatic review settings August 12, 2026 07:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 54 out of 55 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

go/pkg/basecamp/eventfeed/loop.go:152

  • ctx is the attempt context, which is a child of runCtx. Connector.Close (or caller cancellation) therefore cancels the pump immediately, before the state machine can reach dispose and call conn.Close. The default WebSocket read may abort the socket on that cancellation, so normal shutdown can still produce the abrupt disconnect that the close-before-cancel ordering is meant to prevent. Give the pump a cancellation scope that is canceled only by disposal after the socket close; HTTP seam calls can continue using the attempt context.
    go/README.md:615
  • This says the connector performs no wire I/O, but the package directly performs the sanctioned WebSocket cable dial. That contradicts both the implementation and the architecture rule. Limit the claim to HTTP requests and explicitly name the cable dial exception.
**Experimental: the Layer-1 seam adapters have not landed yet.** The connector performs
no wire I/O of its own — every HTTP exchange reaches the wire through a seam backed by
a generated operation — and the adapters that build those seams over the generated
`CreateStreamTicket` and `PollEvents` operations are still to come. Until they do, a
consumer must supply the `TicketMinter` and `PollSource` implementations itself, and the
exported surface may still change as they land.

go/pkg/basecamp/eventfeed/loop.go:638

  • Cancellation is not re-checked after Load. A custom store that honors the supplied context can return context.Canceled when the caller cancels or calls Close, and this path then yields checkpoint_load even though cancellation is documented to end iteration cleanly. Match the mint/poll paths by giving runCtx cancellation precedence over the load error.

Comment thread go/pkg/basecamp/eventfeed/catchup.go Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 93a3d5a457

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread go/pkg/basecamp/eventfeed/catchup.go Outdated
Comment thread go/pkg/basecamp/eventfeed/cable.go Outdated
Comment thread go/pkg/basecamp/eventfeed/loop.go
Comment thread go/pkg/basecamp/eventfeed/loop.go

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated no new comments.

Suppressed comments (5)

go/pkg/basecamp/eventfeed/scenario_fixture_test.go:737

  • Decoder.More is not a top-level EOF check: trailing ] or } makes it return false, so malformed fixture input is accepted despite this helper's strict-decoding contract. Check the unconsumed bytes (or require a second decode to return io.EOF) instead.
	if dec.More() {
		return fmt.Errorf("trailing JSON content")

go/pkg/basecamp/eventfeed/connector.go:535

  • The restriction is broader than consumer callbacks. TicketMinter and CheckpointStore.Load/Save are also invoked synchronously on the run goroutine, so calling Wait from one of those implementations waits for itself; consequently “safe from anywhere ELSE” is incorrect and can lead custom seam authors into a deadlock.
// It is not callable from a consumer callback. Every callback — an observer, a
// signal handler, the loop body — runs ON the run goroutine, so waiting for
// that goroutine from inside one waits for itself. Close is the call that is
// safe from anywhere; this is the one that is safe from anywhere ELSE.

SPEC.md:2518

  • This constraint must cover all code running synchronously on the connector's execution context, not only consumer callbacks. For example, a custom checkpoint store's Save runs there too and self-deadlocks if it calls wait(); the cross-language contract should state the broader restriction.
  `wait()` is not callable from a consumer callback, for the reason `close()` does not wait.

go/pkg/basecamp/eventfeed/scenario_conformance_test.go:15

  • This header contradicts both the implementation and the new README disclosure: outbound frames (and saves outside the delivery witness) are queued and can satisfy a later expect step, so they are not arrival-strict. Please describe the tracked gap accurately here rather than claiming the driver follows the rule exactly.
// Strictness follows the family README's per-action-class rules exactly.
// Checkpoint saves and outbound frames are ARRIVAL-STRICT: a save carries the

go/pkg/basecamp/eventfeed/loop.go:1528

  • The PR description still says a durableGate prevents saves from commencing after Close, but this implementation deliberately removed that gate, permits an accepted save after Close, and uses Wait as the quiescence boundary. Please update the description (and the #784 status it cites) so reviewers and consumers are evaluating the lifecycle contract this diff actually implements.
	// A position that reached here was ACCEPTED, and its events were delivered
	// to the consumer before it was. It is written unconditionally, including
	// after Close: the store is what tells the next run where those deliveries
	// stopped, and dropping the write because the consumer closed a moment
	// earlier silently re-delivers them. Ordering a second connector against

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e2cb78a250

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread go/pkg/basecamp/eventfeed/catchup.go
Comment thread go/pkg/basecamp/eventfeed/loop.go
jeremy added 18 commits August 19, 2026 15:19
…2 driver

The other half of the connector, stacked on the foundations PR. Everything here
is the running machine and its harness:

  connector.go    New, the options, Events, Close — the consumer surface
  loop.go         the state machine: states, transitions, timers, live buffer
  catchup.go      the poll walk, its page boundary, and the drain
  recovery.go     the 400/409/410 recovery matrix
  scenario_*      the tier-2 conformance driver, its fixture model, harness,
                  and self-tests
  export_test.go  the bridge from the external test package to the internal
                  observation points

Rebasing #705's 29 commits onto the foundations PR was attempted and abandoned:
8 of them touch ONLY foundation files and would replay empty, and 21 of the
remaining 22 mix both halves, so every one conflicts against the four fixes the
foundations PR applies (the credential boundary, the dedupe contract, the file
store, the redactor). Resolving 22 interleaved conflicts by hand is a worse
guarantee than re-cutting the tree, which is byte-exact by construction: the
foundation paths come from that PR's tip and these 17 from #705's.

The development history is not lost. It is tagged `pre-split/705-head` at
1646f2c1f, which is what the SHAs cited in #705's review threads still resolve
against, and the per-file reasoning it carries is unchanged in the code and its
comments.
Daybreak blocker 2. The walk took page.Position on trust, and an empty one
silently skipped history in two different ways depending on the entry class.

Position-resume: acceptPosition("") sets l.position = "", and entryCursor
selects on `l.position != ""` — so it does not preserve the old cursor, it falls
through to the StartResume default, which is a BARE PRESENT ENTRY. The feed
resumes at the server's head with everything between the real position and now
skipped, and reports nothing.

Present-class is worse. `held` uses "" as its sentinel for "the final entry was
not present-class", so an empty position is not saved as empty — it collapses
into the sentinel, `held != ""` skips acceptPosition AND saveCheckpoint
outright, and caught_up is announced anyway. The entry position is DISCARDED,
with the drain's deliveries already handed to the consumer and nothing durable
recording them.

Both are the same silent skip that a found-but-empty STORED position is already
refused for at load (d379f2e), arriving by the other door. The guard is the
same shape and in the same spirit, and it also makes saveCheckpoint's stated
contract true: it documents that "the live cursor is neither regressed nor
blanked", which the assignment one line above its call site did.

Terminal(poll_failed) is what §23 already classifies "an unexpected shape" from
the poll lane as, and it is not retried — a server that answered without a
position answers the same way to the same request.

The refusal precedes delivery, the counter resets, and every mutation of the
walk's state. That placement is the part worth pinning, so it is what the
mutation check exercises: moving the guard to AFTER the delivery loop fails
three of the four subtests on `delivered ids = [101], want []`, while the
"is not retried" subtest still passes — the subtests discriminate different
properties rather than all keying on one.

Removing the guard entirely fails all four, but by watchdog timeout rather than
on an assertion, and that is worth stating plainly: the un-fixed feed does not
misbehave visibly, it proceeds into Streaming as though nothing happened. That
IS the defect.
…s only

Two independent items, both about what leaves the connector and when.

#763 — Observer.Connected fired between the socket opening and the window that
measures silence on it. Observer callbacks are host code on the consumer's
goroutine (a log write, a metrics emission, an error-tracker breadcrumb), and
newStaleHolder reads the window's origin at construction — so whatever the
callback spent was time the window never counted, and a peer that went silent
immediately got that much extra grace. §23 says the window arms at socket open;
now it does. Starting the pump ahead of the callback is the same trade and
harmless: it only fills a bounded channel this goroutine drains later.

The test observes the ordering from INSIDE the callback, which is the only place
it is visible — asserting on the timer set afterwards passes either way, because
the set is identical and only its origin moves. Restoring the old order fails it
with `timers armed when Connected fired = [handshake-deadline]`.

Blocker 7 — the redactor from the foundations PR is applied to both URL-bearing
observer surfaces. Observer.Gap carried the server's resume URL whole, and
Cursor.PageURL carries the same URL at a walk announcement: an accepted 410
latches it as reconnect state (deliberately, so a socket torn down before the
resume page cannot re-select the cursor the server already refused), so the
reconnect announces its walk carrying it. Redacting Gap alone would have left
the identical URL leaving through a different callback one reconnect later.

The handler still receives the URL WHOLE and the resume poll still follows it
whole, and both tests assert that pairing — a test checking only the observer
would be satisfied by redacting both, which would break the disposition.

Reaching the CatchUpStarted leak took the one sequence that keeps the latch
alive. Dropping the resume poll's socket does not: transition 21 observes that
at the page boundary, AFTER the resume page is accepted and saved, which
releases the latch and reconnects at pos-1. Written that way first, it proved
nothing; PollUnauthorized is what rides the reconnect cycle with the latch
intact. The first version also raced the second walk's announcement, which -race
reported.

Observer.Disconnected(err) was audited and deliberately NOT redacted. An error
is opaque text, and stripping a credential out of arbitrary text means modelling
the credential — the one thing §23's "opaque bearer" contract forbids, and the
trap dialFailure's own comment documents three review rounds of. Every error the
connector puts there is its own sentinel or a seam error, and the built-in
transport draws its causes from a closed vocabulary.

The audit did find a real gap, and it is a contract gap rather than a code one:
DialError.Err is supplied by the CableTransport and concatenated into Error(),
and CableTransport is a documented extension point whose seam stated no
obligation about the URL. So the obligation is now stated where an implementer
reads it — on Dial, ReadFrame and WriteFrame — and Observer.Disconnected's doc
says plainly that err is unredacted and why. Stating it on the seam is the only
place it can hold; a guard here could not see inside an opaque error.
…e promise

Daybreak blocker 6. Close's doc claimed "no seam call (mint, dial, poll) and no
delivery can BEGIN after Close has returned". That is not true and cannot be
made true: the run goroutine checks its context at each dispatch point and then
acts, so a Close landing between a check and the call it guards cannot stop the
call from starting — only from starting on a live context. Closing that window
would mean holding a lock across every seam call, i.e. across arbitrary host
code, trading a benign race for a deadlock reachable from any callback.

So the promise is narrowed to what holds: cancellation is visible before Close
returns, and anything that begins afterwards begins on an already-cancelled
context. For reads that is self-limiting — the call returns promptly, its result
is discarded, the run takes the Closed edge.

One effect is not self-limiting, and it is the one now ORDERED. A checkpoint
save is the connector's only effect that outlives the process, and cancellation
cannot stop it: a CheckpointStore is entitled to ignore ctx, and the built-in
FileCheckpointStore documents that it does, deliberately, so a position already
accepted is not dropped by a shutdown race. The cancelled context Close
publishes is precisely the signal that store is specified not to act on.

durableGate closes it. A save holds the gate across the store call; Close
latches the gate, waiting for at most one in-flight write. Either a save
commenced before Close (and Close waits for it, which is right — its position
was accepted and delivered before Close, and abandoning a durable write
half-done is worse) or it does not commence at all. The failure that motivates
it is sequential, not adversarial: Close, then a second connector over the same
store, then a save the first had not yet begun landing on a lineage the new run
has already loaded.

It is a second lock, not mu, so a host's write cannot hold up a concurrent
Close caller's cheap serialization; and it is released BEFORE either observer
callback, because Close from inside a callback is documented as supported and
holding the gate across host code would deadlock Observer.Checkpoint against the
very save that fired it. That hazard is real, not theoretical: holding the gate
across the callbacks deadlocks the Checkpoint subtest, and Go's timeout names it.

Third item: a checkpoint LOAD that failed because the run was cancelled is no
longer Terminal(checkpoint_load). The load happens on the first iteration and
before the first mint, which is exactly the window a prompt Close lands in, and
diagnosing the consumer's store for the consumer's own shutdown is wrong — §23
ends a closed iterator with no error element. Classified on the CONTEXT, not the
error's shape, because a store is under no obligation to wrap ctx.Err().

The first version of that test closed BEFORE the run started, which never
reaches the store at all — Events takes the isClosed latch with zero wire
attempts — so it passed with the classification deleted. It now parks the load
in flight and lands Close on it, and the paired test keeps an UNCANCELLED load
failure terminal, so the guard cannot be widened into swallowing the real edge.
…tal scan

#760, and the answer turned out to be removing a mechanism rather than adding
one.

drainScan returned the moment it found the deferral slot occupied, on the
reasoning that "everything the pump has queued arrived behind this one, so the
scan stops here either way". That is a claim about arrival ORDER, where §23's
carve-out is a claim about which VERDICT governs: a raw
invalid_event_stream_command observed during Draining is
Terminal(protocol_fatal) with the drain not completed, the held position not
saved and no caught_up. With any non-fatal outcome parked ahead of it the scan
looked at nothing, the drain completed, the position saved and caught_up
announced — all three of the things the carve-out exists to prevent.

The budget that scan runs under is pumpDepth+1, sized in drain's own comment
for the express purpose of reaching every frame the pump had already read. An
occupied slot spent NONE of it.

rev 15 selected a bounded deferral queue carved out of pumpDepth for this, and
that is not needed. Reading the dispatch path, only ONE deferral is ever
dispatched: dispatchDeferred sends it to pumpExited, dispatchDisconnect, or the
invalid-frame teardown, and every one ends the cycle. A second entry could
never be reached, so a queue would be a buffer sized for a delivery that cannot
happen. The scan keeps the FIRST outcome — which is what preserves arrival
order for the one that is reported — and discards the rest as it passes them.

That removes the whole capacity question with it. Nothing new is retained, so
the published memory bound ((pump depth + liveBufferCapacity) × MAX_FRAME_BYTES)
is untouched, pumpDepth is untouched, and so is the depth at which the pump
blocks — no fixture sweep, no §23 restatement, no ExportPumpDepth change.

The new subtest needs both halves of the two around it and neither alone
reaches the defect: deferring during the entry poll is what occupies the slot,
and queuing the fatal frame MID-DRAIN is what puts it somewhere only the scan
can find. Queued any earlier the ownership cut consumes it first and the
carve-out fires without the scan being involved — which is how the first draft
passed against un-fixed code.

#758/#759, the bounded wait: no code change, and the reason is documented in
place. The overshoot is real — a suspended firing re-arms a full window, so a
deferral landing just after a frame waits close to two windows — but it cannot
exceed two, because the post-select predicate runs on every wake and the first
re-armed firing is necessarily past a deadline set one window after the
deferral. A shorter re-armed window would be a staleness window that is not
one, and a dedicated timer is the seventh kind §23 pins against, so the bound
is stated rather than patched.

I could not reproduce rev 15's missing-wake-source hang. Every path leaves one:
a suspended firing always re-arms, and a latched authoritative expiry is
reported by evaluate before any wait can consume its timer. The overshoot is
what is real, and it is now published as the two-window worst case — carried to
bc3 as an open §23 contract question rather than asserted as settled here.
Review of 4cff076 (B1 excluded, since uncommitted at the time). All four P1s
reproduce; each fix is red-proven against the reported shape.

P1 — Observer.Disconnected still leaked ticket text. Both arguments carry
peer-controlled strings: a raw disconnect frame's reason, and a WebSocket close
reason rendered through the error. Both were BOUNDED by §9's cap, which limits
how much of a credential escapes rather than whether any does — the identical
trap dialFailure documents three review rounds of. The cable server is exactly
the party that knows the ticket: it was dialed with it.

Both now go through closed vocabularies. observableDisconnectReason keeps the
two reasons that change behavior and reports everything else as "other";
observableSocketError passes the connector's own sentinels and typed errors and
degrades anything from a seam to a generic cause. CloseError.Error() renders
only the code — an integer cannot carry a credential, and RFC 6455 codes are
what an operator classifies on; Reason stays a readable FIELD.

A canary planting a ticket in every peer-controlled teardown string found MORE
than was reported: raw seam read errors leak too, which seam documentation
cannot repair because the connector forwarded them verbatim. Four arms, all
red before and green after.

P1 — durableGate deadlocked reentrantly and blocked Close. It held the lock
across CheckpointStore.Save while Close waited for it: a store whose Save calls
Close self-deadlocks on the caller's own goroutine, and a merely stalled store
blocked EVERY Close indefinitely — contradicting the one thing Close promises
unconditionally. The two promises could not coexist, so the waiting one is
dropped: the gate is claimed and released atomically, Close latches and
returns, and a save that already claimed still completes. The guarantee is
unchanged in substance — no save COMMENCES after Close returns — with
commencing defined as claiming the gate, which takes no host code with it. The
old test asserted Close WAITS and is replaced by one asserting it does not; the
gate-holding variant deadlocks the new test at 40s.

P1 — Close precedence, reopened by #763. Arming staleness before Connected also
starts the pump before it, so a fatal frame can already be queued when a
Connected callback calls Close, leaving two ready select cases. Reproduced:
25/50 rounds emitted a terminal element after Close returned. Fixed at the ONE
exit (emitTerminal) rather than per-select — many selects, one exit, and a rule
every future select must remember is what produced this.

P1 — B2 discarded an earlier socket verdict. A deferred protocol-fatal followed
by a positionless page took poll_failed, because disposal clears the deferral.
The failed-poll branch already dispatches the deferral first, with a comment
giving this exact reason; the new guard did not follow it. Now it does.

Also fixed: TestNoCheckpointSaveCommencesAfterClose was vacuous (it closed
before the run reached a page) and now closes from Observer.PageDelivered, the
callback immediately preceding the save; the cancellation check after the
checkpoint load covers every result rather than only the failure, since a
found-empty result became terminal and a successful one let the run fire
Connecting after Close; and deliver()'s stale "no delivery begins after Close
returns" claim is corrected in place — it is a check-then-act, and the honest
guarantee is the one Close states.

Two of these touch foundations files that belong to #777 — CloseError in
seams.go and its test. They stay here because the leak is only observable
through the loop's observer path, which is this PR's, and the canary that
proves it lives here. TestCloseError_Message is INVERTED, not adjusted: it
required Error() to render the peer's reason, so it pinned the wrong contract.

Verified: build, vet, -race, 22/22 fixtures, go-lint 0 issues, gosec 0 issues.
All three are documentation-accuracy fixes with no behaviour change; the
verification is that each comment now matches the code beside it.

Observer.Disconnected still promised that err is "passed through UNREDACTED".
That stopped being true when the ticket-leak fix routed both arguments through
closed vocabularies: observableDisconnectReason collapses every unrecognized
peer reason to "other", and observableSocketError replaces every unrecognized
error with errSocketFailed. The old text told a custom-transport author their
diagnostics reach the observer, which is exactly backwards -- so the corollary
is now stated outright rather than left to be discovered by an author whose
error vanished.

go/README.md claimed the connector "performs no wire I/O of its own" two
sentences after describing the Action Cable subscription it dials. AGENTS.md
and doc.go both name that dial as the one sanctioned non-HTTP wire act;
doc.go's version of the sentence already carried the qualification and the
README's did not. Scoped to HTTP API I/O, with the dial named.

durableGate's opening paragraph promises to prevent a replacement connector
overwriting a lineage, and "begun" there is the gate's term of art for
claiming the gate -- defined twenty lines below the sentence that leans on it.
A reviewer read it the other way, which is the evidence it needed saying. The
residual window is real and narrowed rather than closed: claim and store call
are not atomic, so a descheduled save can still land behind a replacement and
move a checkpoint backward. Closing it needs Close to wait (a deadlock from
any callback) or a fencing token on CheckpointStore (a six-SDK contract
change), so it is stated here and tracked in #784.
observableSocketError exists so the connector does not DEPEND on a custom
transport honoring the seam's no-cable-URL obligation. For four of its arms it
did depend on exactly that.

Matching a sentinel is not the same as being one. errors.Is walks the chain and
reports; it does not extract. So a transport returning

    fmt.Errorf("read %s: %w", cableURL, context.Canceled)

matched the recognized arm, and returning the argument handed the wrapper --
ticket and all -- to Observer.Disconnected, which is a logging surface. The
credential rides in the URL the peer was dialed with, so the one place text is
most likely to be annotated is the one place the vocabulary was open.

The typed arms below were already safe by a different mechanism: errors.As
assigns the inner value, so returning it drops the wrapper. That asymmetry is
why the two halves look different and must, and the test pins the As half too --
a refactor unifying both onto errors.Is would silently reopen this.

Each arm now returns the canonical sentinel rather than its argument. errors.Is
still matches for a consumer that branches on it; what does not survive is the
wrapper's text.

Red-proven with only the switch reverted: all four sentinel cases FAIL with the
literal ticket in the message, while the wrapped-CloseError case PASSES against
un-fixed code -- so the test is not vacuously red, and it discriminates the two
mechanisms rather than asserting a blanket property.

Also corrected, wrong in the same direction: Observer.Disconnected's godoc had
just been rewritten to claim the vocabulary needs nothing of a custom transport,
and the PR description still said the callback was "audited and deliberately not
redacted". Both now describe what the code does.
CableTransport.Dial told implementers its returned error "reaches
Observer.Disconnected, which hosts log". It does not, and cannot: a dial
failure has no socket, so there is no teardown to report. DialPolicy becomes
Terminal(invalid_cable_url); every other kind takes transition 7 to backoff,
where the connector reports the classification and drops the cause.

Left alone, that reads as a licence to relax -- the callback never fires, so
why redact? The real exposure runs the other way and is stronger. A DialPolicy
error is yielded as the ITERATION's terminal error: the consumer's own error
value, which they cannot decline to receive, where a callback is something they
may never have registered. So the no-credential obligation is not softened by
the correction, it is what keeps a ticket out of the error a caller gets back
from Events.

ReadFrame's identically worded claim is left as it stands, because for a read
error it is true.

Fourth of these today -- prose that was accurate when written and falsified by
a later, correct change. Noted on the PR as a pattern rather than filed: this
package states unusually precise contracts in comments and nothing checks them,
so each correct fix quietly ages the paragraph above it.
… enforces

SPEC §23 gives the feed exactly one final error element, and that element ends
iteration. The driver checked neither half. It recorded a terminal and kept
ranging, overwriting `terminal` each time, so three non-conforming shapes
passed every terminal fixture identically to a compliant connector:

  - yielding the expected terminal reason TWICE (the second overwrote the
    first with an equal value, so every later assertion still passed);
  - delivering an event AFTER the error element;
  - never ending iteration at all -- the deferred cancel ended the range
    afterwards, so nothing downstream could tell an iteration that ended by
    itself from one that had to be cancelled.

None of the three is observable from an end-state assertion, which is why the
gap survived: the fixtures in the tree pass either way. A test that cannot
fail the contract it claims to enforce is worse than no test, because it is
counted as coverage.

The count and the post-terminal delivery are recorded as violations, which
block every subsequent await, so a scenario fails at its next step rather than
silently at the end. The first element is retained rather than the latest --
"exactly one final element" means the first one is the real one. Requiring the
iteration to have ended is a driver-side await, bounded by the same watchdog
as every other rendezvous, so a connector that stays open fails instead of
hanging.

Red-proven per case: with the three guards reverted, all three hostile shapes
report "the driver accepted ..." and the compliant shape still passes, so the
probes discriminate rather than failing on everything. All three were
reachable; none had to be dropped as unreachable.

Found by Copilot as a suppressed comment, which is the second time today that
block carried the sharper finding.
…'s to write

`observableSocketError` exists so that `Observer.Disconnected` — a logging
surface whose destination the connector knows nothing about — carries only
values the connector authored. Recognition is by type, which is only as strong
as the type's authorship, and two arms failed that test.

`*DialError` and `*TerminalError` are both exported with exported fields, and
both render free text: DialError its `Reason` plus `Err.Error()`, TerminalError
its `Msg` plus `Err.Error()`. `CableTransport` is a documented extension point,
so a `CableConn.ReadFrame` returning `&TerminalError{Msg: cableURL}` reached the
callback verbatim — the ticket rides in that URL's query string, and §23 calls
the ticket an opaque bearer credential that is never logged. Neither passes
through now.

The three arms that remain are safe by construction rather than by convention:
the sentinels are package-level values whose text is fixed in this file,
`*CloseError` renders an integer code alone, and `*invalidFrameError` renders
one of two shape constants. Nothing is lost on the live path — dial failures
never reach this function, and a terminated feed's reason and detail arrive
through the iteration's terminal element, which is the semantic channel and
keeps its SPEC-mandated wording. `seams.go`'s contract said the removed types
were preserved; it now states the closed vocabulary and points at the terminal
element for the rest.

The canary arm that should have caught this was a false negative three times
over, and each fault alone was enough. `{"type":"message",...}` is not a
correlated broadcast — that shape is the TYPELESS one — so it never reached
`decodeMessageEvent`. The payload was truncated JSON, so it failed the envelope
unmarshal first and the payload was never read. And the canary sat in
`identifier`, which nothing renders. It now serves a well-formed typeless
broadcast under the connector's own identifier with the canary in `created_at`,
the one field whose decoder quotes its input back.

Four arms added for the errors a seam can author, at both seams that produce
them: ReadFrame for the two types, and WriteFrame for the subscribe write —
which had to be armed before welcome, since that is when the write happens.

The assertions gained an identity check, because "the canary is absent" is
satisfied by a callback firing with a nil error, which is not the guarantee.
Mutation confirms the whole set: restoring the two arms fails nine cases, and
three of them fail ONLY on the identity assertion.
#760 established that a protocol-fatal disconnect the pump has ALREADY READ
must outrank whatever non-fatal outcome happens to be parked ahead of it, and
fixed that inside the drain. The obligation is not the drain's. SPEC §23 says
so twice — "terminal from EVERY socket-open state", and "a raw
invalid_event_stream_command frame READ in AwaitingWelcome, CatchingUp, or
Draining ... never Backoff. An explicitly non-retryable protocol rejection must
not reconnect from any state." READ is the boundary the obligation attaches to.

The walk has four points where it disposes of a deferred receive, and all four
dispatched with the pump's queue never inspected. A `remote` disconnect in the
slot — recoverable, arrived first — beat a fatal sitting right behind it: the
cycle ended as an ordinary socket failure and the connector re-minted against a
server that had already refused the command. Confirmed at every boundary before
the fix, by mint count rather than by terminal reason alone, since a connector
that reconnects once and then terminates on the re-issued frame still ends at
protocol_fatal.

`drainScan` becomes `fatalScan`, called at all four boundaries through
`probeFatal` with a fresh pumpDepth+1 budget. That bound is the drain's, and for
the drain's reason: the queue holds at most pumpDepth, the pump may be blocked
in hand-off with one more, and the ownership cut's liveBufferCapacity is
caller-configurable down to 1 — one queued ping under
WithLiveBufferCapacity(1) spends an entire admission pass, which is what the
page-boundary case reproduces.

What the scan does with what it dequeues is what makes it safe mid-walk: live
events are admitted to the buffer, which is what CatchingUp does with them
anyway; the FIRST non-fatal socket outcome goes to the same single slot the
caller is about to dispatch from, preserving arrival order for the outcome
actually reported; and later ones are discarded, which is all that could happen
to them, since every dispatch of a socket outcome ends the cycle.

One case per boundary, and the mutation matrix is a clean diagonal: removing
the probe at any one site fails exactly its own case and no other. Two of the
four needed the case built for them — the page boundary only exposes its probe
under a buffer capacity of 1 with a frame wedged ahead of the fatal, and the
superseded boundary needs the call abandoned rather than returned.

TestSupersededPollBoundSurvivesPumpBackPressure needed the harness's
frame-handled drain that the blocked-hand-off test already carries: the
notification send is blocking and a probe handles a whole queue in one pass, so
the loop wedged in the test's own callback. That inline workaround is now a
harness helper, since any test that fills the queue needs it.
… guarantee

`durableGate` is deleted, and `Connector.Wait` replaces what it was reaching
for.

The gate refused a checkpoint save once Close had latched. Its own doc states
the residual honestly: claiming the gate and writing are not atomic together,
so the window in which a replacement connector's newer position gets
overwritten is NARROWED, from [decision, write] to [claim, write], not closed.
Weigh that against what it cost, which the test pinning it spelled out without
naming it as a cost. Close taken from Observer.PageDelivered — a callback that
fires immediately after the page's events reach the consumer and immediately
before its position is written — meant that position was never persisted. The
next run re-delivers those events with no record they ever arrived.

So: a bounded-replay harm avoided some of the time, paid for with a
bounded-replay harm caused every time. And it prevented at most about one save
anyway, since runCtx is cancelled synchronously and the walk's dispatch points
unwind almost immediately.

The guarantee it wanted is free. The iterator returning IS quiescence — the run
has exited, so no save can be in flight, by construction rather than by a window
narrow enough to be unlikely. `Wait` exposes that to a caller who closes from a
goroutine that does not own the range loop; a caller that owns the loop already
has it. The contract is now "await termination — or Wait — before opening a
second connector over the same store", which drops the term of art ("commence")
that two reviewers independently read as a stronger promise than it was.
Closes #784.

The second half is Close precedence, which `emitTerminal` had established for
the terminal element and nothing else. Every lifecycle announcement kept firing
after a Close, because the nearest runCtx check sat AFTER the callback — placed
to catch a Close taken from INSIDE it, which is a different Close from one taken
earlier. Five checks now, each with a case that only it can satisfy: before
`connecting`, `catch_up_started` on both the post-confirmation and repair paths,
`caught_up`, the ownership cut, and socketCheck.

Two of those are about announcements the walk makes on the way past. The
ownership cut and socketCheck both DISPATCH what they dequeue, so a disconnect
frame queued when the consumer closes was announced through
Observer.Disconnected — a teardown report for a socket the consumer had already
stopped caring about.

Ordering within the page boundary is deliberate and runs the other way from the
rest: the page's save happens BEFORE the check, its announcements after. A check
placed before the save would reintroduce exactly the dropped save the gate was
deleted for. Finish the page, then observe the close.

Getting the mutation matrix to a clean diagonal took three corrections worth
recording. The `caught_up` check and the page boundary's masked each other under
a position-resume entry, so the case that isolates `caught_up` had to use a
PRESENT-class entry, where the held save fires after the walk has returned. The
socketCheck case was driving the ownership cut instead — the default harness
resolves to present-class — and claimed in its name to be about the other. And
its hand-off wait was satisfied by a stale token from the handshake, so Close
landed before the frame was ever queued and the code under test had nothing to
find.

One check has no test and says so: the repair path's, which backstops a select
in `stream` where `runCtx.Done()` and `repair.C()` can both be ready and Go
chooses. That is the same race `emitTerminal` documents, and it cannot be driven
deterministically from outside.
…ontract

`wait()` is a public act with a cross-SDK obligation, not a Go convenience, so
§23's Consumer Surface states it rather than leaving the other five SDKs to
infer it from the Go source.

What the section said about `close()` was true and incomplete in the same
breath: it abandons, it is callable from anywhere — and it silently implied
that closing was enough to hand a checkpoint store to a replacement connector.
It is not, and no gate inside `close()` can make it so. So the section now says
both halves: a save decided just before the close may be written just after,
deliberately, because its events were already delivered; and `wait()` is the
quiescence point, with the iteration terminating being the same guarantee for a
consumer that owns the loop.

The naming note says what has to exist rather than what has to be added. A
language whose streaming idiom already exposes the run's completion — a Kotlin
Job, a Swift Task — has `wait()` already; the requirement is that observing the
run's exit be possible WITHOUT `close()`, since `close()` is the one call that
cannot wait.
…es not have

Copilot's two suppressed comments on this PR are right. The README states,
normatively and for all six drivers, that checkpoint saves and outbound frames
are ARRIVAL-strict — "observed while the current step is anything other than
their matching expect step, the scenario fails immediately." The Go reference
driver appends both to queues and lets the matching expect step consume
whatever is there, with no record of which step was active on arrival and no
field to record it against. That is the order-strict rule, which is what the
parked classes get.

The rule is not weakened to match the implementation, because the rule is
normative for five drivers not yet written and this is one implementation
behind it. What the README now carries is the gap itself, sized: early-SAVE
ordering is not at risk — every save records the delivery count it arrived at,
and fixture 12 fails a save that precedes its page's deliveries — so the
residue is an outbound frame written earlier than its step, concretely a
subscribe command sent at dial rather than on welcome.

Not patched here because the fix is not the queue check it looks like. The
rule's own second sentence is the hard part: an action that becomes legal the
instant a rendezvous is satisfied must match the NEXT step rather than fail
against the stale one, so the handoff between adjacent steps has to be atomic
against the harness mutex. A naive active-step guard fails exactly there and
would turn every rendezvous-adjacent action into a false failure. Filed as #792
with the red proof it needs: a connector that subscribes at dial passes all 23
fixtures today.
Four defects from daybreak's review, each one a place where an argument I made
correctly in one spot was not carried to the next.

**A seam-authored DialPolicy verdict reached the terminal element.** The
previous commit established that *DialError and *TerminalError are exported,
seam-constructible, and render free text, and used that to strip them from
Observer.Disconnected — then exempted the terminal channel on the grounds that
"terminal elements keep their SPEC-mandated text". That is true of the one case
§23 actually mandates, filter_invalid, and false as a blanket. A DialPolicy
error comes out of CableTransport.Dial, a documented extension point, and its
Reason and cause went verbatim into the terminal's Msg and chain. The KIND is
still read; the text is now the connector's own and no cause is retained. The
connector's own pre-check keeps its text and is not the same case: its reasons
are written in transport.go, and its two interpolations are a URL scheme and a
port, which url.Parse constrains to alphanumerics and digits.

**CloseError passed through with its peer Reason intact.** The comment said
"renders only its code — an integer cannot carry a credential", which is true
of Error() and irrelevant to the struct: Reason is exported and held the peer's
string, so %+v, a structured logger, or ce.Reason put it on the surface the
closed vocabulary exists to protect. Rendering is not the only way out of a
struct. It is rebuilt code-only now.

**The accepted-page save was handed a cancelled context.** Deleting the durable
gate was supposed to make an accepted position durable even when Close lands at
the save boundary. It passed l.runCtx, which Close cancels synchronously, so a
store that HONORS its context — which CheckpointStore permits and says nothing
against — returns ctx.Err() and the position is lost, exactly as under the gate.
Both stores this package ships ignore ctx, which is why every test passed: the
guarantee was tested only against implementations that could not fail it.
context.WithoutCancel keeps the run's values and drops the cancellation. The
trade is named in the code: a store that blocks forever now blocks the run's
exit rather than being released by Close, which is the lesser harm, and the
narrower one, since a store may already ignore ctx entirely.

**The fatal scan could sample inside the pump's read window.** The carve-out is
stated over "every frame the pump had ALREADY READ", and pumpDepth+1 delivers
that only for a frame blocked inside the hand-off with a full queue. Between
ReadFrame returning and the send starting, the frame is invisible: the scan
sees an empty queue, completes the drain, saves the held position and announces
caught_up with the fatal sitting in the pump's stack. Red-proven with a new
pumpRead hook that parks the pump in exactly that window — checkpoint ledger
[pos-1] where the carve-out requires none.

Peeking at what the pump holds would move the race one instruction earlier
rather than close it, so the scan's completion condition changed instead: an
empty queue ends the scan only when the pump also holds nothing it has read.
That second half is an atomic flag covering a stretch with no I/O and no
blocking except the hand-off, which the scan's own dequeue releases, so the
spin terminates and the budget bounds it regardless.

Mutation: all four die, each on its own test and no other. The fourth needed
its mutant rewritten — removing the guard alone left `runtime` unused, and a
vet failure is not a kill.
§23 said a save decided just before close "may still be written just after"
because "a store is entitled to ignore the cancelled context" — which describes
the accident that made the Go implementation work rather than the rule the
other five SDKs need. A store that HONORS its context is equally compliant, and
against one the guarantee evaporates: the position for events already delivered
is dropped, silently, and the next run re-delivers them.

So the rule is now the detached context, stated with its trade: values carried,
cancellation dropped, and a store that blocks indefinitely delays the run's
exit — and `wait()` — instead of being released by `close()`. Bounded by the
store's own behavior, against re-delivery that is bounded by nothing.
…ary moves

The previous commit claimed the fatal scan covered "every frame the pump had
ALREADY READ" and added an atomic flag to deliver it. daybreak is right that it
does not: the flag is published AFTER the read returns, leaving a pre-publish
window, and the scan loads it AFTER its own empty select, leaving a second
window in which the frame lands and the flag clears between the two reads. Two
samples are not a happens-before, and the hook I added to test it sat after the
store, so it excluded both interleavings it was supposed to catch.

Closing the window for real needs the read to complete inside a critical
section the scan can enter. The read blocks indefinitely on a quiet socket, so
that lock deadlocks the drain against a peer that stopped talking — which is
why this is a narrowed boundary rather than a third sample.

So the flag, its hook, and its test are gone, and §23 now states what every
implementation must actually cover: everything HANDED to the state machine,
plus the one frame a blocked hand-off is holding. "Observed" was always the
spec's word; the reader is a single goroutine, so the plus-one is exactly one,
and it is why the budget is pump depth + 1. The test drives the guarantee
through the hand-off rendezvous rather than through a window no scan can see.

Two more from the same review, both current-head:

A dial returning (conn, err) leaked the connection — legal in Go, and easy for
a transport that fails mid-handshake — which under the reconnect cycle is a
leak per backoff round. It is closed before the error is classified. A (nil,
nil) return now takes the transient failure edge instead of installing a nil
CableConn and panicking on the first read.

The detached save's test could not tell WithoutCancel from Background: both
survive cancellation, and only the first keeps the caller's context values — a
trace span, a tenant, a request id. The store now records what it saw and the
test asserts it, which kills the Background mutant for that reason alone.

doc.go still said FOUNDATIONS ONLY with no Connector and no run loop. It names
the one piece genuinely outstanding instead: the Layer-1 adapters, and the two
obligations that ride on them rather than on anything here.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated no new comments.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9fe0a8798a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread go/pkg/basecamp/eventfeed/loop.go
Comment thread go/pkg/basecamp/eventfeed/loop.go
jeremy added 3 commits August 19, 2026 22:45
…rival-strict

`conformance/event-feed/README.md` binds every driver to one rule: a checkpoint
save or an outbound frame observed while the current step is anything other
than its matching expect step fails the scenario there and then. The Go
reference driver did not do that. `Save` appended to a queue, `readPeer`
appended to another, and the matching expect step later consumed whatever was
sitting there — so an action emitted during an earlier, non-matching step still
satisfied its expectation. There was no active-step field to judge against, and
the README carried a paragraph saying so, which this deletes.

The rule's own second sentence names why this is not a queue check. The
connector reacts to a rendezvous being satisfied before the driver has stepped
over it, so a pointer compared naively reports every rendezvous-adjacent action
as an early arrival — not an edge case: with the read-through below removed,
five of the twenty-two fixtures fail outright. Three things make the handoff
atomic with respect to the harness mutex. `await` advances the pointer inside
the critical section that observed the satisfying state, which covers a driver
blocked ON the rendezvous. `currentStepLocked` reads through steps the recorded
history has already satisfied but the driver has not consumed, which covers a
driver that has not reached the rendezvous at all — a poll's page is delivered
and then checkpointed by one causal chain in the connector, and the driver need
not have run its `expectDelivered` step by the time the save lands. And a step
the driver PERFORMS rather than waits for hands the pointer on before it acts,
since nothing the connector does can precede the act; that is also what stops
the read-through dead at an action step the driver has not performed, which is
what keeps a subscribe written at dial — before the `serve welcome` that
legalizes it — an early arrival rather than a match on the step after.

The judgement is made before the action is recorded, so the matching step's own
"one of mine is waiting" condition is still false and the read-through cannot
run past it. `expectConnect` now takes the dial and installs the socket under
the mutex, because the lookahead reads `d.peer` and `d.connectsTaken` from the
connector's goroutines.

Fixture 02's reordering probe now fails on arrival rather than on the save's
delivery-count witness, which is the stronger diagnosis of the same defect. The
witness stays anyway: the read-through is deliberately permissive — its job is
only to say where the script has got to, and erring strict would stall the
pointer and fail a compliant connector — so the witness is what still speaks if
an arm is ever loosened.

What the rule cannot reach in this lane is worth writing down, since five more
drivers follow. A save is observed where the connector makes it, but an
outbound frame is observed when the harness reads it off the loopback socket, a
hop behind the write. A connector mutated to write its subscribe at dial is
read only after the driver has served the welcome and stepped on, so no fixture
catches it on arrival — 8 of 8 runs, with and without the action-step handoff.
Fixture 07 catches that mutant structurally instead, as a frame where the
script expects a close, and now names the step it belonged to. The rule is not
the weaker half; the observation is, and pinning the write's own instant would
need a witness at the write rather than a stricter reading of the rule.

The driver's own tests pin each half against the mutation that deletes it, on
the harness record paths where the rule lives, with every legal case paired
against the illegal one it differs from by a single fact — including the
subscribe-at-dial shape, which is deterministic there even though the socket
makes it racy in a fixture.
…ait could not hear

#758. Every socket-open wait in the connector carries the staleness cases;
pollPage did not. A socket that goes silently half-open produces no frame and
no read error, so the firing is the only evidence there will ever be — and it
sat unconsumed while a PollSource that returns only on cancellation held
CatchingUp forever, with nothing able to cancel it, because the cancel would
have come from the teardown the wait was preventing.

Both reviewers prescribed disposing the attempt where the expiry is observed,
and that breaks a pinned ordering: TestWalkFailureBetweenPages/staleness_expiry
advances a full window while the first poll is in flight and requires the walk
to still accept, deliver and SAVE that page. It is transition 21's
finish-the-page rule, and it is load-bearing — the reconnect re-enters at pos-1
rather than replaying. That mutant is in the record: checkpoint ledger = [],
want [pos-1].

So the expiry is treated as what it is, a socket outcome, and takes the same
route as a disconnect frame: deferred into the single slot, dispatched at the
page boundary. The slot gains a third form, carrying the age rather than a
frame, and the protocol-fatal probe excludes it by name rather than trusting
what parsing a zero item happens to return.

That left the bounded wait unbounded again, which is the part worth explaining.
The wait relies on the staleness timer's firings and re-arms as WAKE-UPS. An
authoritative expiry latches, and nothing re-arms over a latch — that is
deliberate, and it is what keeps a frame arriving after the deadline from
erasing the verdict — so consuming the firing here consumed the last wake the
wait had. graceWake re-arms the same `staleness` kind purely as a wake source,
keeping the latch so the pump's per-frame reset and a blocked hand-off's
release cannot touch it. The outstanding set is untouched: {staleness} stays
{staleness}, and §23's six kinds and per-state exact sets are not a thing a Go
fix gets to change.

The bound is now published rather than living in a Go comment. Worst case is
DETECTION WINDOW + GRACE PHASE, both one staleness window, and the grace phase
is a deadline read at the instant of deferral with two immunities that are the
whole point of naming it: a frame arriving inside it must not extend it, and a
blocked hand-off must not suspend it. The second is what the earlier borrowed
bound failed — this wait is the one that deliberately stops draining, so the
full-queue premise the suspension rule rests on is false here by construction.
awaitSupersededPoll's "close to two windows ... cannot exceed two" paragraph is
replaced by that statement rather than deleted; it was describing an accident
of borrowing the timer, where this is a contract for six SDKs.

Six tests, each able to fail alone. Four of them were written first and shown
to hang against the unfixed code, reported as bounded watchdog failures rather
than a wedged binary. The immunity pair needed more care than expected: a frame
inside the grace phase hands the wait a WAKE as well as a re-armed window, and
a wake alone is enough to carry an implementation that also moved its deadline
to a correct-looking teardown — so the test consumes that wake first, through a
new hook, and then requires the phase to finish on a wake of its own. Without
that, the no-latch mutant survived.

One guard is honestly unkillable and stays anyway. Dropping the probe's
`!d.stale` changes no behavior today, because protocolFatalFrame rejects a zero
item through parseFrame failing on nil bytes. That is an accident of an
unrelated function's error handling, and a probe resting on it is one refactor
away from reading a staleness expiry as a disconnect frame.
Four threads arrived on the push. One is stale — the connection returned
alongside a dial error is already closed before classification, two commits
back. The post-confirmation welcome is real and filed as #800, with the reason
it is not a one-liner: that dispatch arm has no bounded-write discipline, and
§23 does not say whether a retransmit re-arms the confirmation deadline.

The nil dial result was mine, introduced when I added the (nil, nil) guard two
rounds ago. Every other exit from that select stops the handshake deadline;
this one returned without doing so, carrying an armed handshake-deadline into
Backoff — whose exact outstanding set §23 pins at {backoff} — and a transport
returning (nil, nil) repeatedly accumulates one ghost timer per cycle, each
firing later into a state with no edge for it. The test asserts the exact SET
rather than the absence of a panic, because a defensive edge that leaves a
timer behind has only traded a crash for a leak. Mutation confirms it:
map[backoff:1 handshake-deadline:1].

The terminal race is real and narrower than it reads. Close latches isClosed
and THEN cancels, both inside one critical section, so a reader of runCtx
landing between those two statements sees a live context while Close is already
committed and about to return — and publishes an error element to a consumer
that closed. emitTerminal now decides by claiming under the mutex that owns the
latch, which has no such interval: the claim runs wholly before or wholly after
Close's critical section.

What it does NOT do is stop a terminal from being published after Close
returns, and the shape it replaced did not either — a claim that wins publishes,
and Close was then called against a feed that had already terminated. That is
correct, and worth stating so the next reader does not read more into the
change than it does.

No test discriminates it, and the first one I wrote passed against the code it
was meant to fail — the hook sat where both versions behave identically, which
is the same placement error I criticised in the pump-read hook a round ago. It
is deleted rather than kept as decoration. Reaching the real window needs a hook
between two adjacent statements inside Close, existing for nothing else, and
even then the behaviour it replaces is racy rather than reliably wrong. The
change is kept because deciding against the authoritative latch is the correct
shape, not because a proof forced it.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated no new comments.

Suppressed comments (1)

go/pkg/basecamp/eventfeed/catchup.go:520

  • A non-staleness outcome deferred late in the current staleness window does not receive a fresh grace-phase wake. When the pre-existing timer fires, awaitSupersededPoll treats evaluate(...) == ok as the verdict and returns immediately, even if this fixed deadline is still almost a full window away. That contradicts the documented full grace phase measured from deferral and can abandon a poll prematurely. Arm graceWake for every deferred socket outcome (not only an already-fired staleness outcome), so the wake is fixed at this new deadline and cannot be inherited from the older detection window.
		deadline := l.cfg.clock.Now().Add(l.cfg.staleAfter)
		if stale {
			// The firing this branch consumed was the wait's own wake source,
			// and an authoritative expiry latches — so without this there is
			// nothing left to wake the grace phase at all, and the bounded wait
			// becomes unbounded again. Armed BEFORE the hook, so a test that
			// rendezvouses on the deferral cannot advance past a wake that is
			// not yet armed.
			at.lc.stale.graceWake()
		}

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated no new comments.

Suppressed comments (3)

go/pkg/basecamp/eventfeed/connector.go:495

  • The PR description still says durableGate prevents a checkpoint save from commencing after Close and frames #784 as the remaining post-claim race. This implementation intentionally removes that guarantee: a pre-close decision may start writing after Close, and Wait is now the ordering primitive. Please update the Blocker 6 / #784 narrative so reviewers and release notes describe the lifecycle contract actually shipped here.
// just before Close can still be written just after. That is intended. The

go/pkg/basecamp/eventfeed/scenario_conformance_test.go:502

  • A schema-valid expectPoll may contain both url and query, but this early return silently skips all query/filter assertions whenever url is present. Such a fixture can therefore pass even if the connector supplies the wrong filters while following the correct continuation URL. Validate the URL and then continue into query validation when both pins are supplied.
	if step.URL != "" {
		if call.cursor.PageURL != step.URL {
			return fmt.Errorf("the poll targeted %+v, want the URL followed verbatim: %s", call.cursor, step.URL)
		}
		return nil

go/README.md:688

  • This lifecycle summary omits the new quiescence requirement. Close can return while a detached checkpoint save is still pending, so a consumer that immediately opens a replacement over the same store can race that save. Document that callers must await iteration termination or call Wait before replacement; otherwise “the feed is resumable” suggests Close alone provides that ordering.
`Close` stops the feed without draining, and cancelling the context, calling `Close`, or
breaking out of the loop all end iteration with **no** error element — a clean stop, and
the feed is resumable by design.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 049cfc5ca2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// even then the behaviour it replaces is racy rather than reliably wrong.
// It is kept because deciding against the authoritative latch is simply
// the correct shape, not because a red proof forced it.
if !l.cfg.claimTerminal() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor caller cancellation before publishing a terminal

When the caller cancels ctx after a terminal outcome is selected but before emitTerminal runs—for example, while a subscription-rejection teardown is waiting for the socket close—claimTerminal checks only the connector's isClosed latch, so this still yields the terminal error. Caller cancellation is documented as the same clean, no-error universal edge as Close; fold cancellation into the terminal-publication ordering rather than allowing an already-canceled run to emit an error.

Useful? React with 👍 / 👎.

if err := checkOriginScheme(canonical); err != nil {
return err
}
if err := checkIdentityText("the API base origin", canonical); err != nil {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate origin UTF-8 before canonicalizing it

When an origin contains invalid UTF-8 in its hostname, url.Parse accepts the bytes and CanonicalOrigin passes them through strings.ToLower, which replaces each invalid byte with U+FFFD; validating only the resulting canonical string therefore succeeds. Distinct inputs such as hosts containing \xff and \xfe collapse to the same canonical origin and consequently the same checkpoint identity, allowing separate configured origins to load and overwrite one lineage; validate cfg.origin before this lossy canonicalization.

Useful? React with 👍 / 👎.

// Transition 12: always terminal — cancel the deadline, explicitly
// close the still-open socket (Action Cable leaves a rejected
// socket open), ZERO reconnects.
l.disposeAttempt(at, *deadline)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Check the deadline before accepting a rejection

When a matching reject_subscription and the confirmation deadline are both ready, the frame can win the select and this teardown ignores the deadline timer's failed Stop, producing the permanent subscription_rejected terminal instead of transition 14's teardown and reconnect. Apply the same expired-timer ordering check used by the neighboring confirm_subscription branch so a rejection processed after its confirmation window cannot terminate the feed.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conformance Conformance test suite go

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants