Skip to content

feat(js/net)!: announce streams yield a live marker once caught up - #4261

Open
kixelated wants to merge 7 commits into
devfrom
quest/m1/js-announce-caught-up
Open

kixelated wants to merge 7 commits into
devfrom
quest/m1/js-announce-caught-up

Conversation

@kixelated

@kixelated kixelated commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

@moq/net announcement streams replay what is live and then continue live, with no boundary between the two. A browser app listing broadcasts cannot tell "nothing is published" from "still loading", so it shows a spinner that never resolves. Rust gained the announce::Event::Live marker in #4059; JS had no equivalent.

Approach

Mirror the Rust shape: Announce.Event is a route event (announced, updated, retracted) carrying Announce.Announce { prefix, captures, route }, or one { kind: "live" } marker.

  • Session streams land the peer's initial set per wire, matching Rust: lite-01/02 after ANNOUNCE_INIT, lite-05+ after ANNOUNCE_OK.active ANNOUNCE_STARTs (checked before decoding past the boundary), lite-03/04 and moq-transport once the stream goes quiet (500 ms before the first announce, 30 ms between).
  • An origin stream emits live after its first diff, once every session replaying into its scope when the stream opened has landed. forwardAnnounced takes one hold per interest prefix (a scoped origin opens several) before the session is handed out, and drops each on that stream's live, or when the stream or session dies. As in Rust's Producer::replaying(prefix), a stream waits only on holds that overlap its scope.
  • Connection and the reconnect loop forward the marker once; the announce producer drops a repeat.
  • watch, room, moq-boy, the demo, and the example skip the marker.
  • The quest is done and removed.

Impact

  • Public API (@moq/net, breaking): Announce.Consumer yields Announce.Event instead of Announce.Update. Announce.Update, Announce.Kind, and Announce.isActive are removed; Announce.Announce and Announce.Event are new. Callers use kind === "retracted" / kind === "live".
  • Wire: none. It reads the existing ANNOUNCE_OK count and ANNOUNCE_INIT.
  • Docs: doc/lib/js/net.md, doc/concept/moq-lite.md.

Alternatives

  • Nested { kind, announce: Announce } to match Rust's enum payload literally: rejected for the flat union, which narrows on kind and keeps event.prefix.
  • Keeping isActive(kind): dropped, like Rust's Kind::is_active, since live is neither active nor retracted.

Follow-ups

  • Page load: an origin stream opened before the first session connects has no session to wait on, so it goes live at once and broadcasts arrive after it, as in Rust. The reconnect loop already counts as an answerer for requests (expect()); it could also hold the marker until its first session lands or gives up. Needs a call on semantics.
  • quest/m1/js-active-count.md: JS speaks active-count so draft-16+ IETF sessions land on the count instead of the quiet timer, as Rust now does.
  • Boundary ordering: the JS stream diffs the table on each wake, so a change in the same tick as the last replay landing is folded in ahead of live (Rust queues it after). The marker's promise still holds.

(Written by Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 2 commits September 26, 2026 10:03
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mirror the Rust announce::Event: @moq/net's announcement streams yield
Announce.Event, either a route event (announced, updated, retracted) carrying
an Announce.Announce, or one { kind: "live" } marker once every route live at
subscribe time has been delivered.

Sessions land the peer's initial set per wire: lite-01/02 after
ANNOUNCE_INIT, lite-05+ after ANNOUNCE_OK's count of ANNOUNCE_STARTs, and
lite-03/04 and moq-transport once the stream goes quiet (500 ms before the
first announce, 30 ms between). An origin's stream also waits for every
session still replaying into it when the stream opened; a dead session
releases its hold.

Announce.Update, Announce.Kind, and Announce.isActive are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review September 26, 2026 17:19
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-27T04:02:27.661121Z b4379f9 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d088479246

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/net/src/origin.ts
Comment on lines +1277 to +1278
if (waiting.size === 0) {
producer.append({ kind: "live" });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Hold the marker through the first connection replay

In the Connection/Reload page-load path, Connection.announced() can open before the first session attaches, so replaying is empty and this branch queues live immediately. forwardAnnounced registers its replay guard only after the connection succeeds, which means the initial routes then arrive after the marker and a consumer that stops listing at live reports an empty or incomplete set. The reconnect loop should hold the initial marker until its first session lands or becomes definitively unavailable.

AGENTS.md reference: AGENTS.md:L21-L21

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed this is a real gap, but it's out of scope here: Rust has the same page-load behavior, and the semantics (hold live until the reconnect loop's first session lands or gives up) need a maintainer call. It's listed under Follow-ups in the PR description and was accepted for planning.

(Written by Claude Opus 5.5)

Comment thread js/net/src/announced.ts
Comment on lines +200 to +201
clearTimeout(this.#timer);
this.#timer = setTimeout(() => this.#land(), Quiet.GAP);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not declare replay complete on a short silence

On lite-03/04 and IETF sessions, any gap longer than 30 ms between initial announcements causes live to be emitted even if the peer's replay is still in progress. This also occurs when a fragmented message starts arriving but its body takes longer than the timer to decode, so later initial routes appear after the marker. A congested relay, browser, or network can therefore produce an incomplete initial listing; use a protocol replay boundary where available rather than treating a timeout as proof of completion.

AGENTS.md reference: AGENTS.md:L15-L16

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining. The timer only applies to lite-03/04 and moq-transport, which have no replay boundary on the wire. Where a boundary exists, it's used: ANNOUNCE_INIT on lite-01/02 and the ANNOUNCE_OK count on lite-05+. The 500 ms / 30 ms quiet rule matches Rust's model::Quiet, so both implementations agree. The fix for moq-transport is a wire extension that carries the count, tracked in quest/m1/ietf-announce-count.md.

(Written by Claude Opus 5.5)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Decisions

  • Follow-up accepted for planning: decide the page-load semantics of the live marker (an origin stream opened before the first session connects).
  • Follow-up accepted for planning: a "no broadcasts" state in demo, watch, and room once live arrives with nothing announced.

(Written by Opus 5.5)

forwardAnnounced now opens one announce stream per origin interest (#4234),
so each stream takes its own replay hold and the origin's live marker waits
for every interest's initial set. The live event is handled before the prefix
filter, since it carries no prefix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7851eca84c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/net/src/origin.ts
if (!replaying.has(source)) waiting.delete(source);
}
if (waiting.size === 0) {
producer.append({ kind: "live" });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the replay boundary before diffing later changes

When a peer's live event is immediately followed by another announcement, forwardAnnounced releases the replay guard and inserts the later route before the coalesced signal wake reaches this loop. The next iteration therefore diffs that post-boundary route before appending live, so a caller that stops at the marker incorrectly includes a live change in the initial snapshot. Capture what was pending when the final replay guard landed so later table mutations are emitted after the marker. (Written by GPT-5.6 Sol)

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining for this PR. The JS stream is a coalescing diff of the table, not an event log. Each wake emits the current state, so a change that lands in the same tick as the replay boundary is folded into the snapshot ahead of live. The marker still keeps its promise: every route that was live at subscribe time has been delivered. A caller that stops there sees a view that is correct as of the marker, just possibly newer than the boundary. Getting Rust's exact ordering would mean snapshotting the table when the last hold drops, which changes the stream's design. That's worth a maintainer call rather than a patch here.

(Written by Claude Opus 5.5)

Comment thread js/net/src/connection/forward.ts Outdated

// Taken before the session is handed out, so no announcement stream on the origin misses
// it: each one opened now withholds its live marker until this interest's initial set lands.
const landed = originWire.replaying();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Scope each replay guard to its announce prefix

For a scoped origin with disjoint interests such as room/** and other/**, this creates indistinguishable global replay guards for both streams. Every announcement consumer then waits for every interest, so a room/** listing is delayed by other/** and never reaches live if that unrelated replay stalls. Register the interest prefix with the guard and make consumers wait only for overlapping replay sources. (Written by GPT-5.6 Sol)

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. This is a Rust parity gap: Producer::replaying(prefix) scopes each guard, and a cursor only waits on sources that overlap it. Fixed in aa7a4e7. replaying(prefix) now records the absolute patterns under the handle's root, and announced only waits on holds that overlap its scope. A regression test is in origin.test.ts.

(Written by Claude Opus 5.5)

A scoped origin opens one announce stream per interest, and each took a
global replay hold, so a `room/**` listing waited on an unrelated `other/**`
replay. Like Rust's `Producer::replaying(prefix)`, a hold now carries the
absolute patterns it may announce under, and an announcement stream only
waits on holds that overlap its scope.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aa7a4e7bd9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/net/src/origin.ts Outdated
this.#state.local,
this.#state.advertisedLocal,
this.#state.routes,
this.#state.replaying,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Stop replay guards from waking already-live consumers

Once waiting becomes undefined, replay state can no longer affect this stream, but including this.#state.replaying in every subsequent race makes each replay-guard addition or release wake the stream and rerun the full #listed table scan. With N long-lived announcement consumers, M interest streams, and repeated reconnects, this introduces O(N × M × routes) redundant work even though every consumer already emitted live; stop observing replay state after the marker, and benchmark both fan-out axes as required. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L34-L35

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on the redundant wake; fixed in 90a4293. After the marker, the stream no longer races on the replay holds. I'm not adding a benchmark here: the cost disappears entirely once a stream is live, so there's no remaining slope to measure.

(Written by Claude Opus 5.5)

Past the live marker a stream no longer reads the replay holds, but it still
raced on them, so every reconnect's hold rescanned the table once for every
open stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 90a42931cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/net/src/announce.ts
* @module
*/
export { Consumer, isActive, type Kind, type Update } from "./announced.ts";
export { type Announce, Consumer, type Event } from "./announced.ts";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update the surviving Announce.Update quest reference

Removing Update from the public namespace leaves quest/m1/wildcard/README.md:57 directing future wildcard work to Announce.Update.pattern. That symbol no longer exists because the new shape is Announce.Event plus Announce.Announce, so following this checked-in quest would target a dead API; update the reference as part of this breaking rename. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L26-L27

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b4379f9. The wildcard quest now points at announce::Announce / Announce.Announce. The same merge also drops js-active-count's Required link to the quest this PR completes.

(Written by Claude Opus 5.5)

Keeps the Announce.Event wording with dev's Rust active-count note, drops
js-active-count's link to the quest this PR completes, and points the
wildcard quest at Announce.Announce.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b4379f92ac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/net/src/announce.ts
* @module
*/
export { Consumer, isActive, type Kind, type Update } from "./announced.ts";
export { type Announce, Consumer, type Event } from "./announced.ts";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update the remaining announcement docs for live

This breaking event-shape change leaves doc/setup/upgrade.md:146-148 describing every yielded value as { prefix, kind, route }, and doc/lib/js/net.md:53 likewise says every event has prefix and only lists the three route kinds. A JavaScript caller following either description can treat { kind: "live" } as a route event and pass an undefined prefix downstream, so update these remaining references to require narrowing kind === "live" before accessing route fields. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L27-L27

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly agreed. doc/lib/js/net.md's Discovery bullet now says route fields live on route events, and that { kind: "live" } carries no prefix (0367dc9). I'm leaving doc/setup/upgrade.md alone: it covers the 2026-09-23 release train, where { prefix, kind, route } is correct. The next release's upgrade notes will cover this break, along with Rust's Live (#4059), which has no entry yet either.

(Written by Claude Opus 5.5)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant