Repository navigation
bug(sdk): StreamController buffers every event forever for callback consumers — unbounded memory growth #389
Description
Activity
Consolidating #477 into this issue — same bug, same file, same mechanism:
onEventdelivers to callback subscribers, then_buffer.push(event)in theelsebranch, and the only drain is the async iterator'snext(). #477 was filed independently while reviewing #470; this issue was filed first (repo-wide audit) and already carries the two things #477 lacks —close()not clearing_buffer, andFilteredStreamControllercompounding.Porting what #477 had that this issue does not:
1. The retention is quantified, and the events are pinned.
A modest 100 events/sec stream at ~200 bytes/event retains ~70 MB/hour, unbounded. It is the
_bufferreference itself that pins the events, so nothing is collectable while the controller lives — and for aliveQuery()or.where()-filtered stream that doubles, since the inner controller buffers every event off the wire while the outer buffers every event matching the filter._bufferhas exactly four references incontroller.ts—:20(declaration),:36(push),:170-171(read/shift). That is the whole surface, which is what makes the "no drain for callback consumers" claim checkable rather than asserted.2. Three fix options, ranked — only one is purely a fix.
- Only buffer once the iterator has been requested. Set a flag in
[Symbol.asyncIterator]()and make:36conditional on it. Preserves the current guarantee forfor awaitconsumers (including one attaching late but before events flow), and reduces a subscribe-only stream to zero retention. - Cap it with a documented bound and drop-oldest — turns an unbounded leak into a bounded one, but silently loses events for a slow iterator.
- Drop the buffer, so
next()only ever resolves from a waiter — simplest, but afor awaitconsumer that awaits something between iterations would miss events. Real regression.
(1) is the only option that is purely a fix; (2) and (3) each trade correctness for simplicity.
3. Acceptance criteria.
- A
.subscribe()-only stream retains no events regardless of how many arrive - A
for awaitconsumer still receives events that arrived between iterations - A consumer that attaches an iterator after subscribing does not lose events that arrived in between, or the change in that behaviour is documented
- Pinned by a test asserting the internal buffer stays empty across N events with only a callback subscriber — use a filtered stream, so the assertion covers both the inner and outer controller
-
close()clears_buffer(from this issue's original scope)
4. Cross-links from #477: #473 (subscriber-throw isolation, same loops — the throw-path buffer growth described there is the smaller of the two), #470 (documents the throw-path case; does not change this).
Related: #477 (closed as duplicate of this), #473, #470. #152 remains the unrelated server-side buffer.
Consolidated by the pm-triage routine on 2026-08-25, on Eric's instruction; verified by code-read against
b2eee9d.- Only buffer once the iterator has been requested. Set a flag in
Priority raised P2 → P1, and #477 consolidated in (see above).
Rationale:
.subscribe()is the primary documented streaming API andliveQuery()is built on it, so this is unbounded memory growth on the default path of a shipped public release — roughly 70 MB/hour at 100 events/sec, doubled for a filtered stream, with no cap and no drain.This is under a re-anchored rubric (the old P0/P1 definitions were pinned to the launch, which shipped 08-19); the new anchor and all five changes are recorded in #501. Flagging rather than assuming — say so if you'd rate it differently.
pm-triage routine, 2026-08-25, on Eric's instruction.
Summary
StreamControllerpushes every event onto an internal_bufferwhenever no async-iterator waiter is pending. The primary documented consumption paths (subscribe()/liveQuery()) are callback-based and never drain that buffer, so it grows without bound;close()doesn't clear it either.Detail
clients/ts/src/stream/controller.ts::27-38onEventdelivers to callback subscribers, thenif (waiter) waiter.resolve(...) else this._buffer.push(event).[Symbol.asyncIterator]().next(), so_buffer.shift()is never reached.close()(:150-163) resolves waiters but does not clear_buffer.FilteredStreamControllercompounds this (inner + outer controllers each buffer).Impact
A long-lived dashboard subscription on a busy table accumulates every event in memory until the tab dies.
Fix direction
Only buffer when at least one async-iterator consumer has been used, or cap/clear the buffer for callback-only mode; clear
_bufferonclose().Found in a repo-wide audit; verified by code trace. #152 is the unrelated server-side buffer.