Skip to content

feat: add experimental_streamMode 'single-consumer' to streamText#17501

Open
Aayush-engineer wants to merge 1 commit into
vercel:mainfrom
Aayush-engineer:feat/single-consumer-stream-mode
Open

feat: add experimental_streamMode 'single-consumer' to streamText#17501
Aayush-engineer wants to merge 1 commit into
vercel:mainfrom
Aayush-engineer:feat/single-consumer-stream-mode

Conversation

@Aayush-engineer

Copy link
Copy Markdown
Contributor

Background

streamText().textStream (and every other stream/result accessor) routes through teeStream(), which calls ReadableStream.tee() to allow multiple independent readers. The source comment on teeStream() already notes this buffers stream content on the server and assumes LLM output is small enough not to matter.

That assumption breaks down for long-running/large-output streams. #16753 reports high memory p99 and memory-limit breaches in a 128 MB Cloudflare Worker when consuming textStream for long generations, caused by this buffering. The reporter had to bypass streamText internals entirely and read doStream() directly to avoid it.

Summary

Adds an opt-in experimental_streamMode: 'single-consumer' option to streamText(). When set, teeStream() skips ReadableStream.tee() entirely and hands the internal baseStream directly to the first caller. Any second attempt to read the stream — via textStream, stream/fullStream, consumeStream(), partialOutputStream, elementStream, or a final-result getter (usage, steps, finishReason, etc.) — throws an AISDKError, since there's no buffered copy left for a second reader.

Reading a final-result getter after another consumer has already fully drained the stream still works, since the result promises are already resolved by that point — the restriction only applies to reading the underlying stream a second time.

Contributor Credit

Thanks to @matthew-gizmo for the original report and reproduction in #16753.

End-to-End Verification

  • Ran the full stream-text.test.ts suite (408 existing tests) — all pass unmodified.
  • Added 4 new tests covering: textStream consumption in single-consumer mode, reading a single final-result getter, throwing on a second consumer access before the first drains, and throwing synchronously on a direct double-access of stream.
  • tsc --build and ultracite check (repo lint/format) both pass clean.
  • Have not independently measured the actual memory/p99 improvement under load — this PR only verifies functional correctness of the new code path, not the memory characteristics that motivated streamText textStream tee path retains large streaming outputs in memory #16753. Would appreciate maintainer input on whether that's expected before merge, or acceptable to validate separately.

Future Work

  • partialOutputStream and elementStream go through the same teeStream() path and should behave the same way in single-consumer mode, but aren't covered by dedicated tests yet.
  • Possible follow-up: measure actual memory/p99 impact in a Worker-like environment to confirm this resolves the original report's numbers, not just the API shape.

Related Issues

Fixes #16753

Adds an opt-in experimental_streamMode: 'single-consumer' option to
streamText(). When set, the internal stream is not split via
ReadableStream.tee() in teeStream(), avoiding the server-side buffering
tee() introduces for large/long-running streams (see vercel#16753).

Only one of textStream, stream/fullStream, consumeStream(),
partialOutputStream, elementStream, or a final-result getter (usage,
steps, finishReason, etc.) may be accessed in this mode; accessing a
second one throws, since there is no buffered copy left to read.

Fixes vercel#16753
@Aayush-engineer
Aayush-engineer force-pushed the feat/single-consumer-stream-mode branch from f9478cd to 0793f55 Compare July 18, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

streamText textStream tee path retains large streaming outputs in memory

1 participant