Repository navigation
fix(blob): harden FileBackedBlob.stream() cancel/cleanup paths (#1457) - #1542
Conversation
Pre-existing reliability/correctness hazards in resources/blob.ts FileBackedBlob.stream(), surfaced by the cross-model review of #1454 and unchanged by that PR: 1. fd not nulled after close → a later close()/cancel() (e.g. GC-driven cancel after a successful read) closes a descriptor the OS may have reassigned to an unrelated file/socket. Route every close through a closeFd() helper that guards on `fd != null` and nulls it. 2. cancel() before start()'s open() resolves called close(undefined) and threw ERR_INVALID_ARG_TYPE — same `fd != null` guard fixes it. 3. The start() open-retry 20ms timer was untracked (different scope than pull()'s `timer`); a cancel() during the file-creation wait leaked an fd. Track it as openTimer and clear it in cancel(). 4. A slow async read completing with an error after the watcher-timeout path already called onError() ran onError twice (double close + duplicated #onError callbacks). Add a per-pull `settled` guard. 5. slice(start,end).stream() read and discarded a 256KB buffer per chunk from byte 0 instead of seeking to HEADER_SIZE + start. Seek directly. This also fixes a latent off-by-HEADER_SIZE in the past-first-chunk slice trim (existing tests only sliced within the first chunk, so it was never exercised); the new test covers cross-chunk slices. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The first pass relied on a per-pull `settled` flag plus an fd-null check, which left two cancel-path defects the cross-model review (Codex + Gemini) caught: - fd leak: a cancel() before start()'s open() resolved could not close the descriptor open() later acquired — pull()/cancel() never run again, so it leaked (a realistic abort-during-open path; verified 50/50 fds leaked in the new regression test without the guard). - cancel race: cancel() nulls fd but cannot reach pull()'s per-pull `settled`, so a late read callback could recurse into read(null) and throw ERR_INVALID_ARG_TYPE synchronously, or reject the already-cancelled pull (unhandled rejection). Add a stream-wide `cancelled` flag set in cancel() and gate every open/read/recursion on it: openFile() stops retrying, the open() success path closes a descriptor acquired after cancel, and readMore()/the read callbacks bail cleanly instead of touching a nulled fd. Adds unit tests for both the cancel-race (no throw/rejection) and the fd-leak paths. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request addresses several robustness issues in FileBackedBlob stream handling, specifically preventing file descriptor leaks, double-close errors, and unhandled rejections during stream cancellation. It introduces a stream-wide cancelled flag, an openTimer to track open-retry timeouts, and a settled flag to ensure pull operations are settled exactly once. Additionally, it optimizes the slicing of multi-chunk blobs by seeking directly to the slice start rather than reading and discarding preceding chunks. Comprehensive unit tests have been added to verify these fixes, including tests for multi-chunk slicing, mid-read cancellation, and cancellation before file opening resolves. I have no further feedback as the changes are well-implemented and thoroughly tested.
|
Reviewed; no blockers found. |
| // lookup) is computed lazily on the first miss, so a healthy read never pays for it. | ||
| let deadline = 0; | ||
| const openFile = (resolve: (value: any) => void, reject: (error: Error) => void) => { | ||
| if (cancelled) return resolve(undefined); // consumer cancelled before/while opening; stop retrying |
There was a problem hiding this comment.
Do we need the undefined?
| if (cancelled) return resolve(undefined); // consumer cancelled before/while opening; stop retrying | |
| if (cancelled) return resolve(); // consumer cancelled before/while opening; stop retrying |
Closes #1457.
Fixes pre-existing reliability/correctness hazards in
resources/blob.tsFileBackedBlob.stream(), surfaced by the cross-model review of #1454 / PR #1456 in code those PRs do not change. None are regressions from #1456 — they predate it.What changed
The five hazards listed in #1457:
fdnot nulled after close → double-close on a reassigned descriptor. All closes now route through acloseFd()helper that guards onfd != nulland nulls it. A later GC-drivencancel()after a successful read can no longer close a descriptor the OS may have reassigned to an unrelated file/socket.cancel()beforeopen()resolves →close(undefined)throw. Samefd != nullguard.start()open-retry 20ms timer was untracked (different scope thanpull()'stimer), leaking an fd if cancelled during the file-creation wait. Tracked asopenTimerand cleared incancel().onError— a slow async read completing with an error after the watcher-timeout path already calledonError()ran it twice (double close + duplicated#onErrorcallbacks). Added a per-pullsettledguard.slice(start,end).stream()read and discarded a 256KB buffer per chunk from offset 0 instead of seeking toHEADER_SIZE + start. Now seeks directly.Latent correctness bug fixed alongside #5
The slice trim used the file
positionwhere it needed the content offset (position - HEADER_SIZE). Any slice whose bound landed past the first0x40000chunk returned bytes shifted byHEADER_SIZE. Existing tests only sliced within the first chunk, so it was never exercised. The seek rewrite corrects the accounting; the new cross-chunk slice test fails on the old code.Cross-model review caught two blockers (also fixed here)
A first pass relied on a per-pull
settledflag, whichcancel()(a different scope) can't set. Codex and Gemini both flagged the gap:cancel()beforestart()'sopen()resolved couldn't close the descriptoropen()later acquired (pull()/cancel()never run again). A realistic abort-during-open path.read(null,…)→ synchronousERR_INVALID_ARG_TYPEcrash, or reject the already-cancelled pull (unhandled rejection).Both are fixed by a stream-wide
cancelledflag set incancel(), gating the open-retry loop, theopen()success path (closes an fd acquired post-cancel),readMore()entry, and both read callbacks.Tests
Added to
unitTests/resources/blob.test.js:stream()(boundary-straddling, both-bounds-in-later-chunk seek path, third-chunk-to-EOF, within-first-chunk regression) — verified to fail on the old code./proc/self/fd; verified 50/50 fds leaked without the guard, flat with it.npm run buildclean; full blob unit file green (58 passing).🤖 Generated with Claude Code