Skip to content

fix(blob): harden FileBackedBlob.stream() cancel/cleanup paths (#1457) - #1542

Merged
kriszyp merged 2 commits into
mainfrom
kris/blob-stream-1457
Jul 2, 2026
Merged

kriszyp merged 2 commits into
mainfrom
kris/blob-stream-1457

Conversation

@kriszyp

@kriszyp kriszyp commented Jun 30, 2026

Copy link
Copy Markdown
Member

Closes #1457.

Fixes pre-existing reliability/correctness hazards in resources/blob.ts FileBackedBlob.stream(), surfaced by the cross-model review of #1454 / PR #1456 in code those PRs do not change. None are regressions from #1456 — they predate it.

What changed

The five hazards listed in #1457:

  1. fd not nulled after close → double-close on a reassigned descriptor. All closes now route through a closeFd() helper that guards on fd != null and nulls it. A later GC-driven cancel() after a successful read can no longer close a descriptor the OS may have reassigned to an unrelated file/socket.
  2. cancel() before open() resolves → close(undefined) throw. Same fd != null guard.
  3. start() open-retry 20ms timer was untracked (different scope than pull()'s timer), leaking an fd if cancelled during the file-creation wait. Tracked as openTimer and cleared in cancel().
  4. Double onError — a slow async read completing with an error after the watcher-timeout path already called onError() ran it twice (double close + duplicated #onError callbacks). Added a per-pull settled guard.
  5. Slice read sequentially from byte 0 — slice(start,end).stream() read and discarded a 256KB buffer per chunk from offset 0 instead of seeking to HEADER_SIZE + start. Now seeks directly.

Latent correctness bug fixed alongside #5

The slice trim used the file position where it needed the content offset (position - HEADER_SIZE). Any slice whose bound landed past the first 0x40000 chunk returned bytes shifted by HEADER_SIZE. Existing tests only sliced within the first chunk, so it was never exercised. The seek rewrite corrects the accounting; the new cross-chunk slice test fails on the old code.

Cross-model review caught two blockers (also fixed here)

A first pass relied on a per-pull settled flag, which cancel() (a different scope) can't set. Codex and Gemini both flagged the gap:

  • fd leak — cancel() before start()'s open() resolved couldn't close the descriptor open() later acquired (pull()/cancel() never run again). A realistic abort-during-open path.
  • cancel race — a late read callback after cancel could recurse into read(null,…) → synchronous ERR_INVALID_ARG_TYPE crash, or reject the already-cancelled pull (unhandled rejection).

Both are fixed by a stream-wide cancelled flag set in cancel(), gating the open-retry loop, the open() success path (closes an fd acquired post-cancel), readMore() entry, and both read callbacks.

Tests

Added to unitTests/resources/blob.test.js:

  • cross-chunk slice via stream() (boundary-straddling, both-bounds-in-later-chunk seek path, third-chunk-to-EOF, within-first-chunk regression) — verified to fail on the old code.
  • cancel mid-read — no throw / no unhandled rejection.
  • cancel before open() resolves — fd accounting via /proc/self/fd; verified 50/50 fds leaked without the guard, flat with it.

npm run build clean; full blob unit file green (58 passing).

🤖 Generated with Claude Code

Kris Zyp and others added 2 commits June 30, 2026 10:10
Pre-existing reliability/correctness hazards in resources/blob.ts
FileBackedBlob.stream(), surfaced by the cross-model review of #1454 and
unchanged by that PR:

1. fd not nulled after close → a later close()/cancel() (e.g. GC-driven
   cancel after a successful read) closes a descriptor the OS may have
   reassigned to an unrelated file/socket. Route every close through a
   closeFd() helper that guards on `fd != null` and nulls it.
2. cancel() before start()'s open() resolves called close(undefined) and
   threw ERR_INVALID_ARG_TYPE — same `fd != null` guard fixes it.
3. The start() open-retry 20ms timer was untracked (different scope than
   pull()'s `timer`); a cancel() during the file-creation wait leaked an
   fd. Track it as openTimer and clear it in cancel().
4. A slow async read completing with an error after the watcher-timeout
   path already called onError() ran onError twice (double close +
   duplicated #onError callbacks). Add a per-pull `settled` guard.
5. slice(start,end).stream() read and discarded a 256KB buffer per chunk
   from byte 0 instead of seeking to HEADER_SIZE + start. Seek directly.
   This also fixes a latent off-by-HEADER_SIZE in the past-first-chunk
   slice trim (existing tests only sliced within the first chunk, so it
   was never exercised); the new test covers cross-chunk slices.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The first pass relied on a per-pull `settled` flag plus an fd-null check,
which left two cancel-path defects the cross-model review (Codex + Gemini)
caught:

- fd leak: a cancel() before start()'s open() resolved could not close the
  descriptor open() later acquired — pull()/cancel() never run again, so it
  leaked (a realistic abort-during-open path; verified 50/50 fds leaked in
  the new regression test without the guard).
- cancel race: cancel() nulls fd but cannot reach pull()'s per-pull
  `settled`, so a late read callback could recurse into read(null) and throw
  ERR_INVALID_ARG_TYPE synchronously, or reject the already-cancelled pull
  (unhandled rejection).

Add a stream-wide `cancelled` flag set in cancel() and gate every
open/read/recursion on it: openFile() stops retrying, the open() success
path closes a descriptor acquired after cancel, and readMore()/the read
callbacks bail cleanly instead of touching a nulled fd. Adds unit tests for
both the cancel-race (no throw/rejection) and the fd-leak paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses several robustness issues in FileBackedBlob stream handling, specifically preventing file descriptor leaks, double-close errors, and unhandled rejections during stream cancellation. It introduces a stream-wide cancelled flag, an openTimer to track open-retry timeouts, and a settled flag to ensure pull operations are settled exactly once. Additionally, it optimizes the slicing of multi-chunk blobs by seeking directly to the slice start rather than reading and discarding preceding chunks. Comprehensive unit tests have been added to verify these fixes, including tests for multi-chunk slicing, mid-read cancellation, and cancellation before file opening resolves. I have no further feedback as the changes are well-implemented and thoroughly tested.

@claude

claude Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

Comment thread resources/blob.ts
// lookup) is computed lazily on the first miss, so a healthy read never pays for it.
let deadline = 0;
const openFile = (resolve: (value: any) => void, reject: (error: Error) => void) => {
if (cancelled) return resolve(undefined); // consumer cancelled before/while opening; stop retrying

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the undefined?

Suggested change
if (cancelled) return resolve(undefined); // consumer cancelled before/while opening; stop retrying
if (cancelled) return resolve(); // consumer cancelled before/while opening; stop retrying

@kriszyp
kriszyp merged commit 5691667 into main Jul 2, 2026
51 of 52 checks passed
@kriszyp
kriszyp deleted the kris/blob-stream-1457 branch July 2, 2026 15:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants