Skip to content

feat(node:zlib): Transform-stream objects + Brotli fns (#1843) - #1866

Merged
proggeramlug merged 1 commit into
mainfrom
worktree-fix-1843-zlib-streams
May 26, 2026
Merged

feat(node:zlib): Transform-stream objects + Brotli fns (#1843)#1866
proggeramlug merged 1 commit into
mainfrom
worktree-fix-1843-zlib-streams

Conversation

@proggeramlug

Copy link
Copy Markdown
Contributor

Summary

Implements the node:zlib Transform-stream interface (cluster 1 of #1843, the highest-leverage gap) plus Brotli functions (cluster 2) and fixes two latent handle-dereference segfaults surfaced along the way.

node:zlib is routed to the perry-ext-zlib crate via well_known_bindings.toml (#466), so the work lands there, wired into perry-stdlib's event loop through a new external-zlib-pump feature (mirroring external-net-pump). A bundled equivalent lives in perry-stdlib's compression feature for the no-flip path.

What now works

zlib.createGzip() / createGunzip() / createDeflate() / createInflate() / createDeflateRaw() / createInflateRaw() / createUnzip() / createBrotliCompress() / createBrotliDecompress() return real stream objects supporting:

  • .write(chunk) / .end([chunk]) (Buffer or string chunks)
  • .on('data' | 'end' | 'error' | 'finish' | 'close', cb) / .once / .addListener
  • .pipe(dest) (forwards datadest.write, enddest.end, returns dest for chaining)
  • .flush() / .close() / .destroy()

Plus one-shot zlib.brotliCompressSync / brotliDecompressSync / brotliCompress / brotliDecompress (the brotli crate was already a compression dep).

Compression is synchronous, so input is buffered across .write() and the codec runs once on .end(); the resulting data/end events are deferred onto a queue drained by the main-thread pump, so listeners registered after .write() still fire and .pipe() can forward chunks.

End-to-end output is byte-for-byte identical to node --experimental-strip-types for gzip/deflate/brotli stream round-trips, .pipe() chains, and zlib.constants.Z_*.

Runtime segfault fixes (general)

Two is_class_object_ptr / js_value_typeof low-address guards only rejected the tiny net/fastify id space (< 0x1008 / > 0x10000). A native-module registry handle in the mid range (zlib uses a 0x60000 stream base to stay clear of those id spaces) sailed past them and segfaulted dereferencing [handle - 8] / [handle + 12]. Both now reject anything < 0x100000 — the same handle/real-pointer boundary js_native_call_method already uses (real heap objects always live well above it). This also raises every handle subsystem's latent id ceiling from 0x1008 to 0x100000, and makes typeof aStreamHandle === "object".

Buffer-aware one-shot input

gzipSync/gunzipSync/deflateSync/inflateSync/brotli*Sync read their argument through a buffer-aware reader, so a real Buffer/Uint8Array (e.g. gunzipSync(Buffer.concat(chunks)), gunzipSync(fs.readFileSync(...))) is read via its BufferHeader rather than misread as a StringHeader.

Out of scope / follow-ups

  • .flush([cb]) is a no-op (the buffer-until-end model has nothing to flush mid-stream); incremental-flush tests like test-zlib-flush-drain-longblock.js are not addressed here.
  • Per-write incremental data emission (output currently emits on .end()).

Closes #1843 (cluster 1 + cluster 2 + cluster 3 Z_* constants).

Implement the zlib stream interface in perry-ext-zlib (the #466
well-known-bound crate for node:zlib): createGzip/createGunzip/
createDeflate/createInflate/createDeflateRaw/createInflateRaw/
createUnzip/createBrotliCompress/createBrotliDecompress return real
stream objects with .write/.end/.on('data'|'end'|'error'|'finish'|
'close')/.once/.pipe/.flush/.close. Compression is synchronous: input
is buffered across .write() and the codec runs on .end(), with
'data'/'end' deferred onto a queue drained by the main-thread pump so
listeners registered after .write() still fire and .pipe() forwards
chunks (returns dest for chaining).

Wired into perry-stdlib's event loop via a new external-zlib-pump
feature (mirrors external-net-pump): dispatch arm + pump + has_active
externs, activated by optimized_libs when the flip strips compression.
A bundled equivalent lives in perry-stdlib's compression feature for
the no-flip path. Also adds Brotli one-shots (brotliCompressSync/
brotliDecompressSync + async) — the brotli crate was already a dep.

Fix two latent handle-deref segfaults: is_class_object_ptr and
js_value_typeof low-address guards (< 0x1008 / > 0x10000) only covered
the tiny net/fastify id space; a mid-range POINTER_TAG registry handle
(zlib uses a 0x60000 stream base) sailed past them and segfaulted
reading [handle-8]/[handle+12]. Both now reject < 0x100000, the same
handle/real-pointer boundary js_native_call_method uses, so
typeof aStreamHandle === "object" and every handle subsystem's latent
id ceiling rises from 0x1008 to 0x100000.

Make the sync one-shots buffer-aware so gunzipSync(Buffer.concat(...))
/ gunzipSync(fs.readFileSync(...)) read via BufferHeader instead of
misreading a real Buffer as a StringHeader.

Stream/brotli round-trips, pipe chains, and Z_* constants are
byte-for-byte identical to node --experimental-strip-types.
@proggeramlug
proggeramlug merged commit 0f7434d into main May 26, 2026
18 of 19 checks passed
@proggeramlug
proggeramlug deleted the worktree-fix-1843-zlib-streams branch May 26, 2026 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

node:zlib: stream objects missing .on/.pipe/.write (Transform interface) + brotli fns + Z_* constants (43 radar runtime-fails)

1 participant