Repository navigation
fix(dx): add ./streams subpath export to package.json - #59
Merged
Merged
Conversation
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
derodero24
added a commit
that referenced
this pull request
Mar 24, 2026
* Set up napi-rs build pipeline (#9) * chore(core): add Rust crate skeleton with napi-rs bindings Minimal `zflate` crate with a single `version()` export to verify the napi-rs build pipeline works end-to-end. - crates/core/Cargo.toml: cdylib crate with napi-rs v3 - crates/core/build.rs: napi_build setup - crates/core/src/lib.rs: version() function * chore: add package.json with napi-rs configuration Configure the npm package with 9 build targets (macOS, Linux, Windows, WASM), dev dependencies for tooling (Biome, Vitest, TypeScript, Changesets, CodSpeed), and build/test/lint scripts. * chore: add build scripts and module entry points - scripts/optimize-wasm.js: post-process WASM with wasm-opt - index.mjs / index.d.mts: ESM entry points - index.js / index.d.ts / browser.js: napi-rs generated files (force-tracked) * Add CI workflow (#10) * test: add initial test suite and ESM smoke test - __test__/index.spec.ts: Vitest spec verifying version() export - __test__/esm-import.mjs: ESM import smoke test for CI * chore: update biome and tsconfig for test files - biome.json: allow console in ESM smoke test - tsconfig.json: format with Biome * ci: add CI workflow Full CI pipeline adapted from rapid-fuzzy: - Lint (Biome, TypeScript, publint, generated file verification) - Dependency audit (npm + cargo-deny) - Rust lint (fmt + clippy) and test - Cross-platform build (9 targets: macOS, Linux, Windows, WASM) - Node.js test matrix (3 versions × 3 OSes) - Coverage (Rust via cargo-llvm-cov + JS via Vitest, uploaded to Codecov) * Add release and automation workflows (#11) * ci: add release workflow with Cargo.toml version sync Changesets-driven release flow: - On develop push: create version PR, then release PR (develop → main) - On main push: build all 9 targets and publish to npm with provenance - scripts/sync-cargo-version.js: keep Cargo.toml version in sync with package.json after changeset version bumps * ci: add CodeQL security analysis Analyze JavaScript/TypeScript and Rust code for security vulnerabilities. Runs on PRs, develop pushes, and weekly schedule (Monday 6 AM UTC). * ci: add CodSpeed benchmark workflow Continuous performance tracking via CodSpeed. Runs JS benchmarks on develop pushes and PRs when crate or bench files change. Rust benchmarks will be added when the bench crate is introduced. * ci: add self-hosted Renovate workflow Automated dependency updates via self-hosted Renovate bot. Runs weekly on Sunday 19:00 UTC (Monday 4:00 JST). * test(bench): add initial benchmark file Minimal benchmark to satisfy CodSpeed workflow. Real compression benchmarks will be added alongside algorithm implementations. * Implement zstd compression and decompression (#12) * feat(zstd): implement zstd compression and decompression Add zstdCompress() and zstdDecompress() functions powered by the zstd C library via the zstd Rust crate. - Compression levels 1-22 (default: 3) - Auto-detection of frame content size for efficient decompression - zstdDecompressWithCapacity() for data exceeding 256 MB default limit - Rust unit tests for round-trip, empty data, large data, and levels * test(zstd): add JS tests and benchmarks for zstd - __test__/zstd.spec.ts: 11 tests covering round-trip, levels, errors - __test__/esm-import.mjs: updated with zstd round-trip smoke test - __test__/index.bench.ts: compress/decompress benchmarks at 150B/10KB/1MB - index.mjs: export zstd functions * chore: update napi-rs generated files and add changeset - Regenerate index.js, index.d.ts, browser.js with zstd exports - Add changeset for minor version bump * ci: add WASI SDK setup for zstd WASM builds The zstd crate compiles C code via zstd-sys, which requires a proper WASI sysroot when targeting wasm32-wasip1-threads. Without it, clang picks up system glibc headers and fails with missing 'bits/libc-header-start.h'. Use bytecodealliance/actions/wasi-sdk/setup to install WASI SDK and set CC, AR, CFLAGS, and WASI_SYSROOT environment variables for the WASM build step in both CI and release workflows. * Fix CodeRabbit review findings from #12 (#22) * fix(zstd): validate capacity parameter in zstdDecompressWithCapacity Reject negative, NaN, and Infinity values before casting f64 to usize. Without validation, a negative f64 wraps to a very large usize, which could cause memory exhaustion. Closes #17 * fix(ci): use RUNNER_TEMP for WASI SDK installation Replace /opt with $RUNNER_TEMP for WASI SDK extraction. /opt may not be writable on all GitHub Actions runner configurations. Also switch curl from -sL to -fsSL to fail on HTTP errors. Closes #18 * docs: add SECURITY.md (#23) * docs: add SECURITY.md * docs: address review feedback on SECURITY.md - Replace vague "extremely large size" with "massive size" for clarity - Unify SLA day units by removing "business" qualifier from initial assessment timeline for consistency * docs: add CONTRIBUTING.md (#24) * test(zstd): expand test coverage and benchmarks (#25) * test(zstd): expand test coverage and benchmarks Add comprehensive JS tests covering compression level boundaries (0, negative, 22), decompressWithCapacity edge cases (insufficient, oversized, invalid inputs), and round-trip verification with random data. Expand benchmarks with random (incompressible) and realistic (JSON, text) data variants alongside existing patterned data. Update Rust doc comments to document negative compression levels and level 0 behavior. Add Rust tests for boundary levels. Update changeset to include all public API functions. Closes #19 * chore: update napi-rs generated type definitions Regenerate index.d.ts to reflect updated doc comments for zstd_compress documenting negative levels and level 0 behavior. * feat(zstd): add streaming compression/decompression API (#26) Implement streaming zstd compression and decompression using the Web Streams API (TransformStream). The Rust layer exposes ZstdCompressContext and ZstdDecompressContext classes with transform/flush/finish methods using zstd::stream::raw for stateful chunked processing. The JS layer wraps these in TransformStream via createZstdCompressStream() and createZstdDecompressStream(), available from both ESM and CJS entry points. Streaming output is fully interoperable with one-shot APIs. Add comprehensive tests covering chunked round-trips, various chunk sizes, large data (1MB), random data, and one-shot/streaming interop. Closes #5 * docs: add README and verify npm package readiness (#28) * docs: add README and verify npm package readiness Add comprehensive README with project description, installation guide, quick-start examples (one-shot and streaming), full API reference, supported algorithms table, platform compatibility matrix, and benchmark results. Verify npm package with publint (no errors). Package includes all required files: index.js, index.mjs, index.d.ts, index.d.mts, streams.js, streams.d.ts, browser.js, LICENSE, and README.md. Closes #8 * docs(readme): address review feedback - Add bun installation instructions - Replace streaming example with self-contained runnable snippet * fix(ci): resolve release workflow failures (#29) * fix(ci): resolve release workflow failures - Generate npm platform package directories for all 9 targets - Add optionalDependencies to root package.json for platform binaries - Fix changeset version command argument parsing in release workflow by using a dedicated npm script instead of inline shell command Closes #27 * fix(ci): remove optionalDependencies from committed package.json The optionalDependencies for platform-specific packages are added at publish time by the prepublishOnly hook (napi prepublish -t npm), not in the committed source. Having them in package.json breaks pnpm install --frozen-lockfile since the packages are not yet on npm. * chore(release): version packages (#30) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(ci): regenerate napi bindings for v0.2.0 and fix version workflow (#34) - Regenerate index.js with v0.2.0 version strings - Update npm platform package versions to 0.2.0 - Add Rust toolchain to version job for napi build during version bump - Add napi version, build, and format to changeset:version script - Fix version test to read from package.json dynamically * feat(gzip): add gzip and deflate compression support (#33) Add gzip and raw deflate compression/decompression with one-shot and streaming APIs. Includes interoperability tests with Node.js built-in zlib module. Closes #6 * feat(brotli): add brotli compression support (#32) * feat(brotli): add brotli compression support Add brotli compression/decompression with one-shot and streaming APIs. Quality levels 0-11 (default: 6). Includes interoperability tests with Node.js built-in zlib brotli module. Closes #7 * test(brotli): add brotli compression benchmarks Add brotli compress/decompress benchmarks matching the zstd benchmark structure for patterned, random, and realistic data at various sizes. * test(gzip): increase timeout for 1MB round-trip tests Extend timeout from 5s to 30s for gzip and deflate 1MB data tests to prevent CI failures on slower runners. * test: increase timeout for all 1MB tests Set 30s timeout on all 1MB data round-trip and streaming tests to prevent failures on slower CI runners (Windows). * docs(readme): update API reference for gzip and brotli (#36) Add gzip, deflate, and brotli to the API reference tables. Update supported algorithms table to reflect all three algorithms are now available with both one-shot and streaming APIs. * chore(deps): update taiki-e/install-action digest to c12d62a (#20) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(exports): add ./streams subpath export to package.json (#58) Add a `./streams` subpath export so users can import streaming APIs directly via `import { ... } from 'zflate/streams'`. Closes #48 * fix(gzip): add compression level validation for gzip and deflate (#60) Validate that the compression level parameter is within the valid range (0-9) for gzip_compress, deflate_compress, GzipCompressContext, and DeflateCompressContext. Return InvalidArg error if out of range, matching the existing validation pattern in brotli_compress. Closes #38 * fix(gzip): add decompression size limits to gzip and deflate (#64) * fix(gzip): add decompression size limits to gzip and deflate Add MAX_DECOMPRESSED_SIZE (256 MB) to gzip_decompress and deflate_decompress to prevent zip bomb attacks. Both functions now use chunked reading with size checking instead of unbounded read_to_end(). Add gzipDecompressWithCapacity and deflateDecompressWithCapacity for explicit control over the maximum decompressed size, matching the existing zstd and brotli patterns. Closes #37 * chore(napi): regenerate bindings for new WithCapacity functions * feat(api): accept Uint8Array input in all compression functions (#69) Change all input parameters from Buffer to Either<Buffer, Uint8Array> across all one-shot and streaming functions. This generates Buffer | Uint8Array TypeScript types, enabling browser/Deno/Bun users to pass Uint8Array directly without type errors. Return types remain Buffer. No runtime behavior changes for existing Buffer usage. Closes #42 * docs: enhance README with examples, comparison table, and migration guides (#63) * test: add cargo-fuzz targets for decompression fuzzing (#71) * test: add edge case and error handling tests (#65) * test: expand Node.js zlib interop tests to brotli and zstd (#67) * ci: add hand-written JS files to workflow path filters (#56) * ci(codeql): suppress rust/access-invalid-pointer false positives (#57) * fix(dx): add ./streams subpath export to package.json (#59) * ci: add cargo-audit to dependency audit pipeline (#61) * feat(core): add unified auto-detect decompression API (#72) Add decompress() function that auto-detects compression format from magic bytes (zstd: 0xFD2FB528, gzip: 0x1F8B) and falls back to brotli heuristic detection. Add detectFormat() for format identification without decompression. Supported formats: zstd, gzip, brotli. Raw deflate cannot be auto-detected due to lack of magic bytes. Closes #16 * ci(dx): add @arethetypeswrong/cli type validation (#62) * perf(bench): add competitive benchmarks across all algorithms (#70) * perf(wasm): add WASM binary size reporting to CI (#66) * fix(exports): add missing ESM exports for new functions (#74) Add decompress, detectFormat, gzipDecompressWithCapacity, and deflateDecompressWithCapacity to index.mjs ESM export list. These functions were available via CJS but missing from ESM imports. Update ESM smoke test to cover all WithCapacity functions, decompress, and detectFormat. Closes #73 * feat(node): add Node.js stream.Transform compatibility layer (#76) Add `zflate/node` subpath export with stream.Transform factories for all compression algorithms (zstd, gzip, deflate, brotli), enabling use with Node.js stream.pipeline() and pipe-based workflows. Closes #43 * fix(brotli): use consistent error message for capacity exceeded (#78) Change brotli_decompress_with_capacity error message from "Destination buffer is too small" to "brotli decompress exceeded maximum size of {} bytes" to match the pattern used by all other decompress functions. Closes #77 * fix(docs): remove merge conflict marker and stale src/ references (#80) * feat(core): add async compression/decompression API (#82) Add async versions of all one-shot compression/decompression functions using napi-rs AsyncTask with the Task trait. This runs compression on the libuv thread pool without blocking the Node.js event loop. New functions: zstdCompressAsync, zstdDecompressAsync, gzipCompressAsync, gzipDecompressAsync, deflateCompressAsync, deflateDecompressAsync, brotliCompressAsync, brotliDecompressAsync. Closes #40 * feat(zstd): add dictionary compression API (#83) Add zstd dictionary training, compression, and decompression functions: - zstdTrainDictionary: train a dictionary from sample data - zstdCompressWithDict: compress with a pre-trained dictionary - zstdDecompressWithDict: decompress data compressed with a dictionary Enable the zdict_builder feature for the zstd crate to support dictionary training. Include Rust unit tests and Vitest integration tests verifying round-trip correctness and improved compression ratios for small, similar data. Closes #41 * docs: remove Cloudflare Workers from supported platforms (#91) * ci: include Node.js WASM loader files in CI artifact (#90) * test(wasm): add Node.js WASM compatibility test with native parity verification (#92) * refactor(core): introduce thiserror for structured error types (#93) Replace manual `Error::new(Status::..., ...)` calls with structured `ZflateError` enum variants across all compression modules. Error messages remain identical to preserve backward compatibility. Closes #51 * test(deno): add Deno WASM smoke test (#96) * test(bun): add Bun WASM smoke test (#95) * test(browser): add Playwright browser WASM test (#94) * ci(security): tighten cargo-deny bans and sources to deny mode (#108) * ci: pin dtolnay/rust-toolchain to SHA digest (#107) * fix(core): add decompression size limits to streaming contexts (#109) Add cumulative output size tracking to all streaming decompression contexts (ZstdDecompressContext, GzipDecompressContext, DeflateDecompressContext, BrotliDecompressContext). Each context now enforces MAX_DECOMPRESSED_SIZE (256 MB) across transform() calls, preventing decompression bomb attacks that could cause unbounded memory growth. Closes #97 * feat(core): narrow detectFormat return type to string literal union (#111) Use napi-rs string_enum to return CompressionFormat instead of plain string, generating a const enum with 'zstd' | 'gzip' | 'brotli' | 'unknown' values in TypeScript. Closes #103 * perf(core): optimize buffer allocation in compression and decompression (#110) * perf(core): optimize buffer allocation in compression and decompression Increase decompression read buffer from 4KB to 32KB in gzip, brotli, and brotli_stream modules to reduce syscall overhead during chunked reads. Pre-allocate compression output Vecs with input.len() capacity for gzip and deflate encoders (both sync and async) to avoid repeated reallocations during compression. Closes #100 * revert: keep BUFFER_SIZE at 4096 to avoid decompress regression The 32KB buffer caused 21 benchmark regressions in CodSpeed due to increased stack zeroing cost. Revert to 4KB. The Vec::with_capacity optimization for compression output is retained. * perf(core): replace Vec clone and split_off with std::mem::take in streaming contexts (#112) Use std::mem::take() instead of .get_ref().clone() + .get_mut().clear() in brotli streaming contexts, and instead of .split_off(0) in gzip/deflate streaming contexts. This avoids unnecessary memory allocation and copying by moving ownership of the buffer contents in-place. Closes #99 * fix(zstd): add compression level validation (#113) Validate that the compression level is within the valid zstd range (-131072 to 22) before passing it to the zstd library. This provides clear error messages instead of opaque library errors. Validation is added to: - zstd_compress (sync) - zstd_compress_async - ZstdCompressContext::new (streaming) - zstd_compress_with_dict Closes #81 * feat(zstd): add streaming dictionary compression/decompression (#114) Add ZstdCompressDictContext and ZstdDecompressDictContext classes that accept a pre-trained dictionary for streaming compression/decompression. Include Web Streams (createZstdCompressDictStream/createZstdDecompressDictStream) and Node.js Transform (createZstdCompressDictTransform/createZstdDecompressDictTransform) wrappers, type definitions, ESM exports, and streaming dict round-trip tests. Closes #104 * perf(core): switch flate2 backend from miniz_oxide to zlib-rs (#115) Replace the `rust_backend` (miniz_oxide) feature with `zlib-rs` for 2-3x faster gzip/deflate compression. zlib-rs is a pure-Rust implementation that works on all targets including WASM, is ISRG audited, and is on track to become flate2's default backend. Relax compression level size ordering test since zlib-rs may produce slightly different output sizes per level than miniz_oxide — round-trip correctness is what matters. Closes #98 * perf(zstd): enable multi-threaded compression for native targets (#116) Add `zstdmt` feature for non-WASM targets via conditional dependency. This enables zstd's internal thread pool for compression, improving performance by 20-80% for large inputs on multi-core systems. WASM targets are excluded because wasi-threads is experimental and the interaction with napi-rs worker-based threading is untested. Closes #102 * docs: comprehensive README update with missing APIs and SVG benchmark (#118) * fix(docs): correct benchmark environment label to Apple M2 / Node.js v22 (#119) * perf(wasm): add --converge flag to wasm-opt optimization (#122) Re-run optimization passes until no further gains are achieved. Increase timeout to 5 minutes to accommodate additional passes. Closes #121 * perf(core): use 4x input size for decompression output pre-allocation (#123) Compressed data typically decompresses to several times its original size. Pre-allocating with input.len() alone leads to frequent reallocations during decompression. Using input.len() * 4 as the initial capacity provides a better heuristic while the existing .min() caps continue to prevent excessive allocation. Closes #120 * ci: migrate release workflow to npm trusted publishing (OIDC) (#68) * ci: migrate release workflow to npm trusted publishing (OIDC) Remove NPM_TOKEN/NODE_AUTH_TOKEN secret dependency and rely on OIDC token exchange via id-token:write permission (already set). Trusted publishing must be configured on npmjs.com for each package before the next release. * fix(ci): restore pinned SHA for dtolnay/rust-toolchain Keep the pinned commit hash consistent with ci.yml and codspeed.yml for supply chain security. The rust-toolchain change was out of scope for the trusted publishing migration. * fix(gzip): support concatenated gzip streams in decompression (#131) Replace GzDecoder with MultiGzDecoder from flate2 to correctly handle concatenated gzip members in both one-shot and streaming decompression. * fix(streaming): add finish-guard state tracking for brotli and zstd compress contexts (#132) * feat(core): add missing async variants for one-shot APIs (#133) * feat(streaming): add auto-detect decompression streams (#145) * docs: add troubleshooting section to CONTRIBUTING.md (#146) * docs: add CODE_OF_CONDUCT.md (#149) * ci: apply least-privilege permissions to workflows (#147) * chore: improve package.json metadata and exports map (#148) * ci: clean up CI workflow configuration (#150) * feat(core): expose CRC32 utility function (#134) * feat(core): expose CRC32 utility function * test(crc32): use skipIf for conditional Node.js zlib interop test * ci(testing): add coverage thresholds to vitest config (#151) * ci: add aggregate gate job to CI workflow (#152) * feat(streaming): add configurable maxOutputSize to decompression contexts (#153) Add optional maxOutputSize parameter to all streaming decompression context constructors (GzipDecompressContext, DeflateDecompressContext, ZstdDecompressContext, BrotliDecompressContext, ZstdDecompressDictContext). When not specified, the default 256 MB limit is used. The parameter is also passed through all streaming factory functions (createXxxDecompressStream, createXxxDecompressTransform, and createDecompressStream/createDecompressTransform). Closes #129 * ci: add environment protection for npm publish (#154) * ci: add cargo-shear for unused dependency detection (#155) * ci: add WASM binary size tracking on pull requests (#156) * ci: add scripts/ to CI workflow path triggers (#159) * fix(wasm): add JS-side streaming adapters for browser WASM environment (#158) Replace native streaming contexts with JS-side adapters in the browser entry point to work around WebAssembly.Memory growth invalidating ArrayBuffer views. Adapters accumulate chunks and delegate to one-shot APIs, maintaining the same transform()/flush()/finish() API surface. - browser-streaming.js: adapter classes for all 10 streaming contexts - browser-entry.js: re-exports WASM APIs with adapter overrides - package.json: browser condition points to browser-entry.js - e2e: streaming tests enabled for browser WASM * feat(gzip): add header metadata support for compression and reading (#160) Closes #130 * chore(renovate): run daily and remove PR concurrent limit (#164) * chore(renovate): run daily and remove PR concurrent limit Closes #163 * chore(renovate): disable lock file maintenance * chore(deps): update codspeedhq/action digest to 1c8ae48 (#161) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dtolnay/rust-toolchain digest to 631a55b (#162) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update github/codeql-action digest to 3869755 (#165) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update taiki-e/install-action digest to b988c18 (#166) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(bench): exclude third-party benchmarks from CodSpeed tracking (#169) * fix(bench): exclude third-party benchmarks from CodSpeed tracking Split benchmark files into zflate-only (*.bench.ts) and comparison (*.compare.bench.ts) variants. CodSpeed now only tracks zflate's own benchmarks, eliminating false regressions from pako/fflate/node:zlib that are caused by GitHub Actions CPU heterogeneity affecting Callgrind instruction counting. Closes #167 * fix(bench): use deterministic random data in CodSpeed benchmarks Replace non-deterministic randomBytes() with a seeded LCG to ensure reproducible benchmark inputs across runs, reducing CodSpeed instruction count variance. * feat(bench): migrate from CodSpeed to Bencher.dev for regression detection (#170) Replace CodSpeed simulation mode (Callgrind instruction counting) with Bencher.dev (statistical wall-clock analysis). CodSpeed's instruction counting was fundamentally incompatible with this napi-rs project due to GitHub Actions CPU heterogeneity and glibc's CPU-adaptive malloc. Changes: - Add Bencher workflow with t-test threshold for regression detection - Add vitest-to-bmf.js conversion script for vitest bench output - Remove CodSpeed workflow and @codspeed/vitest-plugin dependency - Remove CodSpeed plugin from vitest config Closes #168 * feat(bench): switch to CodSpeed Walltime mode for accurate benchmarks (#172) Replace Bencher.dev with CodSpeed Walltime mode (Macro Runners) which runs benchmarks on dedicated bare-metal hardware, eliminating GitHub Actions CPU variance. CodSpeed also provides native vitest integration via @codspeed/vitest-plugin, removing the need for conversion scripts. Key change: `mode: walltime` instead of the previous `mode: simulation` which used Callgrind instruction counting. Closes #171 * refactor(bench): split benchmark files per algorithm for granular CodSpeed reporting (#174) Split index.bench.ts (mixed zstd+brotli) into separate zstd.bench.ts and brotli.bench.ts so CodSpeed PR reports only show benchmarks relevant to the changed algorithm. Extract shared test fixtures into bench-fixtures.ts and add realistic data benchmarks (JSON, text) to gzip and deflate for consistent coverage across all four algorithms. Closes #173 * perf(core): eliminate double-copy buffer pattern in decompression (#178) * perf(core): eliminate double-copy buffer pattern in decompression Replace the manual 4 KiB read loop (stack buffer → Vec copy) with read_to_end + Take, which writes directly into Vec spare capacity. This eliminates one full copy of every decompressed byte and uses adaptive buffer sizing instead of fixed 4 KiB chunks. Affected: gzip, deflate, brotli (sync + async, default + with-capacity) Closes #175 * fix(core): use saturating_mul for init capacity and update size limit tests Address CodeRabbit review feedback: - Replace len() * 4 with len().saturating_mul(4) across all call sites to prevent overflow on 32-bit targets (wasm32-wasip1-threads) - Replace old chunk-loop size limit tests with Take+read_to_end tests that exercise the actual decompression pattern (avoiding napi::Error symbol resolution issues in standalone test binaries) * chore(bench): add bench:ci script and Rust CodSpeed benchmarks (#179) * chore(bench): add bench:ci script and Rust CodSpeed benchmarks - Add bench:ci script that excludes comparison benchmarks - Simplify CodSpeed workflow to use bench:ci instead of explicit file list - Use glob pattern for bench file path triggers - Add crates/bench workspace member with Criterion benchmarks (gzip, deflate, brotli, zstd) using codspeed-criterion-compat - Add rust-benchmarks job to CodSpeed workflow (simulation mode) Closes #177 * fix(ci): add Cargo.toml, Cargo.lock, package.json to CodSpeed path triggers Ensures CodSpeed benchmarks run when workspace config or dependency lockfiles change, which could affect benchmark behavior. * feat(core): add LZ4 frame compression support (#180) * feat(core): add LZ4 frame compression support Add LZ4 as a fourth compression algorithm using lz4_flex (pure Rust): - Bulk API: lz4Compress, lz4Decompress, lz4DecompressWithCapacity - Async API: lz4CompressAsync, lz4DecompressAsync, lz4DecompressWithCapacityAsync - Streaming: Lz4CompressContext, Lz4DecompressContext - Web Streams: createLz4CompressStream, createLz4DecompressStream - Format detection: detectFormat returns "lz4" for LZ4 frame magic bytes - Auto-decompress: decompress/decompressAsync and createDecompressStream now handle LZ4 data automatically Closes #176 * fix(core): add Read import to LZ4 test modules Tests use read_to_end which requires the Read trait in scope. * refactor(core): validate capacity as non-negative integer (#183) * refactor(core): validate capacity as non-negative integer across all algorithms Extract shared validate_capacity() that rejects NaN, Infinity, negative, fractional values, and values exceeding usize::MAX. Applied to all 10 DecompressWithCapacity functions (sync + async × 5 algorithms). Closes #181 * test: add regression tests for fractional and oversized capacity * perf(lz4): implement incremental streaming for LZ4 compression (#184) * perf(lz4): implement incremental streaming for LZ4 compression Replace full-input buffering with cursor-based incremental output in Lz4CompressContext. The FrameEncoder now persists across transform() calls, emitting compressed blocks as they become available instead of buffering all input until finish(). Decompression remains buffered due to lz4_flex FrameDecoder requiring full input (documented in #182). Closes #182 * perf(lz4): implement incremental streaming for LZ4 compression Replace full-input buffering with FrameEncoder-based incremental output in Lz4CompressContext. Compressed blocks are emitted as internal buffers fill, and old bytes are drained to bound memory usage. Decompression remains buffered due to lz4_flex FrameDecoder requiring full input (documented in #182). Closes #182 * chore(deps): update taiki-e/install-action digest to 0620367 (#185) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update rust crate lz4_flex to 0.13 (#186) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore: rename package from zflate to comprs (#190) * chore: rename package from zflate to comprs Rename npm package, Rust crates, napi binary name, platform packages, and all references from "zflate" to "comprs". The name "comprs" conveys "compress" with a Rust flavor, is available on npm, and avoids the typosquat protection that blocks "zflate". Changes across 70 files: - npm: comprs, comprs-darwin-arm64, comprs-wasm32-wasi, etc. - Rust: comprs (core), comprs-bench - napi binaryName: comprs (→ comprs.*.node, comprs.*.wasm) - README, workflows, changesets, tests, benchmarks GitHub repository URL (derodero24/zflate) is preserved. Closes #189 * chore: update GitHub repo URLs to derodero24/comprs Update all repository references to match the planned GitHub repo rename from derodero24/zflate to derodero24/comprs. * chore(core): rename ZflateError to ComprsError and update bench vars (#192) Rename internal Rust error type `ZflateError` → `ComprsError` across all source files and update benchmark variable names `*_ZFLATE` → `*_COMPRS` for consistency with the package rename in #190. Closes #191 * fix: add missing LZ4 docs, browser exports, Node.js transforms, and publish files (#194) - Add LZ4 one-shot and streaming context exports to browser-entry.js - Add Lz4CompressContext/Lz4DecompressContext adapters to browser-streaming.js - Add createLz4CompressTransform/createLz4DecompressTransform to node.js - Add LZ4 type declarations to node.d.ts - Add 'lz4' case to createDecompressContext in node.js - Add browser-entry.js and browser-streaming.js to package.json files array - Document LZ4 API in README.md (one-shot, async, streaming, comparison) - Fix streaming import path in README.md (comprs → comprs/streams) - Update auto-detect descriptions to include LZ4 - Add "lz4" to package.json keywords and description - Add LZ4 changeset for upcoming release Closes #193 * fix: add missing LZ4 exports to ESM entry point and complete API docs (#196) - Add LZ4 one-shot APIs, Context classes, and streaming functions to index.mjs (ESM users could not access LZ4 at all) - Add createLz4CompressStream/createLz4DecompressStream to index.mjs - Add gzipCompressWithHeader and gzipReadHeader to browser-entry.js - Document gzipCompressWithHeader, gzipReadHeader in README - Document crc32 and version() utilities in README - Document createDecompressStream and createDecompressTransform in README Closes #195 * fix(docs): add missing lz4 to decompressAsync doc and clarify async section (#198) The decompressAsync Rust doc comment listed only zstd, gzip, brotli as supported formats, but the implementation also handles lz4. The README async section claimed "all one-shot functions" have async variants, which is inaccurate for utility functions like crc32, version, gzipReadHeader. Closes #197 * chore(release): version packages (#35) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: Shintaro Okada <38486312+derodero24@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> Co-authored-by: derodero24 <shintaro@payn.io>
derodero24
added a commit
that referenced
this pull request
Mar 24, 2026
* Set up napi-rs build pipeline (#9) * chore(core): add Rust crate skeleton with napi-rs bindings Minimal `zflate` crate with a single `version()` export to verify the napi-rs build pipeline works end-to-end. - crates/core/Cargo.toml: cdylib crate with napi-rs v3 - crates/core/build.rs: napi_build setup - crates/core/src/lib.rs: version() function * chore: add package.json with napi-rs configuration Configure the npm package with 9 build targets (macOS, Linux, Windows, WASM), dev dependencies for tooling (Biome, Vitest, TypeScript, Changesets, CodSpeed), and build/test/lint scripts. * chore: add build scripts and module entry points - scripts/optimize-wasm.js: post-process WASM with wasm-opt - index.mjs / index.d.mts: ESM entry points - index.js / index.d.ts / browser.js: napi-rs generated files (force-tracked) * Add CI workflow (#10) * test: add initial test suite and ESM smoke test - __test__/index.spec.ts: Vitest spec verifying version() export - __test__/esm-import.mjs: ESM import smoke test for CI * chore: update biome and tsconfig for test files - biome.json: allow console in ESM smoke test - tsconfig.json: format with Biome * ci: add CI workflow Full CI pipeline adapted from rapid-fuzzy: - Lint (Biome, TypeScript, publint, generated file verification) - Dependency audit (npm + cargo-deny) - Rust lint (fmt + clippy) and test - Cross-platform build (9 targets: macOS, Linux, Windows, WASM) - Node.js test matrix (3 versions × 3 OSes) - Coverage (Rust via cargo-llvm-cov + JS via Vitest, uploaded to Codecov) * Add release and automation workflows (#11) * ci: add release workflow with Cargo.toml version sync Changesets-driven release flow: - On develop push: create version PR, then release PR (develop → main) - On main push: build all 9 targets and publish to npm with provenance - scripts/sync-cargo-version.js: keep Cargo.toml version in sync with package.json after changeset version bumps * ci: add CodeQL security analysis Analyze JavaScript/TypeScript and Rust code for security vulnerabilities. Runs on PRs, develop pushes, and weekly schedule (Monday 6 AM UTC). * ci: add CodSpeed benchmark workflow Continuous performance tracking via CodSpeed. Runs JS benchmarks on develop pushes and PRs when crate or bench files change. Rust benchmarks will be added when the bench crate is introduced. * ci: add self-hosted Renovate workflow Automated dependency updates via self-hosted Renovate bot. Runs weekly on Sunday 19:00 UTC (Monday 4:00 JST). * test(bench): add initial benchmark file Minimal benchmark to satisfy CodSpeed workflow. Real compression benchmarks will be added alongside algorithm implementations. * Implement zstd compression and decompression (#12) * feat(zstd): implement zstd compression and decompression Add zstdCompress() and zstdDecompress() functions powered by the zstd C library via the zstd Rust crate. - Compression levels 1-22 (default: 3) - Auto-detection of frame content size for efficient decompression - zstdDecompressWithCapacity() for data exceeding 256 MB default limit - Rust unit tests for round-trip, empty data, large data, and levels * test(zstd): add JS tests and benchmarks for zstd - __test__/zstd.spec.ts: 11 tests covering round-trip, levels, errors - __test__/esm-import.mjs: updated with zstd round-trip smoke test - __test__/index.bench.ts: compress/decompress benchmarks at 150B/10KB/1MB - index.mjs: export zstd functions * chore: update napi-rs generated files and add changeset - Regenerate index.js, index.d.ts, browser.js with zstd exports - Add changeset for minor version bump * ci: add WASI SDK setup for zstd WASM builds The zstd crate compiles C code via zstd-sys, which requires a proper WASI sysroot when targeting wasm32-wasip1-threads. Without it, clang picks up system glibc headers and fails with missing 'bits/libc-header-start.h'. Use bytecodealliance/actions/wasi-sdk/setup to install WASI SDK and set CC, AR, CFLAGS, and WASI_SYSROOT environment variables for the WASM build step in both CI and release workflows. * Fix CodeRabbit review findings from #12 (#22) * fix(zstd): validate capacity parameter in zstdDecompressWithCapacity Reject negative, NaN, and Infinity values before casting f64 to usize. Without validation, a negative f64 wraps to a very large usize, which could cause memory exhaustion. Closes #17 * fix(ci): use RUNNER_TEMP for WASI SDK installation Replace /opt with $RUNNER_TEMP for WASI SDK extraction. /opt may not be writable on all GitHub Actions runner configurations. Also switch curl from -sL to -fsSL to fail on HTTP errors. Closes #18 * docs: add SECURITY.md (#23) * docs: add SECURITY.md * docs: address review feedback on SECURITY.md - Replace vague "extremely large size" with "massive size" for clarity - Unify SLA day units by removing "business" qualifier from initial assessment timeline for consistency * docs: add CONTRIBUTING.md (#24) * test(zstd): expand test coverage and benchmarks (#25) * test(zstd): expand test coverage and benchmarks Add comprehensive JS tests covering compression level boundaries (0, negative, 22), decompressWithCapacity edge cases (insufficient, oversized, invalid inputs), and round-trip verification with random data. Expand benchmarks with random (incompressible) and realistic (JSON, text) data variants alongside existing patterned data. Update Rust doc comments to document negative compression levels and level 0 behavior. Add Rust tests for boundary levels. Update changeset to include all public API functions. Closes #19 * chore: update napi-rs generated type definitions Regenerate index.d.ts to reflect updated doc comments for zstd_compress documenting negative levels and level 0 behavior. * feat(zstd): add streaming compression/decompression API (#26) Implement streaming zstd compression and decompression using the Web Streams API (TransformStream). The Rust layer exposes ZstdCompressContext and ZstdDecompressContext classes with transform/flush/finish methods using zstd::stream::raw for stateful chunked processing. The JS layer wraps these in TransformStream via createZstdCompressStream() and createZstdDecompressStream(), available from both ESM and CJS entry points. Streaming output is fully interoperable with one-shot APIs. Add comprehensive tests covering chunked round-trips, various chunk sizes, large data (1MB), random data, and one-shot/streaming interop. Closes #5 * docs: add README and verify npm package readiness (#28) * docs: add README and verify npm package readiness Add comprehensive README with project description, installation guide, quick-start examples (one-shot and streaming), full API reference, supported algorithms table, platform compatibility matrix, and benchmark results. Verify npm package with publint (no errors). Package includes all required files: index.js, index.mjs, index.d.ts, index.d.mts, streams.js, streams.d.ts, browser.js, LICENSE, and README.md. Closes #8 * docs(readme): address review feedback - Add bun installation instructions - Replace streaming example with self-contained runnable snippet * fix(ci): resolve release workflow failures (#29) * fix(ci): resolve release workflow failures - Generate npm platform package directories for all 9 targets - Add optionalDependencies to root package.json for platform binaries - Fix changeset version command argument parsing in release workflow by using a dedicated npm script instead of inline shell command Closes #27 * fix(ci): remove optionalDependencies from committed package.json The optionalDependencies for platform-specific packages are added at publish time by the prepublishOnly hook (napi prepublish -t npm), not in the committed source. Having them in package.json breaks pnpm install --frozen-lockfile since the packages are not yet on npm. * chore(release): version packages (#30) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(ci): regenerate napi bindings for v0.2.0 and fix version workflow (#34) - Regenerate index.js with v0.2.0 version strings - Update npm platform package versions to 0.2.0 - Add Rust toolchain to version job for napi build during version bump - Add napi version, build, and format to changeset:version script - Fix version test to read from package.json dynamically * feat(gzip): add gzip and deflate compression support (#33) Add gzip and raw deflate compression/decompression with one-shot and streaming APIs. Includes interoperability tests with Node.js built-in zlib module. Closes #6 * feat(brotli): add brotli compression support (#32) * feat(brotli): add brotli compression support Add brotli compression/decompression with one-shot and streaming APIs. Quality levels 0-11 (default: 6). Includes interoperability tests with Node.js built-in zlib brotli module. Closes #7 * test(brotli): add brotli compression benchmarks Add brotli compress/decompress benchmarks matching the zstd benchmark structure for patterned, random, and realistic data at various sizes. * test(gzip): increase timeout for 1MB round-trip tests Extend timeout from 5s to 30s for gzip and deflate 1MB data tests to prevent CI failures on slower runners. * test: increase timeout for all 1MB tests Set 30s timeout on all 1MB data round-trip and streaming tests to prevent failures on slower CI runners (Windows). * docs(readme): update API reference for gzip and brotli (#36) Add gzip, deflate, and brotli to the API reference tables. Update supported algorithms table to reflect all three algorithms are now available with both one-shot and streaming APIs. * chore(deps): update taiki-e/install-action digest to c12d62a (#20) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(exports): add ./streams subpath export to package.json (#58) Add a `./streams` subpath export so users can import streaming APIs directly via `import { ... } from 'zflate/streams'`. Closes #48 * fix(gzip): add compression level validation for gzip and deflate (#60) Validate that the compression level parameter is within the valid range (0-9) for gzip_compress, deflate_compress, GzipCompressContext, and DeflateCompressContext. Return InvalidArg error if out of range, matching the existing validation pattern in brotli_compress. Closes #38 * fix(gzip): add decompression size limits to gzip and deflate (#64) * fix(gzip): add decompression size limits to gzip and deflate Add MAX_DECOMPRESSED_SIZE (256 MB) to gzip_decompress and deflate_decompress to prevent zip bomb attacks. Both functions now use chunked reading with size checking instead of unbounded read_to_end(). Add gzipDecompressWithCapacity and deflateDecompressWithCapacity for explicit control over the maximum decompressed size, matching the existing zstd and brotli patterns. Closes #37 * chore(napi): regenerate bindings for new WithCapacity functions * feat(api): accept Uint8Array input in all compression functions (#69) Change all input parameters from Buffer to Either<Buffer, Uint8Array> across all one-shot and streaming functions. This generates Buffer | Uint8Array TypeScript types, enabling browser/Deno/Bun users to pass Uint8Array directly without type errors. Return types remain Buffer. No runtime behavior changes for existing Buffer usage. Closes #42 * docs: enhance README with examples, comparison table, and migration guides (#63) * test: add cargo-fuzz targets for decompression fuzzing (#71) * test: add edge case and error handling tests (#65) * test: expand Node.js zlib interop tests to brotli and zstd (#67) * ci: add hand-written JS files to workflow path filters (#56) * ci(codeql): suppress rust/access-invalid-pointer false positives (#57) * fix(dx): add ./streams subpath export to package.json (#59) * ci: add cargo-audit to dependency audit pipeline (#61) * feat(core): add unified auto-detect decompression API (#72) Add decompress() function that auto-detects compression format from magic bytes (zstd: 0xFD2FB528, gzip: 0x1F8B) and falls back to brotli heuristic detection. Add detectFormat() for format identification without decompression. Supported formats: zstd, gzip, brotli. Raw deflate cannot be auto-detected due to lack of magic bytes. Closes #16 * ci(dx): add @arethetypeswrong/cli type validation (#62) * perf(bench): add competitive benchmarks across all algorithms (#70) * perf(wasm): add WASM binary size reporting to CI (#66) * fix(exports): add missing ESM exports for new functions (#74) Add decompress, detectFormat, gzipDecompressWithCapacity, and deflateDecompressWithCapacity to index.mjs ESM export list. These functions were available via CJS but missing from ESM imports. Update ESM smoke test to cover all WithCapacity functions, decompress, and detectFormat. Closes #73 * feat(node): add Node.js stream.Transform compatibility layer (#76) Add `zflate/node` subpath export with stream.Transform factories for all compression algorithms (zstd, gzip, deflate, brotli), enabling use with Node.js stream.pipeline() and pipe-based workflows. Closes #43 * fix(brotli): use consistent error message for capacity exceeded (#78) Change brotli_decompress_with_capacity error message from "Destination buffer is too small" to "brotli decompress exceeded maximum size of {} bytes" to match the pattern used by all other decompress functions. Closes #77 * fix(docs): remove merge conflict marker and stale src/ references (#80) * feat(core): add async compression/decompression API (#82) Add async versions of all one-shot compression/decompression functions using napi-rs AsyncTask with the Task trait. This runs compression on the libuv thread pool without blocking the Node.js event loop. New functions: zstdCompressAsync, zstdDecompressAsync, gzipCompressAsync, gzipDecompressAsync, deflateCompressAsync, deflateDecompressAsync, brotliCompressAsync, brotliDecompressAsync. Closes #40 * feat(zstd): add dictionary compression API (#83) Add zstd dictionary training, compression, and decompression functions: - zstdTrainDictionary: train a dictionary from sample data - zstdCompressWithDict: compress with a pre-trained dictionary - zstdDecompressWithDict: decompress data compressed with a dictionary Enable the zdict_builder feature for the zstd crate to support dictionary training. Include Rust unit tests and Vitest integration tests verifying round-trip correctness and improved compression ratios for small, similar data. Closes #41 * docs: remove Cloudflare Workers from supported platforms (#91) * ci: include Node.js WASM loader files in CI artifact (#90) * test(wasm): add Node.js WASM compatibility test with native parity verification (#92) * refactor(core): introduce thiserror for structured error types (#93) Replace manual `Error::new(Status::..., ...)` calls with structured `ZflateError` enum variants across all compression modules. Error messages remain identical to preserve backward compatibility. Closes #51 * test(deno): add Deno WASM smoke test (#96) * test(bun): add Bun WASM smoke test (#95) * test(browser): add Playwright browser WASM test (#94) * ci(security): tighten cargo-deny bans and sources to deny mode (#108) * ci: pin dtolnay/rust-toolchain to SHA digest (#107) * fix(core): add decompression size limits to streaming contexts (#109) Add cumulative output size tracking to all streaming decompression contexts (ZstdDecompressContext, GzipDecompressContext, DeflateDecompressContext, BrotliDecompressContext). Each context now enforces MAX_DECOMPRESSED_SIZE (256 MB) across transform() calls, preventing decompression bomb attacks that could cause unbounded memory growth. Closes #97 * feat(core): narrow detectFormat return type to string literal union (#111) Use napi-rs string_enum to return CompressionFormat instead of plain string, generating a const enum with 'zstd' | 'gzip' | 'brotli' | 'unknown' values in TypeScript. Closes #103 * perf(core): optimize buffer allocation in compression and decompression (#110) * perf(core): optimize buffer allocation in compression and decompression Increase decompression read buffer from 4KB to 32KB in gzip, brotli, and brotli_stream modules to reduce syscall overhead during chunked reads. Pre-allocate compression output Vecs with input.len() capacity for gzip and deflate encoders (both sync and async) to avoid repeated reallocations during compression. Closes #100 * revert: keep BUFFER_SIZE at 4096 to avoid decompress regression The 32KB buffer caused 21 benchmark regressions in CodSpeed due to increased stack zeroing cost. Revert to 4KB. The Vec::with_capacity optimization for compression output is retained. * perf(core): replace Vec clone and split_off with std::mem::take in streaming contexts (#112) Use std::mem::take() instead of .get_ref().clone() + .get_mut().clear() in brotli streaming contexts, and instead of .split_off(0) in gzip/deflate streaming contexts. This avoids unnecessary memory allocation and copying by moving ownership of the buffer contents in-place. Closes #99 * fix(zstd): add compression level validation (#113) Validate that the compression level is within the valid zstd range (-131072 to 22) before passing it to the zstd library. This provides clear error messages instead of opaque library errors. Validation is added to: - zstd_compress (sync) - zstd_compress_async - ZstdCompressContext::new (streaming) - zstd_compress_with_dict Closes #81 * feat(zstd): add streaming dictionary compression/decompression (#114) Add ZstdCompressDictContext and ZstdDecompressDictContext classes that accept a pre-trained dictionary for streaming compression/decompression. Include Web Streams (createZstdCompressDictStream/createZstdDecompressDictStream) and Node.js Transform (createZstdCompressDictTransform/createZstdDecompressDictTransform) wrappers, type definitions, ESM exports, and streaming dict round-trip tests. Closes #104 * perf(core): switch flate2 backend from miniz_oxide to zlib-rs (#115) Replace the `rust_backend` (miniz_oxide) feature with `zlib-rs` for 2-3x faster gzip/deflate compression. zlib-rs is a pure-Rust implementation that works on all targets including WASM, is ISRG audited, and is on track to become flate2's default backend. Relax compression level size ordering test since zlib-rs may produce slightly different output sizes per level than miniz_oxide — round-trip correctness is what matters. Closes #98 * perf(zstd): enable multi-threaded compression for native targets (#116) Add `zstdmt` feature for non-WASM targets via conditional dependency. This enables zstd's internal thread pool for compression, improving performance by 20-80% for large inputs on multi-core systems. WASM targets are excluded because wasi-threads is experimental and the interaction with napi-rs worker-based threading is untested. Closes #102 * docs: comprehensive README update with missing APIs and SVG benchmark (#118) * fix(docs): correct benchmark environment label to Apple M2 / Node.js v22 (#119) * perf(wasm): add --converge flag to wasm-opt optimization (#122) Re-run optimization passes until no further gains are achieved. Increase timeout to 5 minutes to accommodate additional passes. Closes #121 * perf(core): use 4x input size for decompression output pre-allocation (#123) Compressed data typically decompresses to several times its original size. Pre-allocating with input.len() alone leads to frequent reallocations during decompression. Using input.len() * 4 as the initial capacity provides a better heuristic while the existing .min() caps continue to prevent excessive allocation. Closes #120 * ci: migrate release workflow to npm trusted publishing (OIDC) (#68) * ci: migrate release workflow to npm trusted publishing (OIDC) Remove NPM_TOKEN/NODE_AUTH_TOKEN secret dependency and rely on OIDC token exchange via id-token:write permission (already set). Trusted publishing must be configured on npmjs.com for each package before the next release. * fix(ci): restore pinned SHA for dtolnay/rust-toolchain Keep the pinned commit hash consistent with ci.yml and codspeed.yml for supply chain security. The rust-toolchain change was out of scope for the trusted publishing migration. * fix(gzip): support concatenated gzip streams in decompression (#131) Replace GzDecoder with MultiGzDecoder from flate2 to correctly handle concatenated gzip members in both one-shot and streaming decompression. * fix(streaming): add finish-guard state tracking for brotli and zstd compress contexts (#132) * feat(core): add missing async variants for one-shot APIs (#133) * feat(streaming): add auto-detect decompression streams (#145) * docs: add troubleshooting section to CONTRIBUTING.md (#146) * docs: add CODE_OF_CONDUCT.md (#149) * ci: apply least-privilege permissions to workflows (#147) * chore: improve package.json metadata and exports map (#148) * ci: clean up CI workflow configuration (#150) * feat(core): expose CRC32 utility function (#134) * feat(core): expose CRC32 utility function * test(crc32): use skipIf for conditional Node.js zlib interop test * ci(testing): add coverage thresholds to vitest config (#151) * ci: add aggregate gate job to CI workflow (#152) * feat(streaming): add configurable maxOutputSize to decompression contexts (#153) Add optional maxOutputSize parameter to all streaming decompression context constructors (GzipDecompressContext, DeflateDecompressContext, ZstdDecompressContext, BrotliDecompressContext, ZstdDecompressDictContext). When not specified, the default 256 MB limit is used. The parameter is also passed through all streaming factory functions (createXxxDecompressStream, createXxxDecompressTransform, and createDecompressStream/createDecompressTransform). Closes #129 * ci: add environment protection for npm publish (#154) * ci: add cargo-shear for unused dependency detection (#155) * ci: add WASM binary size tracking on pull requests (#156) * ci: add scripts/ to CI workflow path triggers (#159) * fix(wasm): add JS-side streaming adapters for browser WASM environment (#158) Replace native streaming contexts with JS-side adapters in the browser entry point to work around WebAssembly.Memory growth invalidating ArrayBuffer views. Adapters accumulate chunks and delegate to one-shot APIs, maintaining the same transform()/flush()/finish() API surface. - browser-streaming.js: adapter classes for all 10 streaming contexts - browser-entry.js: re-exports WASM APIs with adapter overrides - package.json: browser condition points to browser-entry.js - e2e: streaming tests enabled for browser WASM * feat(gzip): add header metadata support for compression and reading (#160) Closes #130 * chore(renovate): run daily and remove PR concurrent limit (#164) * chore(renovate): run daily and remove PR concurrent limit Closes #163 * chore(renovate): disable lock file maintenance * chore(deps): update codspeedhq/action digest to 1c8ae48 (#161) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dtolnay/rust-toolchain digest to 631a55b (#162) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update github/codeql-action digest to 3869755 (#165) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update taiki-e/install-action digest to b988c18 (#166) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(bench): exclude third-party benchmarks from CodSpeed tracking (#169) * fix(bench): exclude third-party benchmarks from CodSpeed tracking Split benchmark files into zflate-only (*.bench.ts) and comparison (*.compare.bench.ts) variants. CodSpeed now only tracks zflate's own benchmarks, eliminating false regressions from pako/fflate/node:zlib that are caused by GitHub Actions CPU heterogeneity affecting Callgrind instruction counting. Closes #167 * fix(bench): use deterministic random data in CodSpeed benchmarks Replace non-deterministic randomBytes() with a seeded LCG to ensure reproducible benchmark inputs across runs, reducing CodSpeed instruction count variance. * feat(bench): migrate from CodSpeed to Bencher.dev for regression detection (#170) Replace CodSpeed simulation mode (Callgrind instruction counting) with Bencher.dev (statistical wall-clock analysis). CodSpeed's instruction counting was fundamentally incompatible with this napi-rs project due to GitHub Actions CPU heterogeneity and glibc's CPU-adaptive malloc. Changes: - Add Bencher workflow with t-test threshold for regression detection - Add vitest-to-bmf.js conversion script for vitest bench output - Remove CodSpeed workflow and @codspeed/vitest-plugin dependency - Remove CodSpeed plugin from vitest config Closes #168 * feat(bench): switch to CodSpeed Walltime mode for accurate benchmarks (#172) Replace Bencher.dev with CodSpeed Walltime mode (Macro Runners) which runs benchmarks on dedicated bare-metal hardware, eliminating GitHub Actions CPU variance. CodSpeed also provides native vitest integration via @codspeed/vitest-plugin, removing the need for conversion scripts. Key change: `mode: walltime` instead of the previous `mode: simulation` which used Callgrind instruction counting. Closes #171 * refactor(bench): split benchmark files per algorithm for granular CodSpeed reporting (#174) Split index.bench.ts (mixed zstd+brotli) into separate zstd.bench.ts and brotli.bench.ts so CodSpeed PR reports only show benchmarks relevant to the changed algorithm. Extract shared test fixtures into bench-fixtures.ts and add realistic data benchmarks (JSON, text) to gzip and deflate for consistent coverage across all four algorithms. Closes #173 * perf(core): eliminate double-copy buffer pattern in decompression (#178) * perf(core): eliminate double-copy buffer pattern in decompression Replace the manual 4 KiB read loop (stack buffer → Vec copy) with read_to_end + Take, which writes directly into Vec spare capacity. This eliminates one full copy of every decompressed byte and uses adaptive buffer sizing instead of fixed 4 KiB chunks. Affected: gzip, deflate, brotli (sync + async, default + with-capacity) Closes #175 * fix(core): use saturating_mul for init capacity and update size limit tests Address CodeRabbit review feedback: - Replace len() * 4 with len().saturating_mul(4) across all call sites to prevent overflow on 32-bit targets (wasm32-wasip1-threads) - Replace old chunk-loop size limit tests with Take+read_to_end tests that exercise the actual decompression pattern (avoiding napi::Error symbol resolution issues in standalone test binaries) * chore(bench): add bench:ci script and Rust CodSpeed benchmarks (#179) * chore(bench): add bench:ci script and Rust CodSpeed benchmarks - Add bench:ci script that excludes comparison benchmarks - Simplify CodSpeed workflow to use bench:ci instead of explicit file list - Use glob pattern for bench file path triggers - Add crates/bench workspace member with Criterion benchmarks (gzip, deflate, brotli, zstd) using codspeed-criterion-compat - Add rust-benchmarks job to CodSpeed workflow (simulation mode) Closes #177 * fix(ci): add Cargo.toml, Cargo.lock, package.json to CodSpeed path triggers Ensures CodSpeed benchmarks run when workspace config or dependency lockfiles change, which could affect benchmark behavior. * feat(core): add LZ4 frame compression support (#180) * feat(core): add LZ4 frame compression support Add LZ4 as a fourth compression algorithm using lz4_flex (pure Rust): - Bulk API: lz4Compress, lz4Decompress, lz4DecompressWithCapacity - Async API: lz4CompressAsync, lz4DecompressAsync, lz4DecompressWithCapacityAsync - Streaming: Lz4CompressContext, Lz4DecompressContext - Web Streams: createLz4CompressStream, createLz4DecompressStream - Format detection: detectFormat returns "lz4" for LZ4 frame magic bytes - Auto-decompress: decompress/decompressAsync and createDecompressStream now handle LZ4 data automatically Closes #176 * fix(core): add Read import to LZ4 test modules Tests use read_to_end which requires the Read trait in scope. * refactor(core): validate capacity as non-negative integer (#183) * refactor(core): validate capacity as non-negative integer across all algorithms Extract shared validate_capacity() that rejects NaN, Infinity, negative, fractional values, and values exceeding usize::MAX. Applied to all 10 DecompressWithCapacity functions (sync + async × 5 algorithms). Closes #181 * test: add regression tests for fractional and oversized capacity * perf(lz4): implement incremental streaming for LZ4 compression (#184) * perf(lz4): implement incremental streaming for LZ4 compression Replace full-input buffering with cursor-based incremental output in Lz4CompressContext. The FrameEncoder now persists across transform() calls, emitting compressed blocks as they become available instead of buffering all input until finish(). Decompression remains buffered due to lz4_flex FrameDecoder requiring full input (documented in #182). Closes #182 * perf(lz4): implement incremental streaming for LZ4 compression Replace full-input buffering with FrameEncoder-based incremental output in Lz4CompressContext. Compressed blocks are emitted as internal buffers fill, and old bytes are drained to bound memory usage. Decompression remains buffered due to lz4_flex FrameDecoder requiring full input (documented in #182). Closes #182 * chore(deps): update taiki-e/install-action digest to 0620367 (#185) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update rust crate lz4_flex to 0.13 (#186) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore: rename package from zflate to comprs (#190) * chore: rename package from zflate to comprs Rename npm package, Rust crates, napi binary name, platform packages, and all references from "zflate" to "comprs". The name "comprs" conveys "compress" with a Rust flavor, is available on npm, and avoids the typosquat protection that blocks "zflate". Changes across 70 files: - npm: comprs, comprs-darwin-arm64, comprs-wasm32-wasi, etc. - Rust: comprs (core), comprs-bench - napi binaryName: comprs (→ comprs.*.node, comprs.*.wasm) - README, workflows, changesets, tests, benchmarks GitHub repository URL (derodero24/zflate) is preserved. Closes #189 * chore: update GitHub repo URLs to derodero24/comprs Update all repository references to match the planned GitHub repo rename from derodero24/zflate to derodero24/comprs. * chore(core): rename ZflateError to ComprsError and update bench vars (#192) Rename internal Rust error type `ZflateError` → `ComprsError` across all source files and update benchmark variable names `*_ZFLATE` → `*_COMPRS` for consistency with the package rename in #190. Closes #191 * fix: add missing LZ4 docs, browser exports, Node.js transforms, and publish files (#194) - Add LZ4 one-shot and streaming context exports to browser-entry.js - Add Lz4CompressContext/Lz4DecompressContext adapters to browser-streaming.js - Add createLz4CompressTransform/createLz4DecompressTransform to node.js - Add LZ4 type declarations to node.d.ts - Add 'lz4' case to createDecompressContext in node.js - Add browser-entry.js and browser-streaming.js to package.json files array - Document LZ4 API in README.md (one-shot, async, streaming, comparison) - Fix streaming import path in README.md (comprs → comprs/streams) - Update auto-detect descriptions to include LZ4 - Add "lz4" to package.json keywords and description - Add LZ4 changeset for upcoming release Closes #193 * fix: add missing LZ4 exports to ESM entry point and complete API docs (#196) - Add LZ4 one-shot APIs, Context classes, and streaming functions to index.mjs (ESM users could not access LZ4 at all) - Add createLz4CompressStream/createLz4DecompressStream to index.mjs - Add gzipCompressWithHeader and gzipReadHeader to browser-entry.js - Document gzipCompressWithHeader, gzipReadHeader in README - Document crc32 and version() utilities in README - Document createDecompressStream and createDecompressTransform in README Closes #195 * fix(docs): add missing lz4 to decompressAsync doc and clarify async section (#198) The decompressAsync Rust doc comment listed only zstd, gzip, brotli as supported formats, but the implementation also handles lz4. The README async section claimed "all one-shot functions" have async variants, which is inaccurate for utility functions like crc32, version, gzipReadHeader. Closes #197 * chore(release): version packages (#35) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * perf(ci): remove JS walltime benchmarks from CodSpeed regression detection (#201) JS walltime benchmarks on standard GitHub Actions runners produce 20-50% variance due to CPU heterogeneity (AMD EPYC vs Intel Xeon assigned randomly) and glibc's CPU-adaptive malloc. This caused CodSpeed Performance Analysis to fail on nearly every PR since #174, including documentation-only changes. Keep only Rust simulation benchmarks in CodSpeed, which use instruction counting via Callgrind and produce <1% variance. Also narrow the workflow path triggers to Rust-only files since JS changes no longer affect the CodSpeed pipeline. JS benchmarks remain available locally via `pnpm run bench`. Closes #200 * ci(release): use granular access token for initial npm publish The first publish requires an NPM_TOKEN since OIDC trusted publishing only works for packages that already exist on the registry. After the initial release, this will be switched back to OIDC. --------- Co-authored-by: Shintaro Okada <38486312+derodero24@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> Co-authored-by: derodero24 <shintaro@payn.io>
derodero24
added a commit
that referenced
this pull request
Apr 3, 2026
* Set up napi-rs build pipeline (#9) * chore(core): add Rust crate skeleton with napi-rs bindings Minimal `zflate` crate with a single `version()` export to verify the napi-rs build pipeline works end-to-end. - crates/core/Cargo.toml: cdylib crate with napi-rs v3 - crates/core/build.rs: napi_build setup - crates/core/src/lib.rs: version() function * chore: add package.json with napi-rs configuration Configure the npm package with 9 build targets (macOS, Linux, Windows, WASM), dev dependencies for tooling (Biome, Vitest, TypeScript, Changesets, CodSpeed), and build/test/lint scripts. * chore: add build scripts and module entry points - scripts/optimize-wasm.js: post-process WASM with wasm-opt - index.mjs / index.d.mts: ESM entry points - index.js / index.d.ts / browser.js: napi-rs generated files (force-tracked) * Add CI workflow (#10) * test: add initial test suite and ESM smoke test - __test__/index.spec.ts: Vitest spec verifying version() export - __test__/esm-import.mjs: ESM import smoke test for CI * chore: update biome and tsconfig for test files - biome.json: allow console in ESM smoke test - tsconfig.json: format with Biome * ci: add CI workflow Full CI pipeline adapted from rapid-fuzzy: - Lint (Biome, TypeScript, publint, generated file verification) - Dependency audit (npm + cargo-deny) - Rust lint (fmt + clippy) and test - Cross-platform build (9 targets: macOS, Linux, Windows, WASM) - Node.js test matrix (3 versions × 3 OSes) - Coverage (Rust via cargo-llvm-cov + JS via Vitest, uploaded to Codecov) * Add release and automation workflows (#11) * ci: add release workflow with Cargo.toml version sync Changesets-driven release flow: - On develop push: create version PR, then release PR (develop → main) - On main push: build all 9 targets and publish to npm with provenance - scripts/sync-cargo-version.js: keep Cargo.toml version in sync with package.json after changeset version bumps * ci: add CodeQL security analysis Analyze JavaScript/TypeScript and Rust code for security vulnerabilities. Runs on PRs, develop pushes, and weekly schedule (Monday 6 AM UTC). * ci: add CodSpeed benchmark workflow Continuous performance tracking via CodSpeed. Runs JS benchmarks on develop pushes and PRs when crate or bench files change. Rust benchmarks will be added when the bench crate is introduced. * ci: add self-hosted Renovate workflow Automated dependency updates via self-hosted Renovate bot. Runs weekly on Sunday 19:00 UTC (Monday 4:00 JST). * test(bench): add initial benchmark file Minimal benchmark to satisfy CodSpeed workflow. Real compression benchmarks will be added alongside algorithm implementations. * Implement zstd compression and decompression (#12) * feat(zstd): implement zstd compression and decompression Add zstdCompress() and zstdDecompress() functions powered by the zstd C library via the zstd Rust crate. - Compression levels 1-22 (default: 3) - Auto-detection of frame content size for efficient decompression - zstdDecompressWithCapacity() for data exceeding 256 MB default limit - Rust unit tests for round-trip, empty data, large data, and levels * test(zstd): add JS tests and benchmarks for zstd - __test__/zstd.spec.ts: 11 tests covering round-trip, levels, errors - __test__/esm-import.mjs: updated with zstd round-trip smoke test - __test__/index.bench.ts: compress/decompress benchmarks at 150B/10KB/1MB - index.mjs: export zstd functions * chore: update napi-rs generated files and add changeset - Regenerate index.js, index.d.ts, browser.js with zstd exports - Add changeset for minor version bump * ci: add WASI SDK setup for zstd WASM builds The zstd crate compiles C code via zstd-sys, which requires a proper WASI sysroot when targeting wasm32-wasip1-threads. Without it, clang picks up system glibc headers and fails with missing 'bits/libc-header-start.h'. Use bytecodealliance/actions/wasi-sdk/setup to install WASI SDK and set CC, AR, CFLAGS, and WASI_SYSROOT environment variables for the WASM build step in both CI and release workflows. * Fix CodeRabbit review findings from #12 (#22) * fix(zstd): validate capacity parameter in zstdDecompressWithCapacity Reject negative, NaN, and Infinity values before casting f64 to usize. Without validation, a negative f64 wraps to a very large usize, which could cause memory exhaustion. Closes #17 * fix(ci): use RUNNER_TEMP for WASI SDK installation Replace /opt with $RUNNER_TEMP for WASI SDK extraction. /opt may not be writable on all GitHub Actions runner configurations. Also switch curl from -sL to -fsSL to fail on HTTP errors. Closes #18 * docs: add SECURITY.md (#23) * docs: add SECURITY.md * docs: address review feedback on SECURITY.md - Replace vague "extremely large size" with "massive size" for clarity - Unify SLA day units by removing "business" qualifier from initial assessment timeline for consistency * docs: add CONTRIBUTING.md (#24) * test(zstd): expand test coverage and benchmarks (#25) * test(zstd): expand test coverage and benchmarks Add comprehensive JS tests covering compression level boundaries (0, negative, 22), decompressWithCapacity edge cases (insufficient, oversized, invalid inputs), and round-trip verification with random data. Expand benchmarks with random (incompressible) and realistic (JSON, text) data variants alongside existing patterned data. Update Rust doc comments to document negative compression levels and level 0 behavior. Add Rust tests for boundary levels. Update changeset to include all public API functions. Closes #19 * chore: update napi-rs generated type definitions Regenerate index.d.ts to reflect updated doc comments for zstd_compress documenting negative levels and level 0 behavior. * feat(zstd): add streaming compression/decompression API (#26) Implement streaming zstd compression and decompression using the Web Streams API (TransformStream). The Rust layer exposes ZstdCompressContext and ZstdDecompressContext classes with transform/flush/finish methods using zstd::stream::raw for stateful chunked processing. The JS layer wraps these in TransformStream via createZstdCompressStream() and createZstdDecompressStream(), available from both ESM and CJS entry points. Streaming output is fully interoperable with one-shot APIs. Add comprehensive tests covering chunked round-trips, various chunk sizes, large data (1MB), random data, and one-shot/streaming interop. Closes #5 * docs: add README and verify npm package readiness (#28) * docs: add README and verify npm package readiness Add comprehensive README with project description, installation guide, quick-start examples (one-shot and streaming), full API reference, supported algorithms table, platform compatibility matrix, and benchmark results. Verify npm package with publint (no errors). Package includes all required files: index.js, index.mjs, index.d.ts, index.d.mts, streams.js, streams.d.ts, browser.js, LICENSE, and README.md. Closes #8 * docs(readme): address review feedback - Add bun installation instructions - Replace streaming example with self-contained runnable snippet * fix(ci): resolve release workflow failures (#29) * fix(ci): resolve release workflow failures - Generate npm platform package directories for all 9 targets - Add optionalDependencies to root package.json for platform binaries - Fix changeset version command argument parsing in release workflow by using a dedicated npm script instead of inline shell command Closes #27 * fix(ci): remove optionalDependencies from committed package.json The optionalDependencies for platform-specific packages are added at publish time by the prepublishOnly hook (napi prepublish -t npm), not in the committed source. Having them in package.json breaks pnpm install --frozen-lockfile since the packages are not yet on npm. * chore(release): version packages (#30) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(ci): regenerate napi bindings for v0.2.0 and fix version workflow (#34) - Regenerate index.js with v0.2.0 version strings - Update npm platform package versions to 0.2.0 - Add Rust toolchain to version job for napi build during version bump - Add napi version, build, and format to changeset:version script - Fix version test to read from package.json dynamically * feat(gzip): add gzip and deflate compression support (#33) Add gzip and raw deflate compression/decompression with one-shot and streaming APIs. Includes interoperability tests with Node.js built-in zlib module. Closes #6 * feat(brotli): add brotli compression support (#32) * feat(brotli): add brotli compression support Add brotli compression/decompression with one-shot and streaming APIs. Quality levels 0-11 (default: 6). Includes interoperability tests with Node.js built-in zlib brotli module. Closes #7 * test(brotli): add brotli compression benchmarks Add brotli compress/decompress benchmarks matching the zstd benchmark structure for patterned, random, and realistic data at various sizes. * test(gzip): increase timeout for 1MB round-trip tests Extend timeout from 5s to 30s for gzip and deflate 1MB data tests to prevent CI failures on slower runners. * test: increase timeout for all 1MB tests Set 30s timeout on all 1MB data round-trip and streaming tests to prevent failures on slower CI runners (Windows). * docs(readme): update API reference for gzip and brotli (#36) Add gzip, deflate, and brotli to the API reference tables. Update supported algorithms table to reflect all three algorithms are now available with both one-shot and streaming APIs. * chore(deps): update taiki-e/install-action digest to c12d62a (#20) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(exports): add ./streams subpath export to package.json (#58) Add a `./streams` subpath export so users can import streaming APIs directly via `import { ... } from 'zflate/streams'`. Closes #48 * fix(gzip): add compression level validation for gzip and deflate (#60) Validate that the compression level parameter is within the valid range (0-9) for gzip_compress, deflate_compress, GzipCompressContext, and DeflateCompressContext. Return InvalidArg error if out of range, matching the existing validation pattern in brotli_compress. Closes #38 * fix(gzip): add decompression size limits to gzip and deflate (#64) * fix(gzip): add decompression size limits to gzip and deflate Add MAX_DECOMPRESSED_SIZE (256 MB) to gzip_decompress and deflate_decompress to prevent zip bomb attacks. Both functions now use chunked reading with size checking instead of unbounded read_to_end(). Add gzipDecompressWithCapacity and deflateDecompressWithCapacity for explicit control over the maximum decompressed size, matching the existing zstd and brotli patterns. Closes #37 * chore(napi): regenerate bindings for new WithCapacity functions * feat(api): accept Uint8Array input in all compression functions (#69) Change all input parameters from Buffer to Either<Buffer, Uint8Array> across all one-shot and streaming functions. This generates Buffer | Uint8Array TypeScript types, enabling browser/Deno/Bun users to pass Uint8Array directly without type errors. Return types remain Buffer. No runtime behavior changes for existing Buffer usage. Closes #42 * docs: enhance README with examples, comparison table, and migration guides (#63) * test: add cargo-fuzz targets for decompression fuzzing (#71) * test: add edge case and error handling tests (#65) * test: expand Node.js zlib interop tests to brotli and zstd (#67) * ci: add hand-written JS files to workflow path filters (#56) * ci(codeql): suppress rust/access-invalid-pointer false positives (#57) * fix(dx): add ./streams subpath export to package.json (#59) * ci: add cargo-audit to dependency audit pipeline (#61) * feat(core): add unified auto-detect decompression API (#72) Add decompress() function that auto-detects compression format from magic bytes (zstd: 0xFD2FB528, gzip: 0x1F8B) and falls back to brotli heuristic detection. Add detectFormat() for format identification without decompression. Supported formats: zstd, gzip, brotli. Raw deflate cannot be auto-detected due to lack of magic bytes. Closes #16 * ci(dx): add @arethetypeswrong/cli type validation (#62) * perf(bench): add competitive benchmarks across all algorithms (#70) * perf(wasm): add WASM binary size reporting to CI (#66) * fix(exports): add missing ESM exports for new functions (#74) Add decompress, detectFormat, gzipDecompressWithCapacity, and deflateDecompressWithCapacity to index.mjs ESM export list. These functions were available via CJS but missing from ESM imports. Update ESM smoke test to cover all WithCapacity functions, decompress, and detectFormat. Closes #73 * feat(node): add Node.js stream.Transform compatibility layer (#76) Add `zflate/node` subpath export with stream.Transform factories for all compression algorithms (zstd, gzip, deflate, brotli), enabling use with Node.js stream.pipeline() and pipe-based workflows. Closes #43 * fix(brotli): use consistent error message for capacity exceeded (#78) Change brotli_decompress_with_capacity error message from "Destination buffer is too small" to "brotli decompress exceeded maximum size of {} bytes" to match the pattern used by all other decompress functions. Closes #77 * fix(docs): remove merge conflict marker and stale src/ references (#80) * feat(core): add async compression/decompression API (#82) Add async versions of all one-shot compression/decompression functions using napi-rs AsyncTask with the Task trait. This runs compression on the libuv thread pool without blocking the Node.js event loop. New functions: zstdCompressAsync, zstdDecompressAsync, gzipCompressAsync, gzipDecompressAsync, deflateCompressAsync, deflateDecompressAsync, brotliCompressAsync, brotliDecompressAsync. Closes #40 * feat(zstd): add dictionary compression API (#83) Add zstd dictionary training, compression, and decompression functions: - zstdTrainDictionary: train a dictionary from sample data - zstdCompressWithDict: compress with a pre-trained dictionary - zstdDecompressWithDict: decompress data compressed with a dictionary Enable the zdict_builder feature for the zstd crate to support dictionary training. Include Rust unit tests and Vitest integration tests verifying round-trip correctness and improved compression ratios for small, similar data. Closes #41 * docs: remove Cloudflare Workers from supported platforms (#91) * ci: include Node.js WASM loader files in CI artifact (#90) * test(wasm): add Node.js WASM compatibility test with native parity verification (#92) * refactor(core): introduce thiserror for structured error types (#93) Replace manual `Error::new(Status::..., ...)` calls with structured `ZflateError` enum variants across all compression modules. Error messages remain identical to preserve backward compatibility. Closes #51 * test(deno): add Deno WASM smoke test (#96) * test(bun): add Bun WASM smoke test (#95) * test(browser): add Playwright browser WASM test (#94) * ci(security): tighten cargo-deny bans and sources to deny mode (#108) * ci: pin dtolnay/rust-toolchain to SHA digest (#107) * fix(core): add decompression size limits to streaming contexts (#109) Add cumulative output size tracking to all streaming decompression contexts (ZstdDecompressContext, GzipDecompressContext, DeflateDecompressContext, BrotliDecompressContext). Each context now enforces MAX_DECOMPRESSED_SIZE (256 MB) across transform() calls, preventing decompression bomb attacks that could cause unbounded memory growth. Closes #97 * feat(core): narrow detectFormat return type to string literal union (#111) Use napi-rs string_enum to return CompressionFormat instead of plain string, generating a const enum with 'zstd' | 'gzip' | 'brotli' | 'unknown' values in TypeScript. Closes #103 * perf(core): optimize buffer allocation in compression and decompression (#110) * perf(core): optimize buffer allocation in compression and decompression Increase decompression read buffer from 4KB to 32KB in gzip, brotli, and brotli_stream modules to reduce syscall overhead during chunked reads. Pre-allocate compression output Vecs with input.len() capacity for gzip and deflate encoders (both sync and async) to avoid repeated reallocations during compression. Closes #100 * revert: keep BUFFER_SIZE at 4096 to avoid decompress regression The 32KB buffer caused 21 benchmark regressions in CodSpeed due to increased stack zeroing cost. Revert to 4KB. The Vec::with_capacity optimization for compression output is retained. * perf(core): replace Vec clone and split_off with std::mem::take in streaming contexts (#112) Use std::mem::take() instead of .get_ref().clone() + .get_mut().clear() in brotli streaming contexts, and instead of .split_off(0) in gzip/deflate streaming contexts. This avoids unnecessary memory allocation and copying by moving ownership of the buffer contents in-place. Closes #99 * fix(zstd): add compression level validation (#113) Validate that the compression level is within the valid zstd range (-131072 to 22) before passing it to the zstd library. This provides clear error messages instead of opaque library errors. Validation is added to: - zstd_compress (sync) - zstd_compress_async - ZstdCompressContext::new (streaming) - zstd_compress_with_dict Closes #81 * feat(zstd): add streaming dictionary compression/decompression (#114) Add ZstdCompressDictContext and ZstdDecompressDictContext classes that accept a pre-trained dictionary for streaming compression/decompression. Include Web Streams (createZstdCompressDictStream/createZstdDecompressDictStream) and Node.js Transform (createZstdCompressDictTransform/createZstdDecompressDictTransform) wrappers, type definitions, ESM exports, and streaming dict round-trip tests. Closes #104 * perf(core): switch flate2 backend from miniz_oxide to zlib-rs (#115) Replace the `rust_backend` (miniz_oxide) feature with `zlib-rs` for 2-3x faster gzip/deflate compression. zlib-rs is a pure-Rust implementation that works on all targets including WASM, is ISRG audited, and is on track to become flate2's default backend. Relax compression level size ordering test since zlib-rs may produce slightly different output sizes per level than miniz_oxide — round-trip correctness is what matters. Closes #98 * perf(zstd): enable multi-threaded compression for native targets (#116) Add `zstdmt` feature for non-WASM targets via conditional dependency. This enables zstd's internal thread pool for compression, improving performance by 20-80% for large inputs on multi-core systems. WASM targets are excluded because wasi-threads is experimental and the interaction with napi-rs worker-based threading is untested. Closes #102 * docs: comprehensive README update with missing APIs and SVG benchmark (#118) * fix(docs): correct benchmark environment label to Apple M2 / Node.js v22 (#119) * perf(wasm): add --converge flag to wasm-opt optimization (#122) Re-run optimization passes until no further gains are achieved. Increase timeout to 5 minutes to accommodate additional passes. Closes #121 * perf(core): use 4x input size for decompression output pre-allocation (#123) Compressed data typically decompresses to several times its original size. Pre-allocating with input.len() alone leads to frequent reallocations during decompression. Using input.len() * 4 as the initial capacity provides a better heuristic while the existing .min() caps continue to prevent excessive allocation. Closes #120 * ci: migrate release workflow to npm trusted publishing (OIDC) (#68) * ci: migrate release workflow to npm trusted publishing (OIDC) Remove NPM_TOKEN/NODE_AUTH_TOKEN secret dependency and rely on OIDC token exchange via id-token:write permission (already set). Trusted publishing must be configured on npmjs.com for each package before the next release. * fix(ci): restore pinned SHA for dtolnay/rust-toolchain Keep the pinned commit hash consistent with ci.yml and codspeed.yml for supply chain security. The rust-toolchain change was out of scope for the trusted publishing migration. * fix(gzip): support concatenated gzip streams in decompression (#131) Replace GzDecoder with MultiGzDecoder from flate2 to correctly handle concatenated gzip members in both one-shot and streaming decompression. * fix(streaming): add finish-guard state tracking for brotli and zstd compress contexts (#132) * feat(core): add missing async variants for one-shot APIs (#133) * feat(streaming): add auto-detect decompression streams (#145) * docs: add troubleshooting section to CONTRIBUTING.md (#146) * docs: add CODE_OF_CONDUCT.md (#149) * ci: apply least-privilege permissions to workflows (#147) * chore: improve package.json metadata and exports map (#148) * ci: clean up CI workflow configuration (#150) * feat(core): expose CRC32 utility function (#134) * feat(core): expose CRC32 utility function * test(crc32): use skipIf for conditional Node.js zlib interop test * ci(testing): add coverage thresholds to vitest config (#151) * ci: add aggregate gate job to CI workflow (#152) * feat(streaming): add configurable maxOutputSize to decompression contexts (#153) Add optional maxOutputSize parameter to all streaming decompression context constructors (GzipDecompressContext, DeflateDecompressContext, ZstdDecompressContext, BrotliDecompressContext, ZstdDecompressDictContext). When not specified, the default 256 MB limit is used. The parameter is also passed through all streaming factory functions (createXxxDecompressStream, createXxxDecompressTransform, and createDecompressStream/createDecompressTransform). Closes #129 * ci: add environment protection for npm publish (#154) * ci: add cargo-shear for unused dependency detection (#155) * ci: add WASM binary size tracking on pull requests (#156) * ci: add scripts/ to CI workflow path triggers (#159) * fix(wasm): add JS-side streaming adapters for browser WASM environment (#158) Replace native streaming contexts with JS-side adapters in the browser entry point to work around WebAssembly.Memory growth invalidating ArrayBuffer views. Adapters accumulate chunks and delegate to one-shot APIs, maintaining the same transform()/flush()/finish() API surface. - browser-streaming.js: adapter classes for all 10 streaming contexts - browser-entry.js: re-exports WASM APIs with adapter overrides - package.json: browser condition points to browser-entry.js - e2e: streaming tests enabled for browser WASM * feat(gzip): add header metadata support for compression and reading (#160) Closes #130 * chore(renovate): run daily and remove PR concurrent limit (#164) * chore(renovate): run daily and remove PR concurrent limit Closes #163 * chore(renovate): disable lock file maintenance * chore(deps): update codspeedhq/action digest to 1c8ae48 (#161) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dtolnay/rust-toolchain digest to 631a55b (#162) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update github/codeql-action digest to 3869755 (#165) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update taiki-e/install-action digest to b988c18 (#166) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(bench): exclude third-party benchmarks from CodSpeed tracking (#169) * fix(bench): exclude third-party benchmarks from CodSpeed tracking Split benchmark files into zflate-only (*.bench.ts) and comparison (*.compare.bench.ts) variants. CodSpeed now only tracks zflate's own benchmarks, eliminating false regressions from pako/fflate/node:zlib that are caused by GitHub Actions CPU heterogeneity affecting Callgrind instruction counting. Closes #167 * fix(bench): use deterministic random data in CodSpeed benchmarks Replace non-deterministic randomBytes() with a seeded LCG to ensure reproducible benchmark inputs across runs, reducing CodSpeed instruction count variance. * feat(bench): migrate from CodSpeed to Bencher.dev for regression detection (#170) Replace CodSpeed simulation mode (Callgrind instruction counting) with Bencher.dev (statistical wall-clock analysis). CodSpeed's instruction counting was fundamentally incompatible with this napi-rs project due to GitHub Actions CPU heterogeneity and glibc's CPU-adaptive malloc. Changes: - Add Bencher workflow with t-test threshold for regression detection - Add vitest-to-bmf.js conversion script for vitest bench output - Remove CodSpeed workflow and @codspeed/vitest-plugin dependency - Remove CodSpeed plugin from vitest config Closes #168 * feat(bench): switch to CodSpeed Walltime mode for accurate benchmarks (#172) Replace Bencher.dev with CodSpeed Walltime mode (Macro Runners) which runs benchmarks on dedicated bare-metal hardware, eliminating GitHub Actions CPU variance. CodSpeed also provides native vitest integration via @codspeed/vitest-plugin, removing the need for conversion scripts. Key change: `mode: walltime` instead of the previous `mode: simulation` which used Callgrind instruction counting. Closes #171 * refactor(bench): split benchmark files per algorithm for granular CodSpeed reporting (#174) Split index.bench.ts (mixed zstd+brotli) into separate zstd.bench.ts and brotli.bench.ts so CodSpeed PR reports only show benchmarks relevant to the changed algorithm. Extract shared test fixtures into bench-fixtures.ts and add realistic data benchmarks (JSON, text) to gzip and deflate for consistent coverage across all four algorithms. Closes #173 * perf(core): eliminate double-copy buffer pattern in decompression (#178) * perf(core): eliminate double-copy buffer pattern in decompression Replace the manual 4 KiB read loop (stack buffer → Vec copy) with read_to_end + Take, which writes directly into Vec spare capacity. This eliminates one full copy of every decompressed byte and uses adaptive buffer sizing instead of fixed 4 KiB chunks. Affected: gzip, deflate, brotli (sync + async, default + with-capacity) Closes #175 * fix(core): use saturating_mul for init capacity and update size limit tests Address CodeRabbit review feedback: - Replace len() * 4 with len().saturating_mul(4) across all call sites to prevent overflow on 32-bit targets (wasm32-wasip1-threads) - Replace old chunk-loop size limit tests with Take+read_to_end tests that exercise the actual decompression pattern (avoiding napi::Error symbol resolution issues in standalone test binaries) * chore(bench): add bench:ci script and Rust CodSpeed benchmarks (#179) * chore(bench): add bench:ci script and Rust CodSpeed benchmarks - Add bench:ci script that excludes comparison benchmarks - Simplify CodSpeed workflow to use bench:ci instead of explicit file list - Use glob pattern for bench file path triggers - Add crates/bench workspace member with Criterion benchmarks (gzip, deflate, brotli, zstd) using codspeed-criterion-compat - Add rust-benchmarks job to CodSpeed workflow (simulation mode) Closes #177 * fix(ci): add Cargo.toml, Cargo.lock, package.json to CodSpeed path triggers Ensures CodSpeed benchmarks run when workspace config or dependency lockfiles change, which could affect benchmark behavior. * feat(core): add LZ4 frame compression support (#180) * feat(core): add LZ4 frame compression support Add LZ4 as a fourth compression algorithm using lz4_flex (pure Rust): - Bulk API: lz4Compress, lz4Decompress, lz4DecompressWithCapacity - Async API: lz4CompressAsync, lz4DecompressAsync, lz4DecompressWithCapacityAsync - Streaming: Lz4CompressContext, Lz4DecompressContext - Web Streams: createLz4CompressStream, createLz4DecompressStream - Format detection: detectFormat returns "lz4" for LZ4 frame magic bytes - Auto-decompress: decompress/decompressAsync and createDecompressStream now handle LZ4 data automatically Closes #176 * fix(core): add Read import to LZ4 test modules Tests use read_to_end which requires the Read trait in scope. * refactor(core): validate capacity as non-negative integer (#183) * refactor(core): validate capacity as non-negative integer across all algorithms Extract shared validate_capacity() that rejects NaN, Infinity, negative, fractional values, and values exceeding usize::MAX. Applied to all 10 DecompressWithCapacity functions (sync + async × 5 algorithms). Closes #181 * test: add regression tests for fractional and oversized capacity * perf(lz4): implement incremental streaming for LZ4 compression (#184) * perf(lz4): implement incremental streaming for LZ4 compression Replace full-input buffering with cursor-based incremental output in Lz4CompressContext. The FrameEncoder now persists across transform() calls, emitting compressed blocks as they become available instead of buffering all input until finish(). Decompression remains buffered due to lz4_flex FrameDecoder requiring full input (documented in #182). Closes #182 * perf(lz4): implement incremental streaming for LZ4 compression Replace full-input buffering with FrameEncoder-based incremental output in Lz4CompressContext. Compressed blocks are emitted as internal buffers fill, and old bytes are drained to bound memory usage. Decompression remains buffered due to lz4_flex FrameDecoder requiring full input (documented in #182). Closes #182 * chore(deps): update taiki-e/install-action digest to 0620367 (#185) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update rust crate lz4_flex to 0.13 (#186) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore: rename package from zflate to comprs (#190) * chore: rename package from zflate to comprs Rename npm package, Rust crates, napi binary name, platform packages, and all references from "zflate" to "comprs". The name "comprs" conveys "compress" with a Rust flavor, is available on npm, and avoids the typosquat protection that blocks "zflate". Changes across 70 files: - npm: comprs, comprs-darwin-arm64, comprs-wasm32-wasi, etc. - Rust: comprs (core), comprs-bench - napi binaryName: comprs (→ comprs.*.node, comprs.*.wasm) - README, workflows, changesets, tests, benchmarks GitHub repository URL (derodero24/zflate) is preserved. Closes #189 * chore: update GitHub repo URLs to derodero24/comprs Update all repository references to match the planned GitHub repo rename from derodero24/zflate to derodero24/comprs. * chore(core): rename ZflateError to ComprsError and update bench vars (#192) Rename internal Rust error type `ZflateError` → `ComprsError` across all source files and update benchmark variable names `*_ZFLATE` → `*_COMPRS` for consistency with the package rename in #190. Closes #191 * fix: add missing LZ4 docs, browser exports, Node.js transforms, and publish files (#194) - Add LZ4 one-shot and streaming context exports to browser-entry.js - Add Lz4CompressContext/Lz4DecompressContext adapters to browser-streaming.js - Add createLz4CompressTransform/createLz4DecompressTransform to node.js - Add LZ4 type declarations to node.d.ts - Add 'lz4' case to createDecompressContext in node.js - Add browser-entry.js and browser-streaming.js to package.json files array - Document LZ4 API in README.md (one-shot, async, streaming, comparison) - Fix streaming import path in README.md (comprs → comprs/streams) - Update auto-detect descriptions to include LZ4 - Add "lz4" to package.json keywords and description - Add LZ4 changeset for upcoming release Closes #193 * fix: add missing LZ4 exports to ESM entry point and complete API docs (#196) - Add LZ4 one-shot APIs, Context classes, and streaming functions to index.mjs (ESM users could not access LZ4 at all) - Add createLz4CompressStream/createLz4DecompressStream to index.mjs - Add gzipCompressWithHeader and gzipReadHeader to browser-entry.js - Document gzipCompressWithHeader, gzipReadHeader in README - Document crc32 and version() utilities in README - Document createDecompressStream and createDecompressTransform in README Closes #195 * fix(docs): add missing lz4 to decompressAsync doc and clarify async section (#198) The decompressAsync Rust doc comment listed only zstd, gzip, brotli as supported formats, but the implementation also handles lz4. The README async section claimed "all one-shot functions" have async variants, which is inaccurate for utility functions like crc32, version, gzipReadHeader. Closes #197 * chore(release): version packages (#35) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * perf(ci): remove JS walltime benchmarks from CodSpeed regression detection (#201) JS walltime benchmarks on standard GitHub Actions runners produce 20-50% variance due to CPU heterogeneity (AMD EPYC vs Intel Xeon assigned randomly) and glibc's CPU-adaptive malloc. This caused CodSpeed Performance Analysis to fail on nearly every PR since #174, including documentation-only changes. Keep only Rust simulation benchmarks in CodSpeed, which use instruction counting via Callgrind and produce <1% variance. Also narrow the workflow path triggers to Rust-only files since JS changes no longer affect the CodSpeed pipeline. JS benchmarks remain available locally via `pnpm run bench`. Closes #200 * ci(release): use granular access token for initial npm publish The first publish requires an NPM_TOKEN since OIDC trusted publishing only works for packages that already exist on the registry. After the initial release, this will be switched back to OIDC. * chore: add .mcp.json to .gitignore (#205) Closes #203 * fix(core): use decompress_with_limit in async auto-detect decompression (#216) Replace manual chunk-read loops with double-copy in DecompressTask for gzip and brotli branches. Now uses crate::decompress_with_limit(), matching the pattern already used by the LZ4 branch and per-algorithm async tasks. Closes #207 * ci: add browser files to CI path triggers (#217) Closes #212 * fix(zstd): validate maxDictSize parameter in dictionary training (#219) * test: update ESM smoke test to cover all exports (#215) * Add LZ4 edge-case and streaming test coverage (#218) * test(lz4): add edge-case and streaming test coverage * test(lz4): fix empty buffer test to match actual lz4 behavior LZ4 returns empty buffer for empty input (like zstd and deflate), rather than throwing an error (like gzip and brotli). * fix(zstd): enforce maxOutputSize in browser dict decompression (#221) * fix(zstd): enforce maxOutputSize in browser dict decompression Add zstd_decompress_with_dict_with_capacity Rust function and use it in browser-streaming.js ZstdDecompressDictContext.flush() when maxOutputSize is provided. Closes #208 * chore: regenerate napi-rs binding files * refactor(core): consolidate MAX_DECOMPRESSED_SIZE constant (#222) Remove duplicated MAX_DECOMPRESSED_SIZE definitions from zstd, gzip, brotli_impl, lz4, and detect modules. Change the lib.rs definition to pub(crate) and reference it via crate::MAX_DECOMPRESSED_SIZE in all modules. * chore(release): version packages (#220) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * docs: add benchmark comparison results to README (#224) Closes #213 * feat(brotli): add dictionary compression and decompression support (#225) * docs: add brotli dictionary to README and update Node.js transforms (#228) * docs: readme visual and structural overhaul (#239) - Center-align hero section with title, tagline, and badges - Add Node.js version and TypeScript badges - Add Table of Contents for navigation - Wrap benchmark tables in collapsible <details> sections - Replace plain blockquote with GitHub Admonition (> [!TIP]) - Replace raw build target triples with human-readable table Closes #229 * docs: reposition and enhance comparison table (#240) - Move comparison table from bottom of README to right after "Why comprs?" - Split "Streaming" into "Web Streams API" and "Node.js Transform" rows - Replace ambiguous `✅*` with `⚠️ Experimental*` for node:zlib zstd - Replace footnote `†` with inline "Chunked†" for pako streaming - Update Table of Contents to reflect new section order - Center-align feature columns for better scannability Closes #236 * docs: improve quick start examples and add runtime-specific usage (#241) - Condense one-shot examples to single primary zstd block + multi-algorithm block - Replace impractical manual ReadableStream example with fetch-style streaming - Add Node.js Transform stream example to Quick Start - Add brief comment explaining async runs on libuv thread pool - Consolidate compression levels into compact format - Add Deno and Bun import examples Closes #235 * docs: add algorithm selection guide (#242) - Add "Choosing an Algorithm" section with use case → recommendation table - Add "Choosing an API mode" subsection for sync/async/streaming/dictionary - Update Table of Contents Closes #234 * docs: document browser WASM requirements (SharedArrayBuffer, COOP/COEP) (#243) - Add SharedArrayBuffer requirement warning with required HTTP headers - Remove misleading "or import directly from a CDN" claim - Link to MDN SharedArrayBuffer security requirements Closes #231 * docs: add SSR framework integration guide (#244) - Add "Framework Integration (SSR)" subsection under Browser Usage - Include Next.js serverExternalPackages and Vite SSR external config - Note automatic WASM fallback on client side Closes #232 * docs: add fflate migration guide (#245) - Add "From fflate" migration section with diff example - Note comprs advantages over fflate (zstd, lz4, brotli, dictionaries, Web Streams) Closes #237 * docs: add known limitations and performance notes (#246) - Add "Notes" section with three admonitions covering: - 256 MB default decompression limit with WithCapacity() workaround - Small payload WASM overhead advisory - Brotli decompression performance nuance vs node:zlib Closes #238 * docs: add all undocumented async and capacity API variants to readme (#247) - Add zstdDecompressWithDictWithCapacity to sync zstd dictionary table - Add 9 missing async variants: WithCapacityAsync for all algorithms, zstd dict async, zstdTrainDictionaryAsync, decompressAsync - Reorder async table to group by algorithm with capacity/dict variants Closes #233 * docs: add benchmark comparison SVG charts for cross-algorithm and brotli (#248) - Add cross-algorithm compression chart (zstd/lz4/gzip/brotli at 10KB) - Add brotli compression speedup chart (134.7x faster than node:zlib) - Embed both charts alongside existing gzip chart in Benchmarks section Closes #230 * docs: fold repetitive API tables into collapsible sections (#250) - Wrap Dictionary API (zstd + brotli) in <details> - Replace Async table header with pattern explanation + code example, fold full function list into <details> - Add Streaming import path + fold table into <details> - Keep Node.js Transform code example visible, fold table into <details> - One-shot core API tables remain visible as primary reference Closes #249 * chore(deps): update taiki-e/install-action digest to 7627fb4 (#251) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @biomejs/biome to v2.4.9 (#252) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update renovatebot/github-action action to v46.1.6 (#254) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency vite to v8.0.3 (#253) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update pnpm to v10.33.0 (#256) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update vitest monorepo to v4.1.1 (#255) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency typescript to v6 (#257) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * feat: add interactive playground with GitHub Pages deployment (#259) Add a browser-based playground for trying comprs compression live. Includes four algorithms (zstd, gzip, brotli, lz4), level slider, comparison table, file upload, and download. Deploys to GitHub Pages via new CI workflow. Uses a JS mock for local dev without a WASM build. Closes #258 * docs: improve playground visibility and add OGP/SEO meta tags (#261) Add prominent text link below badges in README header and playground as a bullet in "Why comprs?". Add Open Graph, Twitter Card, and canonical URL meta tags to playground/index.html for social sharing. Closes #260 * fix(playground): handle COI not established on first visit (#263) On first visit, the COI service worker has not yet reloaded the page, so crossOriginIsolated is false. Calling the WASM function immediately throws because SharedArrayBuffer is unavailable. This showed a misleading "unsupported browser" error even in modern browsers like Vivaldi, Brave, or Arc. - Check crossOriginIsolated before invoking WASM - Show "Enabling security features... reloading" while the service worker handles the reload - After 3 s, show a manual Reload button if auto-reload did not happen - Include the actual error message when WASM init itself fails - Add .loading-reload-btn styles to style.css Closes #262 * fix(playground): add Buffer polyfill for emnapi WASM compatibility (#265) The napi-rs WASM runtime (emnapi) requires globalThis.Buffer when creating memory views for function arguments/return values. Browsers do not define Buffer natively, causing zstdCompress and other WASM calls to throw immediately. Add the 'buffer' npm package (v6 browser polyfill) to playground dependencies and set globalThis.Buffer before any WASM calls. Closes #264 * fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init (#267) * fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init Static imports are hoisted and evaluated before any module body code runs. This caused globalThis.Buffer to be unset when comprs-wasm32-wasi evaluated __emnapiInstantiateNapiModuleSync (which calls emnapi_create_memory_view). Switch to a dynamic import so the Buffer polyfill assignment executes first. Closes #266 * fix(playground): allow Vite dev server to serve WASM from parent directory When comprs.wasi-browser.js is present in the repo root, the WASM file is fetched at runtime via /@fs/ from outside the playground directory. Add server.fs.allow so Vite does not block the request with 403. Only applied when a local WASM build is detected (hasLocalWasm=true). * fix(playground): fix comparison table label direction and badge layout (#271) - Rename bar chart column 'Ratio (smaller = better)' to 'Savings' and numeric column 'Ratio' to 'Saved' to accurately reflect that the metric is compression savings % (larger = better) - Change 'best ratio' badge to 'best' (shorter, clearer) - Add 'fastest' badge to the fastest algorithm row; remove the duplicate 'Fastest:' text from the footer note - Change badge column from fixed 72px to auto to prevent overflow Closes #270 * fix(playground): fix negative ratio text, CSS variable, download label, and file size alert (#273) - Show "X% larger" instead of "-X% smaller" when compressed output exceeds input - Fix reload button using undefined var(--accent); use var(--text)/var(--bg) instead - Add file extension to download button (e.g. "Download .zst") via #download-ext span - Replace alert() for oversized file with inline error message that auto-dismisses after 5s Closes #272 * chore(deps): update codecov/codecov-action digest to 75cd116 (#268) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update vitest monorepo to v4.1.2 (#269) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(playground): add FileReader error handler and textarea aria-label (#275) - Add reader.onerror handler to show inline error when file reading fails instead of silently doing nothing (e.g. on disk error or revoked access) - Add aria-label to the input textarea for screen reader accessibility Closes #274 * chore(deps): update dtolnay/rust-toolchain digest to 29eef33 (#276) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update github/codeql-action digest to c10b806 (#277) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update actions/configure-pages action to v6 (#283) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update actions/deploy-pages action to v5 (#284) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update actions/upload-pages-artifact action to v4 (#285) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update taiki-e/install-action digest to d858f81 (#278) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update rust-dependencies to v3.8.4 (#281) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.2 (#280) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @biomejs/biome to v2.4.10 (#279) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update codecov/codecov-action action to v6 (#286) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 (#287) * chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 Update @napi-rs/cli from v3.5.1 to v3.6.0 along with @emnapi/core and @emnapi/runtime from v1.9.1 to v1.9.2 to resolve emnapi version mismatch that caused WASM build failures. * fix(deps): align emnapi transitive dependency to v1.9.2 Update the emnapi package (transitive via @napi-rs/cli) from v1.9.1 to v1.9.2, matching @emnapi/core and @emnapi/runtime versions. * chore(deps): update renovatebot/github-action action to v46.1.7 (#292) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @types/node to v24.12.2 (#291) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(config): update biome.json schema to v2.4.10 (#294) * chore(deps): update dependency @playwright/test to v1.59.1 (#293) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(core): remove unused wildcard import in crc test module (#295) * fix(ci): update setup-deno and changesets/action to resolve Renovate lookup failures (#296) Pin to specific version tags instead of major version floating tags, which do not exist in these repositories. This allows Renovate to resolve digests for tracking updates. - denoland/setup-deno: v2 → v2.0.4 - changesets/action: v1 → v1.7.0 Closes #290 * chore(release): version packages to v0.4.0 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: Shintaro Okada <38486312+derodero24@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> Co-authored-by: derodero24 <shintaro@payn.io>
derodero24
added a commit
that referenced
this pull request
Apr 3, 2026
* Set up napi-rs build pipeline (#9) * chore(core): add Rust crate skeleton with napi-rs bindings Minimal `zflate` crate with a single `version()` export to verify the napi-rs build pipeline works end-to-end. - crates/core/Cargo.toml: cdylib crate with napi-rs v3 - crates/core/build.rs: napi_build setup - crates/core/src/lib.rs: version() function * chore: add package.json with napi-rs configuration Configure the npm package with 9 build targets (macOS, Linux, Windows, WASM), dev dependencies for tooling (Biome, Vitest, TypeScript, Changesets, CodSpeed), and build/test/lint scripts. * chore: add build scripts and module entry points - scripts/optimize-wasm.js: post-process WASM with wasm-opt - index.mjs / index.d.mts: ESM entry points - index.js / index.d.ts / browser.js: napi-rs generated files (force-tracked) * Add CI workflow (#10) * test: add initial test suite and ESM smoke test - __test__/index.spec.ts: Vitest spec verifying version() export - __test__/esm-import.mjs: ESM import smoke test for CI * chore: update biome and tsconfig for test files - biome.json: allow console in ESM smoke test - tsconfig.json: format with Biome * ci: add CI workflow Full CI pipeline adapted from rapid-fuzzy: - Lint (Biome, TypeScript, publint, generated file verification) - Dependency audit (npm + cargo-deny) - Rust lint (fmt + clippy) and test - Cross-platform build (9 targets: macOS, Linux, Windows, WASM) - Node.js test matrix (3 versions × 3 OSes) - Coverage (Rust via cargo-llvm-cov + JS via Vitest, uploaded to Codecov) * Add release and automation workflows (#11) * ci: add release workflow with Cargo.toml version sync Changesets-driven release flow: - On develop push: create version PR, then release PR (develop → main) - On main push: build all 9 targets and publish to npm with provenance - scripts/sync-cargo-version.js: keep Cargo.toml version in sync with package.json after changeset version bumps * ci: add CodeQL security analysis Analyze JavaScript/TypeScript and Rust code for security vulnerabilities. Runs on PRs, develop pushes, and weekly schedule (Monday 6 AM UTC). * ci: add CodSpeed benchmark workflow Continuous performance tracking via CodSpeed. Runs JS benchmarks on develop pushes and PRs when crate or bench files change. Rust benchmarks will be added when the bench crate is introduced. * ci: add self-hosted Renovate workflow Automated dependency updates via self-hosted Renovate bot. Runs weekly on Sunday 19:00 UTC (Monday 4:00 JST). * test(bench): add initial benchmark file Minimal benchmark to satisfy CodSpeed workflow. Real compression benchmarks will be added alongside algorithm implementations. * Implement zstd compression and decompression (#12) * feat(zstd): implement zstd compression and decompression Add zstdCompress() and zstdDecompress() functions powered by the zstd C library via the zstd Rust crate. - Compression levels 1-22 (default: 3) - Auto-detection of frame content size for efficient decompression - zstdDecompressWithCapacity() for data exceeding 256 MB default limit - Rust unit tests for round-trip, empty data, large data, and levels * test(zstd): add JS tests and benchmarks for zstd - __test__/zstd.spec.ts: 11 tests covering round-trip, levels, errors - __test__/esm-import.mjs: updated with zstd round-trip smoke test - __test__/index.bench.ts: compress/decompress benchmarks at 150B/10KB/1MB - index.mjs: export zstd functions * chore: update napi-rs generated files and add changeset - Regenerate index.js, index.d.ts, browser.js with zstd exports - Add changeset for minor version bump * ci: add WASI SDK setup for zstd WASM builds The zstd crate compiles C code via zstd-sys, which requires a proper WASI sysroot when targeting wasm32-wasip1-threads. Without it, clang picks up system glibc headers and fails with missing 'bits/libc-header-start.h'. Use bytecodealliance/actions/wasi-sdk/setup to install WASI SDK and set CC, AR, CFLAGS, and WASI_SYSROOT environment variables for the WASM build step in both CI and release workflows. * Fix CodeRabbit review findings from #12 (#22) * fix(zstd): validate capacity parameter in zstdDecompressWithCapacity Reject negative, NaN, and Infinity values before casting f64 to usize. Without validation, a negative f64 wraps to a very large usize, which could cause memory exhaustion. Closes #17 * fix(ci): use RUNNER_TEMP for WASI SDK installation Replace /opt with $RUNNER_TEMP for WASI SDK extraction. /opt may not be writable on all GitHub Actions runner configurations. Also switch curl from -sL to -fsSL to fail on HTTP errors. Closes #18 * docs: add SECURITY.md (#23) * docs: add SECURITY.md * docs: address review feedback on SECURITY.md - Replace vague "extremely large size" with "massive size" for clarity - Unify SLA day units by removing "business" qualifier from initial assessment timeline for consistency * docs: add CONTRIBUTING.md (#24) * test(zstd): expand test coverage and benchmarks (#25) * test(zstd): expand test coverage and benchmarks Add comprehensive JS tests covering compression level boundaries (0, negative, 22), decompressWithCapacity edge cases (insufficient, oversized, invalid inputs), and round-trip verification with random data. Expand benchmarks with random (incompressible) and realistic (JSON, text) data variants alongside existing patterned data. Update Rust doc comments to document negative compression levels and level 0 behavior. Add Rust tests for boundary levels. Update changeset to include all public API functions. Closes #19 * chore: update napi-rs generated type definitions Regenerate index.d.ts to reflect updated doc comments for zstd_compress documenting negative levels and level 0 behavior. * feat(zstd): add streaming compression/decompression API (#26) Implement streaming zstd compression and decompression using the Web Streams API (TransformStream). The Rust layer exposes ZstdCompressContext and ZstdDecompressContext classes with transform/flush/finish methods using zstd::stream::raw for stateful chunked processing. The JS layer wraps these in TransformStream via createZstdCompressStream() and createZstdDecompressStream(), available from both ESM and CJS entry points. Streaming output is fully interoperable with one-shot APIs. Add comprehensive tests covering chunked round-trips, various chunk sizes, large data (1MB), random data, and one-shot/streaming interop. Closes #5 * docs: add README and verify npm package readiness (#28) * docs: add README and verify npm package readiness Add comprehensive README with project description, installation guide, quick-start examples (one-shot and streaming), full API reference, supported algorithms table, platform compatibility matrix, and benchmark results. Verify npm package with publint (no errors). Package includes all required files: index.js, index.mjs, index.d.ts, index.d.mts, streams.js, streams.d.ts, browser.js, LICENSE, and README.md. Closes #8 * docs(readme): address review feedback - Add bun installation instructions - Replace streaming example with self-contained runnable snippet * fix(ci): resolve release workflow failures (#29) * fix(ci): resolve release workflow failures - Generate npm platform package directories for all 9 targets - Add optionalDependencies to root package.json for platform binaries - Fix changeset version command argument parsing in release workflow by using a dedicated npm script instead of inline shell command Closes #27 * fix(ci): remove optionalDependencies from committed package.json The optionalDependencies for platform-specific packages are added at publish time by the prepublishOnly hook (napi prepublish -t npm), not in the committed source. Having them in package.json breaks pnpm install --frozen-lockfile since the packages are not yet on npm. * chore(release): version packages (#30) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(ci): regenerate napi bindings for v0.2.0 and fix version workflow (#34) - Regenerate index.js with v0.2.0 version strings - Update npm platform package versions to 0.2.0 - Add Rust toolchain to version job for napi build during version bump - Add napi version, build, and format to changeset:version script - Fix version test to read from package.json dynamically * feat(gzip): add gzip and deflate compression support (#33) Add gzip and raw deflate compression/decompression with one-shot and streaming APIs. Includes interoperability tests with Node.js built-in zlib module. Closes #6 * feat(brotli): add brotli compression support (#32) * feat(brotli): add brotli compression support Add brotli compression/decompression with one-shot and streaming APIs. Quality levels 0-11 (default: 6). Includes interoperability tests with Node.js built-in zlib brotli module. Closes #7 * test(brotli): add brotli compression benchmarks Add brotli compress/decompress benchmarks matching the zstd benchmark structure for patterned, random, and realistic data at various sizes. * test(gzip): increase timeout for 1MB round-trip tests Extend timeout from 5s to 30s for gzip and deflate 1MB data tests to prevent CI failures on slower runners. * test: increase timeout for all 1MB tests Set 30s timeout on all 1MB data round-trip and streaming tests to prevent failures on slower CI runners (Windows). * docs(readme): update API reference for gzip and brotli (#36) Add gzip, deflate, and brotli to the API reference tables. Update supported algorithms table to reflect all three algorithms are now available with both one-shot and streaming APIs. * chore(deps): update taiki-e/install-action digest to c12d62a (#20) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(exports): add ./streams subpath export to package.json (#58) Add a `./streams` subpath export so users can import streaming APIs directly via `import { ... } from 'zflate/streams'`. Closes #48 * fix(gzip): add compression level validation for gzip and deflate (#60) Validate that the compression level parameter is within the valid range (0-9) for gzip_compress, deflate_compress, GzipCompressContext, and DeflateCompressContext. Return InvalidArg error if out of range, matching the existing validation pattern in brotli_compress. Closes #38 * fix(gzip): add decompression size limits to gzip and deflate (#64) * fix(gzip): add decompression size limits to gzip and deflate Add MAX_DECOMPRESSED_SIZE (256 MB) to gzip_decompress and deflate_decompress to prevent zip bomb attacks. Both functions now use chunked reading with size checking instead of unbounded read_to_end(). Add gzipDecompressWithCapacity and deflateDecompressWithCapacity for explicit control over the maximum decompressed size, matching the existing zstd and brotli patterns. Closes #37 * chore(napi): regenerate bindings for new WithCapacity functions * feat(api): accept Uint8Array input in all compression functions (#69) Change all input parameters from Buffer to Either<Buffer, Uint8Array> across all one-shot and streaming functions. This generates Buffer | Uint8Array TypeScript types, enabling browser/Deno/Bun users to pass Uint8Array directly without type errors. Return types remain Buffer. No runtime behavior changes for existing Buffer usage. Closes #42 * docs: enhance README with examples, comparison table, and migration guides (#63) * test: add cargo-fuzz targets for decompression fuzzing (#71) * test: add edge case and error handling tests (#65) * test: expand Node.js zlib interop tests to brotli and zstd (#67) * ci: add hand-written JS files to workflow path filters (#56) * ci(codeql): suppress rust/access-invalid-pointer false positives (#57) * fix(dx): add ./streams subpath export to package.json (#59) * ci: add cargo-audit to dependency audit pipeline (#61) * feat(core): add unified auto-detect decompression API (#72) Add decompress() function that auto-detects compression format from magic bytes (zstd: 0xFD2FB528, gzip: 0x1F8B) and falls back to brotli heuristic detection. Add detectFormat() for format identification without decompression. Supported formats: zstd, gzip, brotli. Raw deflate cannot be auto-detected due to lack of magic bytes. Closes #16 * ci(dx): add @arethetypeswrong/cli type validation (#62) * perf(bench): add competitive benchmarks across all algorithms (#70) * perf(wasm): add WASM binary size reporting to CI (#66) * fix(exports): add missing ESM exports for new functions (#74) Add decompress, detectFormat, gzipDecompressWithCapacity, and deflateDecompressWithCapacity to index.mjs ESM export list. These functions were available via CJS but missing from ESM imports. Update ESM smoke test to cover all WithCapacity functions, decompress, and detectFormat. Closes #73 * feat(node): add Node.js stream.Transform compatibility layer (#76) Add `zflate/node` subpath export with stream.Transform factories for all compression algorithms (zstd, gzip, deflate, brotli), enabling use with Node.js stream.pipeline() and pipe-based workflows. Closes #43 * fix(brotli): use consistent error message for capacity exceeded (#78) Change brotli_decompress_with_capacity error message from "Destination buffer is too small" to "brotli decompress exceeded maximum size of {} bytes" to match the pattern used by all other decompress functions. Closes #77 * fix(docs): remove merge conflict marker and stale src/ references (#80) * feat(core): add async compression/decompression API (#82) Add async versions of all one-shot compression/decompression functions using napi-rs AsyncTask with the Task trait. This runs compression on the libuv thread pool without blocking the Node.js event loop. New functions: zstdCompressAsync, zstdDecompressAsync, gzipCompressAsync, gzipDecompressAsync, deflateCompressAsync, deflateDecompressAsync, brotliCompressAsync, brotliDecompressAsync. Closes #40 * feat(zstd): add dictionary compression API (#83) Add zstd dictionary training, compression, and decompression functions: - zstdTrainDictionary: train a dictionary from sample data - zstdCompressWithDict: compress with a pre-trained dictionary - zstdDecompressWithDict: decompress data compressed with a dictionary Enable the zdict_builder feature for the zstd crate to support dictionary training. Include Rust unit tests and Vitest integration tests verifying round-trip correctness and improved compression ratios for small, similar data. Closes #41 * docs: remove Cloudflare Workers from supported platforms (#91) * ci: include Node.js WASM loader files in CI artifact (#90) * test(wasm): add Node.js WASM compatibility test with native parity verification (#92) * refactor(core): introduce thiserror for structured error types (#93) Replace manual `Error::new(Status::..., ...)` calls with structured `ZflateError` enum variants across all compression modules. Error messages remain identical to preserve backward compatibility. Closes #51 * test(deno): add Deno WASM smoke test (#96) * test(bun): add Bun WASM smoke test (#95) * test(browser): add Playwright browser WASM test (#94) * ci(security): tighten cargo-deny bans and sources to deny mode (#108) * ci: pin dtolnay/rust-toolchain to SHA digest (#107) * fix(core): add decompression size limits to streaming contexts (#109) Add cumulative output size tracking to all streaming decompression contexts (ZstdDecompressContext, GzipDecompressContext, DeflateDecompressContext, BrotliDecompressContext). Each context now enforces MAX_DECOMPRESSED_SIZE (256 MB) across transform() calls, preventing decompression bomb attacks that could cause unbounded memory growth. Closes #97 * feat(core): narrow detectFormat return type to string literal union (#111) Use napi-rs string_enum to return CompressionFormat instead of plain string, generating a const enum with 'zstd' | 'gzip' | 'brotli' | 'unknown' values in TypeScript. Closes #103 * perf(core): optimize buffer allocation in compression and decompression (#110) * perf(core): optimize buffer allocation in compression and decompression Increase decompression read buffer from 4KB to 32KB in gzip, brotli, and brotli_stream modules to reduce syscall overhead during chunked reads. Pre-allocate compression output Vecs with input.len() capacity for gzip and deflate encoders (both sync and async) to avoid repeated reallocations during compression. Closes #100 * revert: keep BUFFER_SIZE at 4096 to avoid decompress regression The 32KB buffer caused 21 benchmark regressions in CodSpeed due to increased stack zeroing cost. Revert to 4KB. The Vec::with_capacity optimization for compression output is retained. * perf(core): replace Vec clone and split_off with std::mem::take in streaming contexts (#112) Use std::mem::take() instead of .get_ref().clone() + .get_mut().clear() in brotli streaming contexts, and instead of .split_off(0) in gzip/deflate streaming contexts. This avoids unnecessary memory allocation and copying by moving ownership of the buffer contents in-place. Closes #99 * fix(zstd): add compression level validation (#113) Validate that the compression level is within the valid zstd range (-131072 to 22) before passing it to the zstd library. This provides clear error messages instead of opaque library errors. Validation is added to: - zstd_compress (sync) - zstd_compress_async - ZstdCompressContext::new (streaming) - zstd_compress_with_dict Closes #81 * feat(zstd): add streaming dictionary compression/decompression (#114) Add ZstdCompressDictContext and ZstdDecompressDictContext classes that accept a pre-trained dictionary for streaming compression/decompression. Include Web Streams (createZstdCompressDictStream/createZstdDecompressDictStream) and Node.js Transform (createZstdCompressDictTransform/createZstdDecompressDictTransform) wrappers, type definitions, ESM exports, and streaming dict round-trip tests. Closes #104 * perf(core): switch flate2 backend from miniz_oxide to zlib-rs (#115) Replace the `rust_backend` (miniz_oxide) feature with `zlib-rs` for 2-3x faster gzip/deflate compression. zlib-rs is a pure-Rust implementation that works on all targets including WASM, is ISRG audited, and is on track to become flate2's default backend. Relax compression level size ordering test since zlib-rs may produce slightly different output sizes per level than miniz_oxide — round-trip correctness is what matters. Closes #98 * perf(zstd): enable multi-threaded compression for native targets (#116) Add `zstdmt` feature for non-WASM targets via conditional dependency. This enables zstd's internal thread pool for compression, improving performance by 20-80% for large inputs on multi-core systems. WASM targets are excluded because wasi-threads is experimental and the interaction with napi-rs worker-based threading is untested. Closes #102 * docs: comprehensive README update with missing APIs and SVG benchmark (#118) * fix(docs): correct benchmark environment label to Apple M2 / Node.js v22 (#119) * perf(wasm): add --converge flag to wasm-opt optimization (#122) Re-run optimization passes until no further gains are achieved. Increase timeout to 5 minutes to accommodate additional passes. Closes #121 * perf(core): use 4x input size for decompression output pre-allocation (#123) Compressed data typically decompresses to several times its original size. Pre-allocating with input.len() alone leads to frequent reallocations during decompression. Using input.len() * 4 as the initial capacity provides a better heuristic while the existing .min() caps continue to prevent excessive allocation. Closes #120 * ci: migrate release workflow to npm trusted publishing (OIDC) (#68) * ci: migrate release workflow to npm trusted publishing (OIDC) Remove NPM_TOKEN/NODE_AUTH_TOKEN secret dependency and rely on OIDC token exchange via id-token:write permission (already set). Trusted publishing must be configured on npmjs.com for each package before the next release. * fix(ci): restore pinned SHA for dtolnay/rust-toolchain Keep the pinned commit hash consistent with ci.yml and codspeed.yml for supply chain security. The rust-toolchain change was out of scope for the trusted publishing migration. * fix(gzip): support concatenated gzip streams in decompression (#131) Replace GzDecoder with MultiGzDecoder from flate2 to correctly handle concatenated gzip members in both one-shot and streaming decompression. * fix(streaming): add finish-guard state tracking for brotli and zstd compress contexts (#132) * feat(core): add missing async variants for one-shot APIs (#133) * feat(streaming): add auto-detect decompression streams (#145) * docs: add troubleshooting section to CONTRIBUTING.md (#146) * docs: add CODE_OF_CONDUCT.md (#149) * ci: apply least-privilege permissions to workflows (#147) * chore: improve package.json metadata and exports map (#148) * ci: clean up CI workflow configuration (#150) * feat(core): expose CRC32 utility function (#134) * feat(core): expose CRC32 utility function * test(crc32): use skipIf for conditional Node.js zlib interop test * ci(testing): add coverage thresholds to vitest config (#151) * ci: add aggregate gate job to CI workflow (#152) * feat(streaming): add configurable maxOutputSize to decompression contexts (#153) Add optional maxOutputSize parameter to all streaming decompression context constructors (GzipDecompressContext, DeflateDecompressContext, ZstdDecompressContext, BrotliDecompressContext, ZstdDecompressDictContext). When not specified, the default 256 MB limit is used. The parameter is also passed through all streaming factory functions (createXxxDecompressStream, createXxxDecompressTransform, and createDecompressStream/createDecompressTransform). Closes #129 * ci: add environment protection for npm publish (#154) * ci: add cargo-shear for unused dependency detection (#155) * ci: add WASM binary size tracking on pull requests (#156) * ci: add scripts/ to CI workflow path triggers (#159) * fix(wasm): add JS-side streaming adapters for browser WASM environment (#158) Replace native streaming contexts with JS-side adapters in the browser entry point to work around WebAssembly.Memory growth invalidating ArrayBuffer views. Adapters accumulate chunks and delegate to one-shot APIs, maintaining the same transform()/flush()/finish() API surface. - browser-streaming.js: adapter classes for all 10 streaming contexts - browser-entry.js: re-exports WASM APIs with adapter overrides - package.json: browser condition points to browser-entry.js - e2e: streaming tests enabled for browser WASM * feat(gzip): add header metadata support for compression and reading (#160) Closes #130 * chore(renovate): run daily and remove PR concurrent limit (#164) * chore(renovate): run daily and remove PR concurrent limit Closes #163 * chore(renovate): disable lock file maintenance * chore(deps): update codspeedhq/action digest to 1c8ae48 (#161) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dtolnay/rust-toolchain digest to 631a55b (#162) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update github/codeql-action digest to 3869755 (#165) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update taiki-e/install-action digest to b988c18 (#166) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(bench): exclude third-party benchmarks from CodSpeed tracking (#169) * fix(bench): exclude third-party benchmarks from CodSpeed tracking Split benchmark files into zflate-only (*.bench.ts) and comparison (*.compare.bench.ts) variants. CodSpeed now only tracks zflate's own benchmarks, eliminating false regressions from pako/fflate/node:zlib that are caused by GitHub Actions CPU heterogeneity affecting Callgrind instruction counting. Closes #167 * fix(bench): use deterministic random data in CodSpeed benchmarks Replace non-deterministic randomBytes() with a seeded LCG to ensure reproducible benchmark inputs across runs, reducing CodSpeed instruction count variance. * feat(bench): migrate from CodSpeed to Bencher.dev for regression detection (#170) Replace CodSpeed simulation mode (Callgrind instruction counting) with Bencher.dev (statistical wall-clock analysis). CodSpeed's instruction counting was fundamentally incompatible with this napi-rs project due to GitHub Actions CPU heterogeneity and glibc's CPU-adaptive malloc. Changes: - Add Bencher workflow with t-test threshold for regression detection - Add vitest-to-bmf.js conversion script for vitest bench output - Remove CodSpeed workflow and @codspeed/vitest-plugin dependency - Remove CodSpeed plugin from vitest config Closes #168 * feat(bench): switch to CodSpeed Walltime mode for accurate benchmarks (#172) Replace Bencher.dev with CodSpeed Walltime mode (Macro Runners) which runs benchmarks on dedicated bare-metal hardware, eliminating GitHub Actions CPU variance. CodSpeed also provides native vitest integration via @codspeed/vitest-plugin, removing the need for conversion scripts. Key change: `mode: walltime` instead of the previous `mode: simulation` which used Callgrind instruction counting. Closes #171 * refactor(bench): split benchmark files per algorithm for granular CodSpeed reporting (#174) Split index.bench.ts (mixed zstd+brotli) into separate zstd.bench.ts and brotli.bench.ts so CodSpeed PR reports only show benchmarks relevant to the changed algorithm. Extract shared test fixtures into bench-fixtures.ts and add realistic data benchmarks (JSON, text) to gzip and deflate for consistent coverage across all four algorithms. Closes #173 * perf(core): eliminate double-copy buffer pattern in decompression (#178) * perf(core): eliminate double-copy buffer pattern in decompression Replace the manual 4 KiB read loop (stack buffer → Vec copy) with read_to_end + Take, which writes directly into Vec spare capacity. This eliminates one full copy of every decompressed byte and uses adaptive buffer sizing instead of fixed 4 KiB chunks. Affected: gzip, deflate, brotli (sync + async, default + with-capacity) Closes #175 * fix(core): use saturating_mul for init capacity and update size limit tests Address CodeRabbit review feedback: - Replace len() * 4 with len().saturating_mul(4) across all call sites to prevent overflow on 32-bit targets (wasm32-wasip1-threads) - Replace old chunk-loop size limit tests with Take+read_to_end tests that exercise the actual decompression pattern (avoiding napi::Error symbol resolution issues in standalone test binaries) * chore(bench): add bench:ci script and Rust CodSpeed benchmarks (#179) * chore(bench): add bench:ci script and Rust CodSpeed benchmarks - Add bench:ci script that excludes comparison benchmarks - Simplify CodSpeed workflow to use bench:ci instead of explicit file list - Use glob pattern for bench file path triggers - Add crates/bench workspace member with Criterion benchmarks (gzip, deflate, brotli, zstd) using codspeed-criterion-compat - Add rust-benchmarks job to CodSpeed workflow (simulation mode) Closes #177 * fix(ci): add Cargo.toml, Cargo.lock, package.json to CodSpeed path triggers Ensures CodSpeed benchmarks run when workspace config or dependency lockfiles change, which could affect benchmark behavior. * feat(core): add LZ4 frame compression support (#180) * feat(core): add LZ4 frame compression support Add LZ4 as a fourth compression algorithm using lz4_flex (pure Rust): - Bulk API: lz4Compress, lz4Decompress, lz4DecompressWithCapacity - Async API: lz4CompressAsync, lz4DecompressAsync, lz4DecompressWithCapacityAsync - Streaming: Lz4CompressContext, Lz4DecompressContext - Web Streams: createLz4CompressStream, createLz4DecompressStream - Format detection: detectFormat returns "lz4" for LZ4 frame magic bytes - Auto-decompress: decompress/decompressAsync and createDecompressStream now handle LZ4 data automatically Closes #176 * fix(core): add Read import to LZ4 test modules Tests use read_to_end which requires the Read trait in scope. * refactor(core): validate capacity as non-negative integer (#183) * refactor(core): validate capacity as non-negative integer across all algorithms Extract shared validate_capacity() that rejects NaN, Infinity, negative, fractional values, and values exceeding usize::MAX. Applied to all 10 DecompressWithCapacity functions (sync + async × 5 algorithms). Closes #181 * test: add regression tests for fractional and oversized capacity * perf(lz4): implement incremental streaming for LZ4 compression (#184) * perf(lz4): implement incremental streaming for LZ4 compression Replace full-input buffering with cursor-based incremental output in Lz4CompressContext. The FrameEncoder now persists across transform() calls, emitting compressed blocks as they become available instead of buffering all input until finish(). Decompression remains buffered due to lz4_flex FrameDecoder requiring full input (documented in #182). Closes #182 * perf(lz4): implement incremental streaming for LZ4 compression Replace full-input buffering with FrameEncoder-based incremental output in Lz4CompressContext. Compressed blocks are emitted as internal buffers fill, and old bytes are drained to bound memory usage. Decompression remains buffered due to lz4_flex FrameDecoder requiring full input (documented in #182). Closes #182 * chore(deps): update taiki-e/install-action digest to 0620367 (#185) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update rust crate lz4_flex to 0.13 (#186) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore: rename package from zflate to comprs (#190) * chore: rename package from zflate to comprs Rename npm package, Rust crates, napi binary name, platform packages, and all references from "zflate" to "comprs". The name "comprs" conveys "compress" with a Rust flavor, is available on npm, and avoids the typosquat protection that blocks "zflate". Changes across 70 files: - npm: comprs, comprs-darwin-arm64, comprs-wasm32-wasi, etc. - Rust: comprs (core), comprs-bench - napi binaryName: comprs (→ comprs.*.node, comprs.*.wasm) - README, workflows, changesets, tests, benchmarks GitHub repository URL (derodero24/zflate) is preserved. Closes #189 * chore: update GitHub repo URLs to derodero24/comprs Update all repository references to match the planned GitHub repo rename from derodero24/zflate to derodero24/comprs. * chore(core): rename ZflateError to ComprsError and update bench vars (#192) Rename internal Rust error type `ZflateError` → `ComprsError` across all source files and update benchmark variable names `*_ZFLATE` → `*_COMPRS` for consistency with the package rename in #190. Closes #191 * fix: add missing LZ4 docs, browser exports, Node.js transforms, and publish files (#194) - Add LZ4 one-shot and streaming context exports to browser-entry.js - Add Lz4CompressContext/Lz4DecompressContext adapters to browser-streaming.js - Add createLz4CompressTransform/createLz4DecompressTransform to node.js - Add LZ4 type declarations to node.d.ts - Add 'lz4' case to createDecompressContext in node.js - Add browser-entry.js and browser-streaming.js to package.json files array - Document LZ4 API in README.md (one-shot, async, streaming, comparison) - Fix streaming import path in README.md (comprs → comprs/streams) - Update auto-detect descriptions to include LZ4 - Add "lz4" to package.json keywords and description - Add LZ4 changeset for upcoming release Closes #193 * fix: add missing LZ4 exports to ESM entry point and complete API docs (#196) - Add LZ4 one-shot APIs, Context classes, and streaming functions to index.mjs (ESM users could not access LZ4 at all) - Add createLz4CompressStream/createLz4DecompressStream to index.mjs - Add gzipCompressWithHeader and gzipReadHeader to browser-entry.js - Document gzipCompressWithHeader, gzipReadHeader in README - Document crc32 and version() utilities in README - Document createDecompressStream and createDecompressTransform in README Closes #195 * fix(docs): add missing lz4 to decompressAsync doc and clarify async section (#198) The decompressAsync Rust doc comment listed only zstd, gzip, brotli as supported formats, but the implementation also handles lz4. The README async section claimed "all one-shot functions" have async variants, which is inaccurate for utility functions like crc32, version, gzipReadHeader. Closes #197 * chore(release): version packages (#35) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * perf(ci): remove JS walltime benchmarks from CodSpeed regression detection (#201) JS walltime benchmarks on standard GitHub Actions runners produce 20-50% variance due to CPU heterogeneity (AMD EPYC vs Intel Xeon assigned randomly) and glibc's CPU-adaptive malloc. This caused CodSpeed Performance Analysis to fail on nearly every PR since #174, including documentation-only changes. Keep only Rust simulation benchmarks in CodSpeed, which use instruction counting via Callgrind and produce <1% variance. Also narrow the workflow path triggers to Rust-only files since JS changes no longer affect the CodSpeed pipeline. JS benchmarks remain available locally via `pnpm run bench`. Closes #200 * ci(release): use granular access token for initial npm publish The first publish requires an NPM_TOKEN since OIDC trusted publishing only works for packages that already exist on the registry. After the initial release, this will be switched back to OIDC. * chore: add .mcp.json to .gitignore (#205) Closes #203 * fix(core): use decompress_with_limit in async auto-detect decompression (#216) Replace manual chunk-read loops with double-copy in DecompressTask for gzip and brotli branches. Now uses crate::decompress_with_limit(), matching the pattern already used by the LZ4 branch and per-algorithm async tasks. Closes #207 * ci: add browser files to CI path triggers (#217) Closes #212 * fix(zstd): validate maxDictSize parameter in dictionary training (#219) * test: update ESM smoke test to cover all exports (#215) * Add LZ4 edge-case and streaming test coverage (#218) * test(lz4): add edge-case and streaming test coverage * test(lz4): fix empty buffer test to match actual lz4 behavior LZ4 returns empty buffer for empty input (like zstd and deflate), rather than throwing an error (like gzip and brotli). * fix(zstd): enforce maxOutputSize in browser dict decompression (#221) * fix(zstd): enforce maxOutputSize in browser dict decompression Add zstd_decompress_with_dict_with_capacity Rust function and use it in browser-streaming.js ZstdDecompressDictContext.flush() when maxOutputSize is provided. Closes #208 * chore: regenerate napi-rs binding files * refactor(core): consolidate MAX_DECOMPRESSED_SIZE constant (#222) Remove duplicated MAX_DECOMPRESSED_SIZE definitions from zstd, gzip, brotli_impl, lz4, and detect modules. Change the lib.rs definition to pub(crate) and reference it via crate::MAX_DECOMPRESSED_SIZE in all modules. * chore(release): version packages (#220) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * docs: add benchmark comparison results to README (#224) Closes #213 * feat(brotli): add dictionary compression and decompression support (#225) * docs: add brotli dictionary to README and update Node.js transforms (#228) * docs: readme visual and structural overhaul (#239) - Center-align hero section with title, tagline, and badges - Add Node.js version and TypeScript badges - Add Table of Contents for navigation - Wrap benchmark tables in collapsible <details> sections - Replace plain blockquote with GitHub Admonition (> [!TIP]) - Replace raw build target triples with human-readable table Closes #229 * docs: reposition and enhance comparison table (#240) - Move comparison table from bottom of README to right after "Why comprs?" - Split "Streaming" into "Web Streams API" and "Node.js Transform" rows - Replace ambiguous `✅*` with `⚠️ Experimental*` for node:zlib zstd - Replace footnote `†` with inline "Chunked†" for pako streaming - Update Table of Contents to reflect new section order - Center-align feature columns for better scannability Closes #236 * docs: improve quick start examples and add runtime-specific usage (#241) - Condense one-shot examples to single primary zstd block + multi-algorithm block - Replace impractical manual ReadableStream example with fetch-style streaming - Add Node.js Transform stream example to Quick Start - Add brief comment explaining async runs on libuv thread pool - Consolidate compression levels into compact format - Add Deno and Bun import examples Closes #235 * docs: add algorithm selection guide (#242) - Add "Choosing an Algorithm" section with use case → recommendation table - Add "Choosing an API mode" subsection for sync/async/streaming/dictionary - Update Table of Contents Closes #234 * docs: document browser WASM requirements (SharedArrayBuffer, COOP/COEP) (#243) - Add SharedArrayBuffer requirement warning with required HTTP headers - Remove misleading "or import directly from a CDN" claim - Link to MDN SharedArrayBuffer security requirements Closes #231 * docs: add SSR framework integration guide (#244) - Add "Framework Integration (SSR)" subsection under Browser Usage - Include Next.js serverExternalPackages and Vite SSR external config - Note automatic WASM fallback on client side Closes #232 * docs: add fflate migration guide (#245) - Add "From fflate" migration section with diff example - Note comprs advantages over fflate (zstd, lz4, brotli, dictionaries, Web Streams) Closes #237 * docs: add known limitations and performance notes (#246) - Add "Notes" section with three admonitions covering: - 256 MB default decompression limit with WithCapacity() workaround - Small payload WASM overhead advisory - Brotli decompression performance nuance vs node:zlib Closes #238 * docs: add all undocumented async and capacity API variants to readme (#247) - Add zstdDecompressWithDictWithCapacity to sync zstd dictionary table - Add 9 missing async variants: WithCapacityAsync for all algorithms, zstd dict async, zstdTrainDictionaryAsync, decompressAsync - Reorder async table to group by algorithm with capacity/dict variants Closes #233 * docs: add benchmark comparison SVG charts for cross-algorithm and brotli (#248) - Add cross-algorithm compression chart (zstd/lz4/gzip/brotli at 10KB) - Add brotli compression speedup chart (134.7x faster than node:zlib) - Embed both charts alongside existing gzip chart in Benchmarks section Closes #230 * docs: fold repetitive API tables into collapsible sections (#250) - Wrap Dictionary API (zstd + brotli) in <details> - Replace Async table header with pattern explanation + code example, fold full function list into <details> - Add Streaming import path + fold table into <details> - Keep Node.js Transform code example visible, fold table into <details> - One-shot core API tables remain visible as primary reference Closes #249 * chore(deps): update taiki-e/install-action digest to 7627fb4 (#251) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @biomejs/biome to v2.4.9 (#252) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update renovatebot/github-action action to v46.1.6 (#254) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency vite to v8.0.3 (#253) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update pnpm to v10.33.0 (#256) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update vitest monorepo to v4.1.1 (#255) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency typescript to v6 (#257) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * feat: add interactive playground with GitHub Pages deployment (#259) Add a browser-based playground for trying comprs compression live. Includes four algorithms (zstd, gzip, brotli, lz4), level slider, comparison table, file upload, and download. Deploys to GitHub Pages via new CI workflow. Uses a JS mock for local dev without a WASM build. Closes #258 * docs: improve playground visibility and add OGP/SEO meta tags (#261) Add prominent text link below badges in README header and playground as a bullet in "Why comprs?". Add Open Graph, Twitter Card, and canonical URL meta tags to playground/index.html for social sharing. Closes #260 * fix(playground): handle COI not established on first visit (#263) On first visit, the COI service worker has not yet reloaded the page, so crossOriginIsolated is false. Calling the WASM function immediately throws because SharedArrayBuffer is unavailable. This showed a misleading "unsupported browser" error even in modern browsers like Vivaldi, Brave, or Arc. - Check crossOriginIsolated before invoking WASM - Show "Enabling security features... reloading" while the service worker handles the reload - After 3 s, show a manual Reload button if auto-reload did not happen - Include the actual error message when WASM init itself fails - Add .loading-reload-btn styles to style.css Closes #262 * fix(playground): add Buffer polyfill for emnapi WASM compatibility (#265) The napi-rs WASM runtime (emnapi) requires globalThis.Buffer when creating memory views for function arguments/return values. Browsers do not define Buffer natively, causing zstdCompress and other WASM calls to throw immediately. Add the 'buffer' npm package (v6 browser polyfill) to playground dependencies and set globalThis.Buffer before any WASM calls. Closes #264 * fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init (#267) * fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init Static imports are hoisted and evaluated before any module body code runs. This caused globalThis.Buffer to be unset when comprs-wasm32-wasi evaluated __emnapiInstantiateNapiModuleSync (which calls emnapi_create_memory_view). Switch to a dynamic import so the Buffer polyfill assignment executes first. Closes #266 * fix(playground): allow Vite dev server to serve WASM from parent directory When comprs.wasi-browser.js is present in the repo root, the WASM file is fetched at runtime via /@fs/ from outside the playground directory. Add server.fs.allow so Vite does not block the request with 403. Only applied when a local WASM build is detected (hasLocalWasm=true). * fix(playground): fix comparison table label direction and badge layout (#271) - Rename bar chart column 'Ratio (smaller = better)' to 'Savings' and numeric column 'Ratio' to 'Saved' to accurately reflect that the metric is compression savings % (larger = better) - Change 'best ratio' badge to 'best' (shorter, clearer) - Add 'fastest' badge to the fastest algorithm row; remove the duplicate 'Fastest:' text from the footer note - Change badge column from fixed 72px to auto to prevent overflow Closes #270 * fix(playground): fix negative ratio text, CSS variable, download label, and file size alert (#273) - Show "X% larger" instead of "-X% smaller" when compressed output exceeds input - Fix reload button using undefined var(--accent); use var(--text)/var(--bg) instead - Add file extension to download button (e.g. "Download .zst") via #download-ext span - Replace alert() for oversized file with inline error message that auto-dismisses after 5s Closes #272 * chore(deps): update codecov/codecov-action digest to 75cd116 (#268) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update vitest monorepo to v4.1.2 (#269) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(playground): add FileReader error handler and textarea aria-label (#275) - Add reader.onerror handler to show inline error when file reading fails instead of silently doing nothing (e.g. on disk error or revoked access) - Add aria-label to the input textarea for screen reader accessibility Closes #274 * chore(deps): update dtolnay/rust-toolchain digest to 29eef33 (#276) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update github/codeql-action digest to c10b806 (#277) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update actions/configure-pages action to v6 (#283) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update actions/deploy-pages action to v5 (#284) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update actions/upload-pages-artifact action to v4 (#285) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update taiki-e/install-action digest to d858f81 (#278) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update rust-dependencies to v3.8.4 (#281) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.2 (#280) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @biomejs/biome to v2.4.10 (#279) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update codecov/codecov-action action to v6 (#286) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 (#287) * chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 Update @napi-rs/cli from v3.5.1 to v3.6.0 along with @emnapi/core and @emnapi/runtime from v1.9.1 to v1.9.2 to resolve emnapi version mismatch that caused WASM build failures. * fix(deps): align emnapi transitive dependency to v1.9.2 Update the emnapi package (transitive via @napi-rs/cli) from v1.9.1 to v1.9.2, matching @emnapi/core and @emnapi/runtime versions. * chore(deps): update renovatebot/github-action action to v46.1.7 (#292) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @types/node to v24.12.2 (#291) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(config): update biome.json schema to v2.4.10 (#294) * chore(deps): update dependency @playwright/test to v1.59.1 (#293) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(core): remove unused wildcard import in crc test module (#295) * fix(ci): update setup-deno and changesets/action to resolve Renovate lookup failures (#296) Pin to specific version tags instead of major version floating tags, which do not exist in these repositories. This allows Renovate to resolve digests for tracking updates. - denoland/setup-deno: v2 → v2.0.4 - changesets/action: v1 → v1.7.0 Closes #290 * chore(release): version packages to v0.4.0 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * refactor: rename npm package to @derodero24/comprs (#298) Rename all npm packages from `comprs-*` to `@derodero24/comprs-*` to bypass npm's typosquatting protection that blocked the `comprs` name. - Root package: comprs → @derodero24/comprs - 9 platform packages: comprs-{platform} → @derodero24/comprs-{platform} - Regenerated napi-rs loader (index.js, browser.js) with new names - Updated release workflow version checks - Updated README install/import examples - Updated playground npm link Closes #297 * fix(playground): update vite alias for renamed WASM package Update the vite resolve alias from 'comprs-wasm32-wasi' to '@derodero24/comprs-wasm32-wasi' to match the package rename in #298. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
derodero24
added a commit
that referenced
this pull request
Apr 3, 2026
* Set up napi-rs build pipeline (#9) * chore(core): add Rust crate skeleton with napi-rs bindings Minimal `zflate` crate with a single `version()` export to verify the napi-rs build pipeline works end-to-end. - crates/core/Cargo.toml: cdylib crate with napi-rs v3 - crates/core/build.rs: napi_build setup - crates/core/src/lib.rs: version() function * chore: add package.json with napi-rs configuration Configure the npm package with 9 build targets (macOS, Linux, Windows, WASM), dev dependencies for tooling (Biome, Vitest, TypeScript, Changesets, CodSpeed), and build/test/lint scripts. * chore: add build scripts and module entry points - scripts/optimize-wasm.js: post-process WASM with wasm-opt - index.mjs / index.d.mts: ESM entry points - index.js / index.d.ts / browser.js: napi-rs generated files (force-tracked) * Add CI workflow (#10) * test: add initial test suite and ESM smoke test - __test__/index.spec.ts: Vitest spec verifying version() export - __test__/esm-import.mjs: ESM import smoke test for CI * chore: update biome and tsconfig for test files - biome.json: allow console in ESM smoke test - tsconfig.json: format with Biome * ci: add CI workflow Full CI pipeline adapted from rapid-fuzzy: - Lint (Biome, TypeScript, publint, generated file verification) - Dependency audit (npm + cargo-deny) - Rust lint (fmt + clippy) and test - Cross-platform build (9 targets: macOS, Linux, Windows, WASM) - Node.js test matrix (3 versions × 3 OSes) - Coverage (Rust via cargo-llvm-cov + JS via Vitest, uploaded to Codecov) * Add release and automation workflows (#11) * ci: add release workflow with Cargo.toml version sync Changesets-driven release flow: - On develop push: create version PR, then release PR (develop → main) - On main push: build all 9 targets and publish to npm with provenance - scripts/sync-cargo-version.js: keep Cargo.toml version in sync with package.json after changeset version bumps * ci: add CodeQL security analysis Analyze JavaScript/TypeScript and Rust code for security vulnerabilities. Runs on PRs, develop pushes, and weekly schedule (Monday 6 AM UTC). * ci: add CodSpeed benchmark workflow Continuous performance tracking via CodSpeed. Runs JS benchmarks on develop pushes and PRs when crate or bench files change. Rust benchmarks will be added when the bench crate is introduced. * ci: add self-hosted Renovate workflow Automated dependency updates via self-hosted Renovate bot. Runs weekly on Sunday 19:00 UTC (Monday 4:00 JST). * test(bench): add initial benchmark file Minimal benchmark to satisfy CodSpeed workflow. Real compression benchmarks will be added alongside algorithm implementations. * Implement zstd compression and decompression (#12) * feat(zstd): implement zstd compression and decompression Add zstdCompress() and zstdDecompress() functions powered by the zstd C library via the zstd Rust crate. - Compression levels 1-22 (default: 3) - Auto-detection of frame content size for efficient decompression - zstdDecompressWithCapacity() for data exceeding 256 MB default limit - Rust unit tests for round-trip, empty data, large data, and levels * test(zstd): add JS tests and benchmarks for zstd - __test__/zstd.spec.ts: 11 tests covering round-trip, levels, errors - __test__/esm-import.mjs: updated with zstd round-trip smoke test - __test__/index.bench.ts: compress/decompress benchmarks at 150B/10KB/1MB - index.mjs: export zstd functions * chore: update napi-rs generated files and add changeset - Regenerate index.js, index.d.ts, browser.js with zstd exports - Add changeset for minor version bump * ci: add WASI SDK setup for zstd WASM builds The zstd crate compiles C code via zstd-sys, which requires a proper WASI sysroot when targeting wasm32-wasip1-threads. Without it, clang picks up system glibc headers and fails with missing 'bits/libc-header-start.h'. Use bytecodealliance/actions/wasi-sdk/setup to install WASI SDK and set CC, AR, CFLAGS, and WASI_SYSROOT environment variables for the WASM build step in both CI and release workflows. * Fix CodeRabbit review findings from #12 (#22) * fix(zstd): validate capacity parameter in zstdDecompressWithCapacity Reject negative, NaN, and Infinity values before casting f64 to usize. Without validation, a negative f64 wraps to a very large usize, which could cause memory exhaustion. Closes #17 * fix(ci): use RUNNER_TEMP for WASI SDK installation Replace /opt with $RUNNER_TEMP for WASI SDK extraction. /opt may not be writable on all GitHub Actions runner configurations. Also switch curl from -sL to -fsSL to fail on HTTP errors. Closes #18 * docs: add SECURITY.md (#23) * docs: add SECURITY.md * docs: address review feedback on SECURITY.md - Replace vague "extremely large size" with "massive size" for clarity - Unify SLA day units by removing "business" qualifier from initial assessment timeline for consistency * docs: add CONTRIBUTING.md (#24) * test(zstd): expand test coverage and benchmarks (#25) * test(zstd): expand test coverage and benchmarks Add comprehensive JS tests covering compression level boundaries (0, negative, 22), decompressWithCapacity edge cases (insufficient, oversized, invalid inputs), and round-trip verification with random data. Expand benchmarks with random (incompressible) and realistic (JSON, text) data variants alongside existing patterned data. Update Rust doc comments to document negative compression levels and level 0 behavior. Add Rust tests for boundary levels. Update changeset to include all public API functions. Closes #19 * chore: update napi-rs generated type definitions Regenerate index.d.ts to reflect updated doc comments for zstd_compress documenting negative levels and level 0 behavior. * feat(zstd): add streaming compression/decompression API (#26) Implement streaming zstd compression and decompression using the Web Streams API (TransformStream). The Rust layer exposes ZstdCompressContext and ZstdDecompressContext classes with transform/flush/finish methods using zstd::stream::raw for stateful chunked processing. The JS layer wraps these in TransformStream via createZstdCompressStream() and createZstdDecompressStream(), available from both ESM and CJS entry points. Streaming output is fully interoperable with one-shot APIs. Add comprehensive tests covering chunked round-trips, various chunk sizes, large data (1MB), random data, and one-shot/streaming interop. Closes #5 * docs: add README and verify npm package readiness (#28) * docs: add README and verify npm package readiness Add comprehensive README with project description, installation guide, quick-start examples (one-shot and streaming), full API reference, supported algorithms table, platform compatibility matrix, and benchmark results. Verify npm package with publint (no errors). Package includes all required files: index.js, index.mjs, index.d.ts, index.d.mts, streams.js, streams.d.ts, browser.js, LICENSE, and README.md. Closes #8 * docs(readme): address review feedback - Add bun installation instructions - Replace streaming example with self-contained runnable snippet * fix(ci): resolve release workflow failures (#29) * fix(ci): resolve release workflow failures - Generate npm platform package directories for all 9 targets - Add optionalDependencies to root package.json for platform binaries - Fix changeset version command argument parsing in release workflow by using a dedicated npm script instead of inline shell command Closes #27 * fix(ci): remove optionalDependencies from committed package.json The optionalDependencies for platform-specific packages are added at publish time by the prepublishOnly hook (napi prepublish -t npm), not in the committed source. Having them in package.json breaks pnpm install --frozen-lockfile since the packages are not yet on npm. * chore(release): version packages (#30) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(ci): regenerate napi bindings for v0.2.0 and fix version workflow (#34) - Regenerate index.js with v0.2.0 version strings - Update npm platform package versions to 0.2.0 - Add Rust toolchain to version job for napi build during version bump - Add napi version, build, and format to changeset:version script - Fix version test to read from package.json dynamically * feat(gzip): add gzip and deflate compression support (#33) Add gzip and raw deflate compression/decompression with one-shot and streaming APIs. Includes interoperability tests with Node.js built-in zlib module. Closes #6 * feat(brotli): add brotli compression support (#32) * feat(brotli): add brotli compression support Add brotli compression/decompression with one-shot and streaming APIs. Quality levels 0-11 (default: 6). Includes interoperability tests with Node.js built-in zlib brotli module. Closes #7 * test(brotli): add brotli compression benchmarks Add brotli compress/decompress benchmarks matching the zstd benchmark structure for patterned, random, and realistic data at various sizes. * test(gzip): increase timeout for 1MB round-trip tests Extend timeout from 5s to 30s for gzip and deflate 1MB data tests to prevent CI failures on slower runners. * test: increase timeout for all 1MB tests Set 30s timeout on all 1MB data round-trip and streaming tests to prevent failures on slower CI runners (Windows). * docs(readme): update API reference for gzip and brotli (#36) Add gzip, deflate, and brotli to the API reference tables. Update supported algorithms table to reflect all three algorithms are now available with both one-shot and streaming APIs. * chore(deps): update taiki-e/install-action digest to c12d62a (#20) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(exports): add ./streams subpath export to package.json (#58) Add a `./streams` subpath export so users can import streaming APIs directly via `import { ... } from 'zflate/streams'`. Closes #48 * fix(gzip): add compression level validation for gzip and deflate (#60) Validate that the compression level parameter is within the valid range (0-9) for gzip_compress, deflate_compress, GzipCompressContext, and DeflateCompressContext. Return InvalidArg error if out of range, matching the existing validation pattern in brotli_compress. Closes #38 * fix(gzip): add decompression size limits to gzip and deflate (#64) * fix(gzip): add decompression size limits to gzip and deflate Add MAX_DECOMPRESSED_SIZE (256 MB) to gzip_decompress and deflate_decompress to prevent zip bomb attacks. Both functions now use chunked reading with size checking instead of unbounded read_to_end(). Add gzipDecompressWithCapacity and deflateDecompressWithCapacity for explicit control over the maximum decompressed size, matching the existing zstd and brotli patterns. Closes #37 * chore(napi): regenerate bindings for new WithCapacity functions * feat(api): accept Uint8Array input in all compression functions (#69) Change all input parameters from Buffer to Either<Buffer, Uint8Array> across all one-shot and streaming functions. This generates Buffer | Uint8Array TypeScript types, enabling browser/Deno/Bun users to pass Uint8Array directly without type errors. Return types remain Buffer. No runtime behavior changes for existing Buffer usage. Closes #42 * docs: enhance README with examples, comparison table, and migration guides (#63) * test: add cargo-fuzz targets for decompression fuzzing (#71) * test: add edge case and error handling tests (#65) * test: expand Node.js zlib interop tests to brotli and zstd (#67) * ci: add hand-written JS files to workflow path filters (#56) * ci(codeql): suppress rust/access-invalid-pointer false positives (#57) * fix(dx): add ./streams subpath export to package.json (#59) * ci: add cargo-audit to dependency audit pipeline (#61) * feat(core): add unified auto-detect decompression API (#72) Add decompress() function that auto-detects compression format from magic bytes (zstd: 0xFD2FB528, gzip: 0x1F8B) and falls back to brotli heuristic detection. Add detectFormat() for format identification without decompression. Supported formats: zstd, gzip, brotli. Raw deflate cannot be auto-detected due to lack of magic bytes. Closes #16 * ci(dx): add @arethetypeswrong/cli type validation (#62) * perf(bench): add competitive benchmarks across all algorithms (#70) * perf(wasm): add WASM binary size reporting to CI (#66) * fix(exports): add missing ESM exports for new functions (#74) Add decompress, detectFormat, gzipDecompressWithCapacity, and deflateDecompressWithCapacity to index.mjs ESM export list. These functions were available via CJS but missing from ESM imports. Update ESM smoke test to cover all WithCapacity functions, decompress, and detectFormat. Closes #73 * feat(node): add Node.js stream.Transform compatibility layer (#76) Add `zflate/node` subpath export with stream.Transform factories for all compression algorithms (zstd, gzip, deflate, brotli), enabling use with Node.js stream.pipeline() and pipe-based workflows. Closes #43 * fix(brotli): use consistent error message for capacity exceeded (#78) Change brotli_decompress_with_capacity error message from "Destination buffer is too small" to "brotli decompress exceeded maximum size of {} bytes" to match the pattern used by all other decompress functions. Closes #77 * fix(docs): remove merge conflict marker and stale src/ references (#80) * feat(core): add async compression/decompression API (#82) Add async versions of all one-shot compression/decompression functions using napi-rs AsyncTask with the Task trait. This runs compression on the libuv thread pool without blocking the Node.js event loop. New functions: zstdCompressAsync, zstdDecompressAsync, gzipCompressAsync, gzipDecompressAsync, deflateCompressAsync, deflateDecompressAsync, brotliCompressAsync, brotliDecompressAsync. Closes #40 * feat(zstd): add dictionary compression API (#83) Add zstd dictionary training, compression, and decompression functions: - zstdTrainDictionary: train a dictionary from sample data - zstdCompressWithDict: compress with a pre-trained dictionary - zstdDecompressWithDict: decompress data compressed with a dictionary Enable the zdict_builder feature for the zstd crate to support dictionary training. Include Rust unit tests and Vitest integration tests verifying round-trip correctness and improved compression ratios for small, similar data. Closes #41 * docs: remove Cloudflare Workers from supported platforms (#91) * ci: include Node.js WASM loader files in CI artifact (#90) * test(wasm): add Node.js WASM compatibility test with native parity verification (#92) * refactor(core): introduce thiserror for structured error types (#93) Replace manual `Error::new(Status::..., ...)` calls with structured `ZflateError` enum variants across all compression modules. Error messages remain identical to preserve backward compatibility. Closes #51 * test(deno): add Deno WASM smoke test (#96) * test(bun): add Bun WASM smoke test (#95) * test(browser): add Playwright browser WASM test (#94) * ci(security): tighten cargo-deny bans and sources to deny mode (#108) * ci: pin dtolnay/rust-toolchain to SHA digest (#107) * fix(core): add decompression size limits to streaming contexts (#109) Add cumulative output size tracking to all streaming decompression contexts (ZstdDecompressContext, GzipDecompressContext, DeflateDecompressContext, BrotliDecompressContext). Each context now enforces MAX_DECOMPRESSED_SIZE (256 MB) across transform() calls, preventing decompression bomb attacks that could cause unbounded memory growth. Closes #97 * feat(core): narrow detectFormat return type to string literal union (#111) Use napi-rs string_enum to return CompressionFormat instead of plain string, generating a const enum with 'zstd' | 'gzip' | 'brotli' | 'unknown' values in TypeScript. Closes #103 * perf(core): optimize buffer allocation in compression and decompression (#110) * perf(core): optimize buffer allocation in compression and decompression Increase decompression read buffer from 4KB to 32KB in gzip, brotli, and brotli_stream modules to reduce syscall overhead during chunked reads. Pre-allocate compression output Vecs with input.len() capacity for gzip and deflate encoders (both sync and async) to avoid repeated reallocations during compression. Closes #100 * revert: keep BUFFER_SIZE at 4096 to avoid decompress regression The 32KB buffer caused 21 benchmark regressions in CodSpeed due to increased stack zeroing cost. Revert to 4KB. The Vec::with_capacity optimization for compression output is retained. * perf(core): replace Vec clone and split_off with std::mem::take in streaming contexts (#112) Use std::mem::take() instead of .get_ref().clone() + .get_mut().clear() in brotli streaming contexts, and instead of .split_off(0) in gzip/deflate streaming contexts. This avoids unnecessary memory allocation and copying by moving ownership of the buffer contents in-place. Closes #99 * fix(zstd): add compression level validation (#113) Validate that the compression level is within the valid zstd range (-131072 to 22) before passing it to the zstd library. This provides clear error messages instead of opaque library errors. Validation is added to: - zstd_compress (sync) - zstd_compress_async - ZstdCompressContext::new (streaming) - zstd_compress_with_dict Closes #81 * feat(zstd): add streaming dictionary compression/decompression (#114) Add ZstdCompressDictContext and ZstdDecompressDictContext classes that accept a pre-trained dictionary for streaming compression/decompression. Include Web Streams (createZstdCompressDictStream/createZstdDecompressDictStream) and Node.js Transform (createZstdCompressDictTransform/createZstdDecompressDictTransform) wrappers, type definitions, ESM exports, and streaming dict round-trip tests. Closes #104 * perf(core): switch flate2 backend from miniz_oxide to zlib-rs (#115) Replace the `rust_backend` (miniz_oxide) feature with `zlib-rs` for 2-3x faster gzip/deflate compression. zlib-rs is a pure-Rust implementation that works on all targets including WASM, is ISRG audited, and is on track to become flate2's default backend. Relax compression level size ordering test since zlib-rs may produce slightly different output sizes per level than miniz_oxide — round-trip correctness is what matters. Closes #98 * perf(zstd): enable multi-threaded compression for native targets (#116) Add `zstdmt` feature for non-WASM targets via conditional dependency. This enables zstd's internal thread pool for compression, improving performance by 20-80% for large inputs on multi-core systems. WASM targets are excluded because wasi-threads is experimental and the interaction with napi-rs worker-based threading is untested. Closes #102 * docs: comprehensive README update with missing APIs and SVG benchmark (#118) * fix(docs): correct benchmark environment label to Apple M2 / Node.js v22 (#119) * perf(wasm): add --converge flag to wasm-opt optimization (#122) Re-run optimization passes until no further gains are achieved. Increase timeout to 5 minutes to accommodate additional passes. Closes #121 * perf(core): use 4x input size for decompression output pre-allocation (#123) Compressed data typically decompresses to several times its original size. Pre-allocating with input.len() alone leads to frequent reallocations during decompression. Using input.len() * 4 as the initial capacity provides a better heuristic while the existing .min() caps continue to prevent excessive allocation. Closes #120 * ci: migrate release workflow to npm trusted publishing (OIDC) (#68) * ci: migrate release workflow to npm trusted publishing (OIDC) Remove NPM_TOKEN/NODE_AUTH_TOKEN secret dependency and rely on OIDC token exchange via id-token:write permission (already set). Trusted publishing must be configured on npmjs.com for each package before the next release. * fix(ci): restore pinned SHA for dtolnay/rust-toolchain Keep the pinned commit hash consistent with ci.yml and codspeed.yml for supply chain security. The rust-toolchain change was out of scope for the trusted publishing migration. * fix(gzip): support concatenated gzip streams in decompression (#131) Replace GzDecoder with MultiGzDecoder from flate2 to correctly handle concatenated gzip members in both one-shot and streaming decompression. * fix(streaming): add finish-guard state tracking for brotli and zstd compress contexts (#132) * feat(core): add missing async variants for one-shot APIs (#133) * feat(streaming): add auto-detect decompression streams (#145) * docs: add troubleshooting section to CONTRIBUTING.md (#146) * docs: add CODE_OF_CONDUCT.md (#149) * ci: apply least-privilege permissions to workflows (#147) * chore: improve package.json metadata and exports map (#148) * ci: clean up CI workflow configuration (#150) * feat(core): expose CRC32 utility function (#134) * feat(core): expose CRC32 utility function * test(crc32): use skipIf for conditional Node.js zlib interop test * ci(testing): add coverage thresholds to vitest config (#151) * ci: add aggregate gate job to CI workflow (#152) * feat(streaming): add configurable maxOutputSize to decompression contexts (#153) Add optional maxOutputSize parameter to all streaming decompression context constructors (GzipDecompressContext, DeflateDecompressContext, ZstdDecompressContext, BrotliDecompressContext, ZstdDecompressDictContext). When not specified, the default 256 MB limit is used. The parameter is also passed through all streaming factory functions (createXxxDecompressStream, createXxxDecompressTransform, and createDecompressStream/createDecompressTransform). Closes #129 * ci: add environment protection for npm publish (#154) * ci: add cargo-shear for unused dependency detection (#155) * ci: add WASM binary size tracking on pull requests (#156) * ci: add scripts/ to CI workflow path triggers (#159) * fix(wasm): add JS-side streaming adapters for browser WASM environment (#158) Replace native streaming contexts with JS-side adapters in the browser entry point to work around WebAssembly.Memory growth invalidating ArrayBuffer views. Adapters accumulate chunks and delegate to one-shot APIs, maintaining the same transform()/flush()/finish() API surface. - browser-streaming.js: adapter classes for all 10 streaming contexts - browser-entry.js: re-exports WASM APIs with adapter overrides - package.json: browser condition points to browser-entry.js - e2e: streaming tests enabled for browser WASM * feat(gzip): add header metadata support for compression and reading (#160) Closes #130 * chore(renovate): run daily and remove PR concurrent limit (#164) * chore(renovate): run daily and remove PR concurrent limit Closes #163 * chore(renovate): disable lock file maintenance * chore(deps): update codspeedhq/action digest to 1c8ae48 (#161) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dtolnay/rust-toolchain digest to 631a55b (#162) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update github/codeql-action digest to 3869755 (#165) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update taiki-e/install-action digest to b988c18 (#166) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(bench): exclude third-party benchmarks from CodSpeed tracking (#169) * fix(bench): exclude third-party benchmarks from CodSpeed tracking Split benchmark files into zflate-only (*.bench.ts) and comparison (*.compare.bench.ts) variants. CodSpeed now only tracks zflate's own benchmarks, eliminating false regressions from pako/fflate/node:zlib that are caused by GitHub Actions CPU heterogeneity affecting Callgrind instruction counting. Closes #167 * fix(bench): use deterministic random data in CodSpeed benchmarks Replace non-deterministic randomBytes() with a seeded LCG to ensure reproducible benchmark inputs across runs, reducing CodSpeed instruction count variance. * feat(bench): migrate from CodSpeed to Bencher.dev for regression detection (#170) Replace CodSpeed simulation mode (Callgrind instruction counting) with Bencher.dev (statistical wall-clock analysis). CodSpeed's instruction counting was fundamentally incompatible with this napi-rs project due to GitHub Actions CPU heterogeneity and glibc's CPU-adaptive malloc. Changes: - Add Bencher workflow with t-test threshold for regression detection - Add vitest-to-bmf.js conversion script for vitest bench output - Remove CodSpeed workflow and @codspeed/vitest-plugin dependency - Remove CodSpeed plugin from vitest config Closes #168 * feat(bench): switch to CodSpeed Walltime mode for accurate benchmarks (#172) Replace Bencher.dev with CodSpeed Walltime mode (Macro Runners) which runs benchmarks on dedicated bare-metal hardware, eliminating GitHub Actions CPU variance. CodSpeed also provides native vitest integration via @codspeed/vitest-plugin, removing the need for conversion scripts. Key change: `mode: walltime` instead of the previous `mode: simulation` which used Callgrind instruction counting. Closes #171 * refactor(bench): split benchmark files per algorithm for granular CodSpeed reporting (#174) Split index.bench.ts (mixed zstd+brotli) into separate zstd.bench.ts and brotli.bench.ts so CodSpeed PR reports only show benchmarks relevant to the changed algorithm. Extract shared test fixtures into bench-fixtures.ts and add realistic data benchmarks (JSON, text) to gzip and deflate for consistent coverage across all four algorithms. Closes #173 * perf(core): eliminate double-copy buffer pattern in decompression (#178) * perf(core): eliminate double-copy buffer pattern in decompression Replace the manual 4 KiB read loop (stack buffer → Vec copy) with read_to_end + Take, which writes directly into Vec spare capacity. This eliminates one full copy of every decompressed byte and uses adaptive buffer sizing instead of fixed 4 KiB chunks. Affected: gzip, deflate, brotli (sync + async, default + with-capacity) Closes #175 * fix(core): use saturating_mul for init capacity and update size limit tests Address CodeRabbit review feedback: - Replace len() * 4 with len().saturating_mul(4) across all call sites to prevent overflow on 32-bit targets (wasm32-wasip1-threads) - Replace old chunk-loop size limit tests with Take+read_to_end tests that exercise the actual decompression pattern (avoiding napi::Error symbol resolution issues in standalone test binaries) * chore(bench): add bench:ci script and Rust CodSpeed benchmarks (#179) * chore(bench): add bench:ci script and Rust CodSpeed benchmarks - Add bench:ci script that excludes comparison benchmarks - Simplify CodSpeed workflow to use bench:ci instead of explicit file list - Use glob pattern for bench file path triggers - Add crates/bench workspace member with Criterion benchmarks (gzip, deflate, brotli, zstd) using codspeed-criterion-compat - Add rust-benchmarks job to CodSpeed workflow (simulation mode) Closes #177 * fix(ci): add Cargo.toml, Cargo.lock, package.json to CodSpeed path triggers Ensures CodSpeed benchmarks run when workspace config or dependency lockfiles change, which could affect benchmark behavior. * feat(core): add LZ4 frame compression support (#180) * feat(core): add LZ4 frame compression support Add LZ4 as a fourth compression algorithm using lz4_flex (pure Rust): - Bulk API: lz4Compress, lz4Decompress, lz4DecompressWithCapacity - Async API: lz4CompressAsync, lz4DecompressAsync, lz4DecompressWithCapacityAsync - Streaming: Lz4CompressContext, Lz4DecompressContext - Web Streams: createLz4CompressStream, createLz4DecompressStream - Format detection: detectFormat returns "lz4" for LZ4 frame magic bytes - Auto-decompress: decompress/decompressAsync and createDecompressStream now handle LZ4 data automatically Closes #176 * fix(core): add Read import to LZ4 test modules Tests use read_to_end which requires the Read trait in scope. * refactor(core): validate capacity as non-negative integer (#183) * refactor(core): validate capacity as non-negative integer across all algorithms Extract shared validate_capacity() that rejects NaN, Infinity, negative, fractional values, and values exceeding usize::MAX. Applied to all 10 DecompressWithCapacity functions (sync + async × 5 algorithms). Closes #181 * test: add regression tests for fractional and oversized capacity * perf(lz4): implement incremental streaming for LZ4 compression (#184) * perf(lz4): implement incremental streaming for LZ4 compression Replace full-input buffering with cursor-based incremental output in Lz4CompressContext. The FrameEncoder now persists across transform() calls, emitting compressed blocks as they become available instead of buffering all input until finish(). Decompression remains buffered due to lz4_flex FrameDecoder requiring full input (documented in #182). Closes #182 * perf(lz4): implement incremental streaming for LZ4 compression Replace full-input buffering with FrameEncoder-based incremental output in Lz4CompressContext. Compressed blocks are emitted as internal buffers fill, and old bytes are drained to bound memory usage. Decompression remains buffered due to lz4_flex FrameDecoder requiring full input (documented in #182). Closes #182 * chore(deps): update taiki-e/install-action digest to 0620367 (#185) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update rust crate lz4_flex to 0.13 (#186) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore: rename package from zflate to comprs (#190) * chore: rename package from zflate to comprs Rename npm package, Rust crates, napi binary name, platform packages, and all references from "zflate" to "comprs". The name "comprs" conveys "compress" with a Rust flavor, is available on npm, and avoids the typosquat protection that blocks "zflate". Changes across 70 files: - npm: comprs, comprs-darwin-arm64, comprs-wasm32-wasi, etc. - Rust: comprs (core), comprs-bench - napi binaryName: comprs (→ comprs.*.node, comprs.*.wasm) - README, workflows, changesets, tests, benchmarks GitHub repository URL (derodero24/zflate) is preserved. Closes #189 * chore: update GitHub repo URLs to derodero24/comprs Update all repository references to match the planned GitHub repo rename from derodero24/zflate to derodero24/comprs. * chore(core): rename ZflateError to ComprsError and update bench vars (#192) Rename internal Rust error type `ZflateError` → `ComprsError` across all source files and update benchmark variable names `*_ZFLATE` → `*_COMPRS` for consistency with the package rename in #190. Closes #191 * fix: add missing LZ4 docs, browser exports, Node.js transforms, and publish files (#194) - Add LZ4 one-shot and streaming context exports to browser-entry.js - Add Lz4CompressContext/Lz4DecompressContext adapters to browser-streaming.js - Add createLz4CompressTransform/createLz4DecompressTransform to node.js - Add LZ4 type declarations to node.d.ts - Add 'lz4' case to createDecompressContext in node.js - Add browser-entry.js and browser-streaming.js to package.json files array - Document LZ4 API in README.md (one-shot, async, streaming, comparison) - Fix streaming import path in README.md (comprs → comprs/streams) - Update auto-detect descriptions to include LZ4 - Add "lz4" to package.json keywords and description - Add LZ4 changeset for upcoming release Closes #193 * fix: add missing LZ4 exports to ESM entry point and complete API docs (#196) - Add LZ4 one-shot APIs, Context classes, and streaming functions to index.mjs (ESM users could not access LZ4 at all) - Add createLz4CompressStream/createLz4DecompressStream to index.mjs - Add gzipCompressWithHeader and gzipReadHeader to browser-entry.js - Document gzipCompressWithHeader, gzipReadHeader in README - Document crc32 and version() utilities in README - Document createDecompressStream and createDecompressTransform in README Closes #195 * fix(docs): add missing lz4 to decompressAsync doc and clarify async section (#198) The decompressAsync Rust doc comment listed only zstd, gzip, brotli as supported formats, but the implementation also handles lz4. The README async section claimed "all one-shot functions" have async variants, which is inaccurate for utility functions like crc32, version, gzipReadHeader. Closes #197 * chore(release): version packages (#35) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * perf(ci): remove JS walltime benchmarks from CodSpeed regression detection (#201) JS walltime benchmarks on standard GitHub Actions runners produce 20-50% variance due to CPU heterogeneity (AMD EPYC vs Intel Xeon assigned randomly) and glibc's CPU-adaptive malloc. This caused CodSpeed Performance Analysis to fail on nearly every PR since #174, including documentation-only changes. Keep only Rust simulation benchmarks in CodSpeed, which use instruction counting via Callgrind and produce <1% variance. Also narrow the workflow path triggers to Rust-only files since JS changes no longer affect the CodSpeed pipeline. JS benchmarks remain available locally via `pnpm run bench`. Closes #200 * ci(release): use granular access token for initial npm publish The first publish requires an NPM_TOKEN since OIDC trusted publishing only works for packages that already exist on the registry. After the initial release, this will be switched back to OIDC. * chore: add .mcp.json to .gitignore (#205) Closes #203 * fix(core): use decompress_with_limit in async auto-detect decompression (#216) Replace manual chunk-read loops with double-copy in DecompressTask for gzip and brotli branches. Now uses crate::decompress_with_limit(), matching the pattern already used by the LZ4 branch and per-algorithm async tasks. Closes #207 * ci: add browser files to CI path triggers (#217) Closes #212 * fix(zstd): validate maxDictSize parameter in dictionary training (#219) * test: update ESM smoke test to cover all exports (#215) * Add LZ4 edge-case and streaming test coverage (#218) * test(lz4): add edge-case and streaming test coverage * test(lz4): fix empty buffer test to match actual lz4 behavior LZ4 returns empty buffer for empty input (like zstd and deflate), rather than throwing an error (like gzip and brotli). * fix(zstd): enforce maxOutputSize in browser dict decompression (#221) * fix(zstd): enforce maxOutputSize in browser dict decompression Add zstd_decompress_with_dict_with_capacity Rust function and use it in browser-streaming.js ZstdDecompressDictContext.flush() when maxOutputSize is provided. Closes #208 * chore: regenerate napi-rs binding files * refactor(core): consolidate MAX_DECOMPRESSED_SIZE constant (#222) Remove duplicated MAX_DECOMPRESSED_SIZE definitions from zstd, gzip, brotli_impl, lz4, and detect modules. Change the lib.rs definition to pub(crate) and reference it via crate::MAX_DECOMPRESSED_SIZE in all modules. * chore(release): version packages (#220) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * docs: add benchmark comparison results to README (#224) Closes #213 * feat(brotli): add dictionary compression and decompression support (#225) * docs: add brotli dictionary to README and update Node.js transforms (#228) * docs: readme visual and structural overhaul (#239) - Center-align hero section with title, tagline, and badges - Add Node.js version and TypeScript badges - Add Table of Contents for navigation - Wrap benchmark tables in collapsible <details> sections - Replace plain blockquote with GitHub Admonition (> [!TIP]) - Replace raw build target triples with human-readable table Closes #229 * docs: reposition and enhance comparison table (#240) - Move comparison table from bottom of README to right after "Why comprs?" - Split "Streaming" into "Web Streams API" and "Node.js Transform" rows - Replace ambiguous `✅*` with `⚠️ Experimental*` for node:zlib zstd - Replace footnote `†` with inline "Chunked†" for pako streaming - Update Table of Contents to reflect new section order - Center-align feature columns for better scannability Closes #236 * docs: improve quick start examples and add runtime-specific usage (#241) - Condense one-shot examples to single primary zstd block + multi-algorithm block - Replace impractical manual ReadableStream example with fetch-style streaming - Add Node.js Transform stream example to Quick Start - Add brief comment explaining async runs on libuv thread pool - Consolidate compression levels into compact format - Add Deno and Bun import examples Closes #235 * docs: add algorithm selection guide (#242) - Add "Choosing an Algorithm" section with use case → recommendation table - Add "Choosing an API mode" subsection for sync/async/streaming/dictionary - Update Table of Contents Closes #234 * docs: document browser WASM requirements (SharedArrayBuffer, COOP/COEP) (#243) - Add SharedArrayBuffer requirement warning with required HTTP headers - Remove misleading "or import directly from a CDN" claim - Link to MDN SharedArrayBuffer security requirements Closes #231 * docs: add SSR framework integration guide (#244) - Add "Framework Integration (SSR)" subsection under Browser Usage - Include Next.js serverExternalPackages and Vite SSR external config - Note automatic WASM fallback on client side Closes #232 * docs: add fflate migration guide (#245) - Add "From fflate" migration section with diff example - Note comprs advantages over fflate (zstd, lz4, brotli, dictionaries, Web Streams) Closes #237 * docs: add known limitations and performance notes (#246) - Add "Notes" section with three admonitions covering: - 256 MB default decompression limit with WithCapacity() workaround - Small payload WASM overhead advisory - Brotli decompression performance nuance vs node:zlib Closes #238 * docs: add all undocumented async and capacity API variants to readme (#247) - Add zstdDecompressWithDictWithCapacity to sync zstd dictionary table - Add 9 missing async variants: WithCapacityAsync for all algorithms, zstd dict async, zstdTrainDictionaryAsync, decompressAsync - Reorder async table to group by algorithm with capacity/dict variants Closes #233 * docs: add benchmark comparison SVG charts for cross-algorithm and brotli (#248) - Add cross-algorithm compression chart (zstd/lz4/gzip/brotli at 10KB) - Add brotli compression speedup chart (134.7x faster than node:zlib) - Embed both charts alongside existing gzip chart in Benchmarks section Closes #230 * docs: fold repetitive API tables into collapsible sections (#250) - Wrap Dictionary API (zstd + brotli) in <details> - Replace Async table header with pattern explanation + code example, fold full function list into <details> - Add Streaming import path + fold table into <details> - Keep Node.js Transform code example visible, fold table into <details> - One-shot core API tables remain visible as primary reference Closes #249 * chore(deps): update taiki-e/install-action digest to 7627fb4 (#251) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @biomejs/biome to v2.4.9 (#252) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update renovatebot/github-action action to v46.1.6 (#254) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency vite to v8.0.3 (#253) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update pnpm to v10.33.0 (#256) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update vitest monorepo to v4.1.1 (#255) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency typescript to v6 (#257) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * feat: add interactive playground with GitHub Pages deployment (#259) Add a browser-based playground for trying comprs compression live. Includes four algorithms (zstd, gzip, brotli, lz4), level slider, comparison table, file upload, and download. Deploys to GitHub Pages via new CI workflow. Uses a JS mock for local dev without a WASM build. Closes #258 * docs: improve playground visibility and add OGP/SEO meta tags (#261) Add prominent text link below badges in README header and playground as a bullet in "Why comprs?". Add Open Graph, Twitter Card, and canonical URL meta tags to playground/index.html for social sharing. Closes #260 * fix(playground): handle COI not established on first visit (#263) On first visit, the COI service worker has not yet reloaded the page, so crossOriginIsolated is false. Calling the WASM function immediately throws because SharedArrayBuffer is unavailable. This showed a misleading "unsupported browser" error even in modern browsers like Vivaldi, Brave, or Arc. - Check crossOriginIsolated before invoking WASM - Show "Enabling security features... reloading" while the service worker handles the reload - After 3 s, show a manual Reload button if auto-reload did not happen - Include the actual error message when WASM init itself fails - Add .loading-reload-btn styles to style.css Closes #262 * fix(playground): add Buffer polyfill for emnapi WASM compatibility (#265) The napi-rs WASM runtime (emnapi) requires globalThis.Buffer when creating memory views for function arguments/return values. Browsers do not define Buffer natively, causing zstdCompress and other WASM calls to throw immediately. Add the 'buffer' npm package (v6 browser polyfill) to playground dependencies and set globalThis.Buffer before any WASM calls. Closes #264 * fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init (#267) * fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init Static imports are hoisted and evaluated before any module body code runs. This caused globalThis.Buffer to be unset when comprs-wasm32-wasi evaluated __emnapiInstantiateNapiModuleSync (which calls emnapi_create_memory_view). Switch to a dynamic import so the Buffer polyfill assignment executes first. Closes #266 * fix(playground): allow Vite dev server to serve WASM from parent directory When comprs.wasi-browser.js is present in the repo root, the WASM file is fetched at runtime via /@fs/ from outside the playground directory. Add server.fs.allow so Vite does not block the request with 403. Only applied when a local WASM build is detected (hasLocalWasm=true). * fix(playground): fix comparison table label direction and badge layout (#271) - Rename bar chart column 'Ratio (smaller = better)' to 'Savings' and numeric column 'Ratio' to 'Saved' to accurately reflect that the metric is compression savings % (larger = better) - Change 'best ratio' badge to 'best' (shorter, clearer) - Add 'fastest' badge to the fastest algorithm row; remove the duplicate 'Fastest:' text from the footer note - Change badge column from fixed 72px to auto to prevent overflow Closes #270 * fix(playground): fix negative ratio text, CSS variable, download label, and file size alert (#273) - Show "X% larger" instead of "-X% smaller" when compressed output exceeds input - Fix reload button using undefined var(--accent); use var(--text)/var(--bg) instead - Add file extension to download button (e.g. "Download .zst") via #download-ext span - Replace alert() for oversized file with inline error message that auto-dismisses after 5s Closes #272 * chore(deps): update codecov/codecov-action digest to 75cd116 (#268) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update vitest monorepo to v4.1.2 (#269) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(playground): add FileReader error handler and textarea aria-label (#275) - Add reader.onerror handler to show inline error when file reading fails instead of silently doing nothing (e.g. on disk error or revoked access) - Add aria-label to the input textarea for screen reader accessibility Closes #274 * chore(deps): update dtolnay/rust-toolchain digest to 29eef33 (#276) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update github/codeql-action digest to c10b806 (#277) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update actions/configure-pages action to v6 (#283) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update actions/deploy-pages action to v5 (#284) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update actions/upload-pages-artifact action to v4 (#285) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update taiki-e/install-action digest to d858f81 (#278) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update rust-dependencies to v3.8.4 (#281) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.2 (#280) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @biomejs/biome to v2.4.10 (#279) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update codecov/codecov-action action to v6 (#286) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 (#287) * chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 Update @napi-rs/cli from v3.5.1 to v3.6.0 along with @emnapi/core and @emnapi/runtime from v1.9.1 to v1.9.2 to resolve emnapi version mismatch that caused WASM build failures. * fix(deps): align emnapi transitive dependency to v1.9.2 Update the emnapi package (transitive via @napi-rs/cli) from v1.9.1 to v1.9.2, matching @emnapi/core and @emnapi/runtime versions. * chore(deps): update renovatebot/github-action action to v46.1.7 (#292) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * chore(deps): update dependency @types/node to v24.12.2 (#291) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(config): update biome.json schema to v2.4.10 (#294) * chore(deps): update dependency @playwright/test to v1.59.1 (#293) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com> * fix(core): remove unused wildcard import in crc test module (#295) * fix(ci): update setup-deno and changesets/action to resolve Renovate lookup failures (#296) Pin to specific version tags instead of major version floating tags, which do not exist in these repositories. This allows Renovate to resolve digests for tracking updates. - denoland/setup-deno: v2 → v2.0.4 - changesets/action: v1 → v1.7.0 Closes #290 * chore(release): version packages to v0.4.0 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * refactor: rename npm package to @derodero24/comprs (#298) Rename all npm packages from `comprs-*` to `@derodero24/comprs-*` to bypass npm's typosquatting protection that blocked the `comprs` name. - Root package: comprs → @derodero24/comprs - 9 platform packages: comprs-{platform} → @derodero24/comprs-{platform} - Regenerated napi-rs loader (index.js, browser.js) with new names - Updated release workflow version checks - Updated README install/import examples - Updated playground npm link Closes #297 * fix(playground): update vite alias for renamed WASM package Update the vite resolve alias from 'comprs-wasm32-wasi' to '@derodero24/comprs-wasm32-wasi' to match the package rename in #298. * fix(ci): add publishConfig and permissions for scoped npm publish (#302) - Add publishConfig.access: "public" to root and all platform package.json files so napi prepublish can publish scoped packages - Add contents: write permission to Publish job for GitHub Release creation by napi prepublish - Add changeset for v0.4.1 patch release Closes #301 * chore(release): version packages to v0.4.1 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
derodero24
added a commit
that referenced
this pull request
Apr 4, 2026
* Set up napi-rs build pipeline (#9)
* chore(core): add Rust crate skeleton with napi-rs bindings
Minimal `zflate` crate with a single `version()` export to verify the
napi-rs build pipeline works end-to-end.
- crates/core/Cargo.toml: cdylib crate with napi-rs v3
- crates/core/build.rs: napi_build setup
- crates/core/src/lib.rs: version() function
* chore: add package.json with napi-rs configuration
Configure the npm package with 9 build targets (macOS, Linux, Windows,
WASM), dev dependencies for tooling (Biome, Vitest, TypeScript,
Changesets, CodSpeed), and build/test/lint scripts.
* chore: add build scripts and module entry points
- scripts/optimize-wasm.js: post-process WASM with wasm-opt
- index.mjs / index.d.mts: ESM entry points
- index.js / index.d.ts / browser.js: napi-rs generated files (force-tracked)
* Add CI workflow (#10)
* test: add initial test suite and ESM smoke test
- __test__/index.spec.ts: Vitest spec verifying version() export
- __test__/esm-import.mjs: ESM import smoke test for CI
* chore: update biome and tsconfig for test files
- biome.json: allow console in ESM smoke test
- tsconfig.json: format with Biome
* ci: add CI workflow
Full CI pipeline adapted from rapid-fuzzy:
- Lint (Biome, TypeScript, publint, generated file verification)
- Dependency audit (npm + cargo-deny)
- Rust lint (fmt + clippy) and test
- Cross-platform build (9 targets: macOS, Linux, Windows, WASM)
- Node.js test matrix (3 versions × 3 OSes)
- Coverage (Rust via cargo-llvm-cov + JS via Vitest, uploaded to Codecov)
* Add release and automation workflows (#11)
* ci: add release workflow with Cargo.toml version sync
Changesets-driven release flow:
- On develop push: create version PR, then release PR (develop → main)
- On main push: build all 9 targets and publish to npm with provenance
- scripts/sync-cargo-version.js: keep Cargo.toml version in sync with
package.json after changeset version bumps
* ci: add CodeQL security analysis
Analyze JavaScript/TypeScript and Rust code for security vulnerabilities.
Runs on PRs, develop pushes, and weekly schedule (Monday 6 AM UTC).
* ci: add CodSpeed benchmark workflow
Continuous performance tracking via CodSpeed. Runs JS benchmarks on
develop pushes and PRs when crate or bench files change.
Rust benchmarks will be added when the bench crate is introduced.
* ci: add self-hosted Renovate workflow
Automated dependency updates via self-hosted Renovate bot.
Runs weekly on Sunday 19:00 UTC (Monday 4:00 JST).
* test(bench): add initial benchmark file
Minimal benchmark to satisfy CodSpeed workflow. Real compression
benchmarks will be added alongside algorithm implementations.
* Implement zstd compression and decompression (#12)
* feat(zstd): implement zstd compression and decompression
Add zstdCompress() and zstdDecompress() functions powered by the zstd
C library via the zstd Rust crate.
- Compression levels 1-22 (default: 3)
- Auto-detection of frame content size for efficient decompression
- zstdDecompressWithCapacity() for data exceeding 256 MB default limit
- Rust unit tests for round-trip, empty data, large data, and levels
* test(zstd): add JS tests and benchmarks for zstd
- __test__/zstd.spec.ts: 11 tests covering round-trip, levels, errors
- __test__/esm-import.mjs: updated with zstd round-trip smoke test
- __test__/index.bench.ts: compress/decompress benchmarks at 150B/10KB/1MB
- index.mjs: export zstd functions
* chore: update napi-rs generated files and add changeset
- Regenerate index.js, index.d.ts, browser.js with zstd exports
- Add changeset for minor version bump
* ci: add WASI SDK setup for zstd WASM builds
The zstd crate compiles C code via zstd-sys, which requires a proper
WASI sysroot when targeting wasm32-wasip1-threads. Without it, clang
picks up system glibc headers and fails with missing
'bits/libc-header-start.h'.
Use bytecodealliance/actions/wasi-sdk/setup to install WASI SDK and
set CC, AR, CFLAGS, and WASI_SYSROOT environment variables for the
WASM build step in both CI and release workflows.
* Fix CodeRabbit review findings from #12 (#22)
* fix(zstd): validate capacity parameter in zstdDecompressWithCapacity
Reject negative, NaN, and Infinity values before casting f64 to usize.
Without validation, a negative f64 wraps to a very large usize, which
could cause memory exhaustion.
Closes #17
* fix(ci): use RUNNER_TEMP for WASI SDK installation
Replace /opt with $RUNNER_TEMP for WASI SDK extraction. /opt may not
be writable on all GitHub Actions runner configurations.
Also switch curl from -sL to -fsSL to fail on HTTP errors.
Closes #18
* docs: add SECURITY.md (#23)
* docs: add SECURITY.md
* docs: address review feedback on SECURITY.md
- Replace vague "extremely large size" with "massive size" for clarity
- Unify SLA day units by removing "business" qualifier from initial
assessment timeline for consistency
* docs: add CONTRIBUTING.md (#24)
* test(zstd): expand test coverage and benchmarks (#25)
* test(zstd): expand test coverage and benchmarks
Add comprehensive JS tests covering compression level boundaries (0,
negative, 22), decompressWithCapacity edge cases (insufficient, oversized,
invalid inputs), and round-trip verification with random data.
Expand benchmarks with random (incompressible) and realistic (JSON, text)
data variants alongside existing patterned data.
Update Rust doc comments to document negative compression levels and
level 0 behavior. Add Rust tests for boundary levels. Update changeset
to include all public API functions.
Closes #19
* chore: update napi-rs generated type definitions
Regenerate index.d.ts to reflect updated doc comments for
zstd_compress documenting negative levels and level 0 behavior.
* feat(zstd): add streaming compression/decompression API (#26)
Implement streaming zstd compression and decompression using the Web
Streams API (TransformStream). The Rust layer exposes ZstdCompressContext
and ZstdDecompressContext classes with transform/flush/finish methods
using zstd::stream::raw for stateful chunked processing.
The JS layer wraps these in TransformStream via createZstdCompressStream()
and createZstdDecompressStream(), available from both ESM and CJS entry
points. Streaming output is fully interoperable with one-shot APIs.
Add comprehensive tests covering chunked round-trips, various chunk
sizes, large data (1MB), random data, and one-shot/streaming interop.
Closes #5
* docs: add README and verify npm package readiness (#28)
* docs: add README and verify npm package readiness
Add comprehensive README with project description, installation guide,
quick-start examples (one-shot and streaming), full API reference,
supported algorithms table, platform compatibility matrix, and
benchmark results.
Verify npm package with publint (no errors). Package includes all
required files: index.js, index.mjs, index.d.ts, index.d.mts,
streams.js, streams.d.ts, browser.js, LICENSE, and README.md.
Closes #8
* docs(readme): address review feedback
- Add bun installation instructions
- Replace streaming example with self-contained runnable snippet
* fix(ci): resolve release workflow failures (#29)
* fix(ci): resolve release workflow failures
- Generate npm platform package directories for all 9 targets
- Add optionalDependencies to root package.json for platform binaries
- Fix changeset version command argument parsing in release workflow
by using a dedicated npm script instead of inline shell command
Closes #27
* fix(ci): remove optionalDependencies from committed package.json
The optionalDependencies for platform-specific packages are added
at publish time by the prepublishOnly hook (napi prepublish -t npm),
not in the committed source. Having them in package.json breaks
pnpm install --frozen-lockfile since the packages are not yet on npm.
* chore(release): version packages (#30)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(ci): regenerate napi bindings for v0.2.0 and fix version workflow (#34)
- Regenerate index.js with v0.2.0 version strings
- Update npm platform package versions to 0.2.0
- Add Rust toolchain to version job for napi build during version bump
- Add napi version, build, and format to changeset:version script
- Fix version test to read from package.json dynamically
* feat(gzip): add gzip and deflate compression support (#33)
Add gzip and raw deflate compression/decompression with one-shot and
streaming APIs. Includes interoperability tests with Node.js built-in
zlib module.
Closes #6
* feat(brotli): add brotli compression support (#32)
* feat(brotli): add brotli compression support
Add brotli compression/decompression with one-shot and streaming APIs.
Quality levels 0-11 (default: 6). Includes interoperability tests with
Node.js built-in zlib brotli module.
Closes #7
* test(brotli): add brotli compression benchmarks
Add brotli compress/decompress benchmarks matching the zstd benchmark
structure for patterned, random, and realistic data at various sizes.
* test(gzip): increase timeout for 1MB round-trip tests
Extend timeout from 5s to 30s for gzip and deflate 1MB data tests
to prevent CI failures on slower runners.
* test: increase timeout for all 1MB tests
Set 30s timeout on all 1MB data round-trip and streaming tests to
prevent failures on slower CI runners (Windows).
* docs(readme): update API reference for gzip and brotli (#36)
Add gzip, deflate, and brotli to the API reference tables. Update
supported algorithms table to reflect all three algorithms are now
available with both one-shot and streaming APIs.
* chore(deps): update taiki-e/install-action digest to c12d62a (#20)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(exports): add ./streams subpath export to package.json (#58)
Add a `./streams` subpath export so users can import streaming APIs
directly via `import { ... } from 'zflate/streams'`.
Closes #48
* fix(gzip): add compression level validation for gzip and deflate (#60)
Validate that the compression level parameter is within the valid range
(0-9) for gzip_compress, deflate_compress, GzipCompressContext, and
DeflateCompressContext. Return InvalidArg error if out of range, matching
the existing validation pattern in brotli_compress.
Closes #38
* fix(gzip): add decompression size limits to gzip and deflate (#64)
* fix(gzip): add decompression size limits to gzip and deflate
Add MAX_DECOMPRESSED_SIZE (256 MB) to gzip_decompress and
deflate_decompress to prevent zip bomb attacks. Both functions now use
chunked reading with size checking instead of unbounded read_to_end().
Add gzipDecompressWithCapacity and deflateDecompressWithCapacity for
explicit control over the maximum decompressed size, matching the
existing zstd and brotli patterns.
Closes #37
* chore(napi): regenerate bindings for new WithCapacity functions
* feat(api): accept Uint8Array input in all compression functions (#69)
Change all input parameters from Buffer to Either<Buffer, Uint8Array>
across all one-shot and streaming functions. This generates
Buffer | Uint8Array TypeScript types, enabling browser/Deno/Bun users
to pass Uint8Array directly without type errors.
Return types remain Buffer. No runtime behavior changes for existing
Buffer usage.
Closes #42
* docs: enhance README with examples, comparison table, and migration guides (#63)
* test: add cargo-fuzz targets for decompression fuzzing (#71)
* test: add edge case and error handling tests (#65)
* test: expand Node.js zlib interop tests to brotli and zstd (#67)
* ci: add hand-written JS files to workflow path filters (#56)
* ci(codeql): suppress rust/access-invalid-pointer false positives (#57)
* fix(dx): add ./streams subpath export to package.json (#59)
* ci: add cargo-audit to dependency audit pipeline (#61)
* feat(core): add unified auto-detect decompression API (#72)
Add decompress() function that auto-detects compression format from
magic bytes (zstd: 0xFD2FB528, gzip: 0x1F8B) and falls back to brotli
heuristic detection. Add detectFormat() for format identification
without decompression.
Supported formats: zstd, gzip, brotli. Raw deflate cannot be
auto-detected due to lack of magic bytes.
Closes #16
* ci(dx): add @arethetypeswrong/cli type validation (#62)
* perf(bench): add competitive benchmarks across all algorithms (#70)
* perf(wasm): add WASM binary size reporting to CI (#66)
* fix(exports): add missing ESM exports for new functions (#74)
Add decompress, detectFormat, gzipDecompressWithCapacity, and
deflateDecompressWithCapacity to index.mjs ESM export list. These
functions were available via CJS but missing from ESM imports.
Update ESM smoke test to cover all WithCapacity functions, decompress,
and detectFormat.
Closes #73
* feat(node): add Node.js stream.Transform compatibility layer (#76)
Add `zflate/node` subpath export with stream.Transform factories for all
compression algorithms (zstd, gzip, deflate, brotli), enabling use with
Node.js stream.pipeline() and pipe-based workflows.
Closes #43
* fix(brotli): use consistent error message for capacity exceeded (#78)
Change brotli_decompress_with_capacity error message from
"Destination buffer is too small" to "brotli decompress exceeded
maximum size of {} bytes" to match the pattern used by all other
decompress functions.
Closes #77
* fix(docs): remove merge conflict marker and stale src/ references (#80)
* feat(core): add async compression/decompression API (#82)
Add async versions of all one-shot compression/decompression functions
using napi-rs AsyncTask with the Task trait. This runs compression on
the libuv thread pool without blocking the Node.js event loop.
New functions: zstdCompressAsync, zstdDecompressAsync,
gzipCompressAsync, gzipDecompressAsync, deflateCompressAsync,
deflateDecompressAsync, brotliCompressAsync, brotliDecompressAsync.
Closes #40
* feat(zstd): add dictionary compression API (#83)
Add zstd dictionary training, compression, and decompression functions:
- zstdTrainDictionary: train a dictionary from sample data
- zstdCompressWithDict: compress with a pre-trained dictionary
- zstdDecompressWithDict: decompress data compressed with a dictionary
Enable the zdict_builder feature for the zstd crate to support
dictionary training. Include Rust unit tests and Vitest integration
tests verifying round-trip correctness and improved compression ratios
for small, similar data.
Closes #41
* docs: remove Cloudflare Workers from supported platforms (#91)
* ci: include Node.js WASM loader files in CI artifact (#90)
* test(wasm): add Node.js WASM compatibility test with native parity verification (#92)
* refactor(core): introduce thiserror for structured error types (#93)
Replace manual `Error::new(Status::..., ...)` calls with structured
`ZflateError` enum variants across all compression modules. Error
messages remain identical to preserve backward compatibility.
Closes #51
* test(deno): add Deno WASM smoke test (#96)
* test(bun): add Bun WASM smoke test (#95)
* test(browser): add Playwright browser WASM test (#94)
* ci(security): tighten cargo-deny bans and sources to deny mode (#108)
* ci: pin dtolnay/rust-toolchain to SHA digest (#107)
* fix(core): add decompression size limits to streaming contexts (#109)
Add cumulative output size tracking to all streaming decompression
contexts (ZstdDecompressContext, GzipDecompressContext,
DeflateDecompressContext, BrotliDecompressContext).
Each context now enforces MAX_DECOMPRESSED_SIZE (256 MB) across
transform() calls, preventing decompression bomb attacks that could
cause unbounded memory growth.
Closes #97
* feat(core): narrow detectFormat return type to string literal union (#111)
Use napi-rs string_enum to return CompressionFormat instead of plain
string, generating a const enum with 'zstd' | 'gzip' | 'brotli' |
'unknown' values in TypeScript.
Closes #103
* perf(core): optimize buffer allocation in compression and decompression (#110)
* perf(core): optimize buffer allocation in compression and decompression
Increase decompression read buffer from 4KB to 32KB in gzip, brotli,
and brotli_stream modules to reduce syscall overhead during chunked
reads. Pre-allocate compression output Vecs with input.len() capacity
for gzip and deflate encoders (both sync and async) to avoid repeated
reallocations during compression.
Closes #100
* revert: keep BUFFER_SIZE at 4096 to avoid decompress regression
The 32KB buffer caused 21 benchmark regressions in CodSpeed due to
increased stack zeroing cost. Revert to 4KB. The Vec::with_capacity
optimization for compression output is retained.
* perf(core): replace Vec clone and split_off with std::mem::take in streaming contexts (#112)
Use std::mem::take() instead of .get_ref().clone() + .get_mut().clear()
in brotli streaming contexts, and instead of .split_off(0) in gzip/deflate
streaming contexts. This avoids unnecessary memory allocation and copying
by moving ownership of the buffer contents in-place.
Closes #99
* fix(zstd): add compression level validation (#113)
Validate that the compression level is within the valid zstd range
(-131072 to 22) before passing it to the zstd library. This provides
clear error messages instead of opaque library errors.
Validation is added to:
- zstd_compress (sync)
- zstd_compress_async
- ZstdCompressContext::new (streaming)
- zstd_compress_with_dict
Closes #81
* feat(zstd): add streaming dictionary compression/decompression (#114)
Add ZstdCompressDictContext and ZstdDecompressDictContext classes that
accept a pre-trained dictionary for streaming compression/decompression.
Include Web Streams (createZstdCompressDictStream/createZstdDecompressDictStream)
and Node.js Transform (createZstdCompressDictTransform/createZstdDecompressDictTransform)
wrappers, type definitions, ESM exports, and streaming dict round-trip tests.
Closes #104
* perf(core): switch flate2 backend from miniz_oxide to zlib-rs (#115)
Replace the `rust_backend` (miniz_oxide) feature with `zlib-rs` for
2-3x faster gzip/deflate compression. zlib-rs is a pure-Rust
implementation that works on all targets including WASM, is ISRG
audited, and is on track to become flate2's default backend.
Relax compression level size ordering test since zlib-rs may produce
slightly different output sizes per level than miniz_oxide — round-trip
correctness is what matters.
Closes #98
* perf(zstd): enable multi-threaded compression for native targets (#116)
Add `zstdmt` feature for non-WASM targets via conditional dependency.
This enables zstd's internal thread pool for compression, improving
performance by 20-80% for large inputs on multi-core systems.
WASM targets are excluded because wasi-threads is experimental and
the interaction with napi-rs worker-based threading is untested.
Closes #102
* docs: comprehensive README update with missing APIs and SVG benchmark (#118)
* fix(docs): correct benchmark environment label to Apple M2 / Node.js v22 (#119)
* perf(wasm): add --converge flag to wasm-opt optimization (#122)
Re-run optimization passes until no further gains are achieved.
Increase timeout to 5 minutes to accommodate additional passes.
Closes #121
* perf(core): use 4x input size for decompression output pre-allocation (#123)
Compressed data typically decompresses to several times its original
size. Pre-allocating with input.len() alone leads to frequent
reallocations during decompression. Using input.len() * 4 as the
initial capacity provides a better heuristic while the existing
.min() caps continue to prevent excessive allocation.
Closes #120
* ci: migrate release workflow to npm trusted publishing (OIDC) (#68)
* ci: migrate release workflow to npm trusted publishing (OIDC)
Remove NPM_TOKEN/NODE_AUTH_TOKEN secret dependency and rely on
OIDC token exchange via id-token:write permission (already set).
Trusted publishing must be configured on npmjs.com for each package
before the next release.
* fix(ci): restore pinned SHA for dtolnay/rust-toolchain
Keep the pinned commit hash consistent with ci.yml and codspeed.yml
for supply chain security. The rust-toolchain change was out of scope
for the trusted publishing migration.
* fix(gzip): support concatenated gzip streams in decompression (#131)
Replace GzDecoder with MultiGzDecoder from flate2 to correctly handle
concatenated gzip members in both one-shot and streaming decompression.
* fix(streaming): add finish-guard state tracking for brotli and zstd compress contexts (#132)
* feat(core): add missing async variants for one-shot APIs (#133)
* feat(streaming): add auto-detect decompression streams (#145)
* docs: add troubleshooting section to CONTRIBUTING.md (#146)
* docs: add CODE_OF_CONDUCT.md (#149)
* ci: apply least-privilege permissions to workflows (#147)
* chore: improve package.json metadata and exports map (#148)
* ci: clean up CI workflow configuration (#150)
* feat(core): expose CRC32 utility function (#134)
* feat(core): expose CRC32 utility function
* test(crc32): use skipIf for conditional Node.js zlib interop test
* ci(testing): add coverage thresholds to vitest config (#151)
* ci: add aggregate gate job to CI workflow (#152)
* feat(streaming): add configurable maxOutputSize to decompression contexts (#153)
Add optional maxOutputSize parameter to all streaming decompression
context constructors (GzipDecompressContext, DeflateDecompressContext,
ZstdDecompressContext, BrotliDecompressContext, ZstdDecompressDictContext).
When not specified, the default 256 MB limit is used.
The parameter is also passed through all streaming factory functions
(createXxxDecompressStream, createXxxDecompressTransform, and
createDecompressStream/createDecompressTransform).
Closes #129
* ci: add environment protection for npm publish (#154)
* ci: add cargo-shear for unused dependency detection (#155)
* ci: add WASM binary size tracking on pull requests (#156)
* ci: add scripts/ to CI workflow path triggers (#159)
* fix(wasm): add JS-side streaming adapters for browser WASM environment (#158)
Replace native streaming contexts with JS-side adapters in the browser
entry point to work around WebAssembly.Memory growth invalidating
ArrayBuffer views. Adapters accumulate chunks and delegate to one-shot
APIs, maintaining the same transform()/flush()/finish() API surface.
- browser-streaming.js: adapter classes for all 10 streaming contexts
- browser-entry.js: re-exports WASM APIs with adapter overrides
- package.json: browser condition points to browser-entry.js
- e2e: streaming tests enabled for browser WASM
* feat(gzip): add header metadata support for compression and reading (#160)
Closes #130
* chore(renovate): run daily and remove PR concurrent limit (#164)
* chore(renovate): run daily and remove PR concurrent limit
Closes #163
* chore(renovate): disable lock file maintenance
* chore(deps): update codspeedhq/action digest to 1c8ae48 (#161)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dtolnay/rust-toolchain digest to 631a55b (#162)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to 3869755 (#165)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to b988c18 (#166)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(bench): exclude third-party benchmarks from CodSpeed tracking (#169)
* fix(bench): exclude third-party benchmarks from CodSpeed tracking
Split benchmark files into zflate-only (*.bench.ts) and comparison
(*.compare.bench.ts) variants. CodSpeed now only tracks zflate's own
benchmarks, eliminating false regressions from pako/fflate/node:zlib
that are caused by GitHub Actions CPU heterogeneity affecting Callgrind
instruction counting.
Closes #167
* fix(bench): use deterministic random data in CodSpeed benchmarks
Replace non-deterministic randomBytes() with a seeded LCG to ensure
reproducible benchmark inputs across runs, reducing CodSpeed instruction
count variance.
* feat(bench): migrate from CodSpeed to Bencher.dev for regression detection (#170)
Replace CodSpeed simulation mode (Callgrind instruction counting) with
Bencher.dev (statistical wall-clock analysis). CodSpeed's instruction
counting was fundamentally incompatible with this napi-rs project due to
GitHub Actions CPU heterogeneity and glibc's CPU-adaptive malloc.
Changes:
- Add Bencher workflow with t-test threshold for regression detection
- Add vitest-to-bmf.js conversion script for vitest bench output
- Remove CodSpeed workflow and @codspeed/vitest-plugin dependency
- Remove CodSpeed plugin from vitest config
Closes #168
* feat(bench): switch to CodSpeed Walltime mode for accurate benchmarks (#172)
Replace Bencher.dev with CodSpeed Walltime mode (Macro Runners) which
runs benchmarks on dedicated bare-metal hardware, eliminating GitHub
Actions CPU variance. CodSpeed also provides native vitest integration
via @codspeed/vitest-plugin, removing the need for conversion scripts.
Key change: `mode: walltime` instead of the previous `mode: simulation`
which used Callgrind instruction counting.
Closes #171
* refactor(bench): split benchmark files per algorithm for granular CodSpeed reporting (#174)
Split index.bench.ts (mixed zstd+brotli) into separate zstd.bench.ts
and brotli.bench.ts so CodSpeed PR reports only show benchmarks relevant
to the changed algorithm. Extract shared test fixtures into
bench-fixtures.ts and add realistic data benchmarks (JSON, text) to
gzip and deflate for consistent coverage across all four algorithms.
Closes #173
* perf(core): eliminate double-copy buffer pattern in decompression (#178)
* perf(core): eliminate double-copy buffer pattern in decompression
Replace the manual 4 KiB read loop (stack buffer → Vec copy) with
read_to_end + Take, which writes directly into Vec spare capacity.
This eliminates one full copy of every decompressed byte and uses
adaptive buffer sizing instead of fixed 4 KiB chunks.
Affected: gzip, deflate, brotli (sync + async, default + with-capacity)
Closes #175
* fix(core): use saturating_mul for init capacity and update size limit tests
Address CodeRabbit review feedback:
- Replace len() * 4 with len().saturating_mul(4) across all call sites
to prevent overflow on 32-bit targets (wasm32-wasip1-threads)
- Replace old chunk-loop size limit tests with Take+read_to_end tests
that exercise the actual decompression pattern (avoiding napi::Error
symbol resolution issues in standalone test binaries)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks (#179)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks
- Add bench:ci script that excludes comparison benchmarks
- Simplify CodSpeed workflow to use bench:ci instead of explicit file list
- Use glob pattern for bench file path triggers
- Add crates/bench workspace member with Criterion benchmarks
(gzip, deflate, brotli, zstd) using codspeed-criterion-compat
- Add rust-benchmarks job to CodSpeed workflow (simulation mode)
Closes #177
* fix(ci): add Cargo.toml, Cargo.lock, package.json to CodSpeed path triggers
Ensures CodSpeed benchmarks run when workspace config or dependency
lockfiles change, which could affect benchmark behavior.
* feat(core): add LZ4 frame compression support (#180)
* feat(core): add LZ4 frame compression support
Add LZ4 as a fourth compression algorithm using lz4_flex (pure Rust):
- Bulk API: lz4Compress, lz4Decompress, lz4DecompressWithCapacity
- Async API: lz4CompressAsync, lz4DecompressAsync, lz4DecompressWithCapacityAsync
- Streaming: Lz4CompressContext, Lz4DecompressContext
- Web Streams: createLz4CompressStream, createLz4DecompressStream
- Format detection: detectFormat returns "lz4" for LZ4 frame magic bytes
- Auto-decompress: decompress/decompressAsync and createDecompressStream
now handle LZ4 data automatically
Closes #176
* fix(core): add Read import to LZ4 test modules
Tests use read_to_end which requires the Read trait in scope.
* refactor(core): validate capacity as non-negative integer (#183)
* refactor(core): validate capacity as non-negative integer across all algorithms
Extract shared validate_capacity() that rejects NaN, Infinity, negative,
fractional values, and values exceeding usize::MAX. Applied to all 10
DecompressWithCapacity functions (sync + async × 5 algorithms).
Closes #181
* test: add regression tests for fractional and oversized capacity
* perf(lz4): implement incremental streaming for LZ4 compression (#184)
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with cursor-based incremental output in
Lz4CompressContext. The FrameEncoder now persists across transform()
calls, emitting compressed blocks as they become available instead of
buffering all input until finish().
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with FrameEncoder-based incremental output
in Lz4CompressContext. Compressed blocks are emitted as internal buffers
fill, and old bytes are drained to bound memory usage.
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* chore(deps): update taiki-e/install-action digest to 0620367 (#185)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust crate lz4_flex to 0.13 (#186)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore: rename package from zflate to comprs (#190)
* chore: rename package from zflate to comprs
Rename npm package, Rust crates, napi binary name, platform packages,
and all references from "zflate" to "comprs". The name "comprs" conveys
"compress" with a Rust flavor, is available on npm, and avoids the
typosquat protection that blocks "zflate".
Changes across 70 files:
- npm: comprs, comprs-darwin-arm64, comprs-wasm32-wasi, etc.
- Rust: comprs (core), comprs-bench
- napi binaryName: comprs (→ comprs.*.node, comprs.*.wasm)
- README, workflows, changesets, tests, benchmarks
GitHub repository URL (derodero24/zflate) is preserved.
Closes #189
* chore: update GitHub repo URLs to derodero24/comprs
Update all repository references to match the planned GitHub repo
rename from derodero24/zflate to derodero24/comprs.
* chore(core): rename ZflateError to ComprsError and update bench vars (#192)
Rename internal Rust error type `ZflateError` → `ComprsError` across
all source files and update benchmark variable names `*_ZFLATE` →
`*_COMPRS` for consistency with the package rename in #190.
Closes #191
* fix: add missing LZ4 docs, browser exports, Node.js transforms, and publish files (#194)
- Add LZ4 one-shot and streaming context exports to browser-entry.js
- Add Lz4CompressContext/Lz4DecompressContext adapters to browser-streaming.js
- Add createLz4CompressTransform/createLz4DecompressTransform to node.js
- Add LZ4 type declarations to node.d.ts
- Add 'lz4' case to createDecompressContext in node.js
- Add browser-entry.js and browser-streaming.js to package.json files array
- Document LZ4 API in README.md (one-shot, async, streaming, comparison)
- Fix streaming import path in README.md (comprs → comprs/streams)
- Update auto-detect descriptions to include LZ4
- Add "lz4" to package.json keywords and description
- Add LZ4 changeset for upcoming release
Closes #193
* fix: add missing LZ4 exports to ESM entry point and complete API docs (#196)
- Add LZ4 one-shot APIs, Context classes, and streaming functions to
index.mjs (ESM users could not access LZ4 at all)
- Add createLz4CompressStream/createLz4DecompressStream to index.mjs
- Add gzipCompressWithHeader and gzipReadHeader to browser-entry.js
- Document gzipCompressWithHeader, gzipReadHeader in README
- Document crc32 and version() utilities in README
- Document createDecompressStream and createDecompressTransform in README
Closes #195
* fix(docs): add missing lz4 to decompressAsync doc and clarify async section (#198)
The decompressAsync Rust doc comment listed only zstd, gzip, brotli as
supported formats, but the implementation also handles lz4. The README
async section claimed "all one-shot functions" have async variants, which
is inaccurate for utility functions like crc32, version, gzipReadHeader.
Closes #197
* chore(release): version packages (#35)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* perf(ci): remove JS walltime benchmarks from CodSpeed regression detection (#201)
JS walltime benchmarks on standard GitHub Actions runners produce 20-50%
variance due to CPU heterogeneity (AMD EPYC vs Intel Xeon assigned randomly)
and glibc's CPU-adaptive malloc. This caused CodSpeed Performance Analysis to
fail on nearly every PR since #174, including documentation-only changes.
Keep only Rust simulation benchmarks in CodSpeed, which use instruction
counting via Callgrind and produce <1% variance. Also narrow the workflow
path triggers to Rust-only files since JS changes no longer affect the
CodSpeed pipeline.
JS benchmarks remain available locally via `pnpm run bench`.
Closes #200
* ci(release): use granular access token for initial npm publish
The first publish requires an NPM_TOKEN since OIDC trusted publishing
only works for packages that already exist on the registry. After the
initial release, this will be switched back to OIDC.
* chore: add .mcp.json to .gitignore (#205)
Closes #203
* fix(core): use decompress_with_limit in async auto-detect decompression (#216)
Replace manual chunk-read loops with double-copy in DecompressTask for
gzip and brotli branches. Now uses crate::decompress_with_limit(),
matching the pattern already used by the LZ4 branch and per-algorithm
async tasks.
Closes #207
* ci: add browser files to CI path triggers (#217)
Closes #212
* fix(zstd): validate maxDictSize parameter in dictionary training (#219)
* test: update ESM smoke test to cover all exports (#215)
* Add LZ4 edge-case and streaming test coverage (#218)
* test(lz4): add edge-case and streaming test coverage
* test(lz4): fix empty buffer test to match actual lz4 behavior
LZ4 returns empty buffer for empty input (like zstd and deflate),
rather than throwing an error (like gzip and brotli).
* fix(zstd): enforce maxOutputSize in browser dict decompression (#221)
* fix(zstd): enforce maxOutputSize in browser dict decompression
Add zstd_decompress_with_dict_with_capacity Rust function and use it
in browser-streaming.js ZstdDecompressDictContext.flush() when
maxOutputSize is provided.
Closes #208
* chore: regenerate napi-rs binding files
* refactor(core): consolidate MAX_DECOMPRESSED_SIZE constant (#222)
Remove duplicated MAX_DECOMPRESSED_SIZE definitions from zstd, gzip,
brotli_impl, lz4, and detect modules. Change the lib.rs definition to
pub(crate) and reference it via crate::MAX_DECOMPRESSED_SIZE in all
modules.
* chore(release): version packages (#220)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* docs: add benchmark comparison results to README (#224)
Closes #213
* feat(brotli): add dictionary compression and decompression support (#225)
* docs: add brotli dictionary to README and update Node.js transforms (#228)
* docs: readme visual and structural overhaul (#239)
- Center-align hero section with title, tagline, and badges
- Add Node.js version and TypeScript badges
- Add Table of Contents for navigation
- Wrap benchmark tables in collapsible <details> sections
- Replace plain blockquote with GitHub Admonition (> [!TIP])
- Replace raw build target triples with human-readable table
Closes #229
* docs: reposition and enhance comparison table (#240)
- Move comparison table from bottom of README to right after "Why comprs?"
- Split "Streaming" into "Web Streams API" and "Node.js Transform" rows
- Replace ambiguous `✅*` with `⚠️ Experimental*` for node:zlib zstd
- Replace footnote `†` with inline "Chunked†" for pako streaming
- Update Table of Contents to reflect new section order
- Center-align feature columns for better scannability
Closes #236
* docs: improve quick start examples and add runtime-specific usage (#241)
- Condense one-shot examples to single primary zstd block + multi-algorithm block
- Replace impractical manual ReadableStream example with fetch-style streaming
- Add Node.js Transform stream example to Quick Start
- Add brief comment explaining async runs on libuv thread pool
- Consolidate compression levels into compact format
- Add Deno and Bun import examples
Closes #235
* docs: add algorithm selection guide (#242)
- Add "Choosing an Algorithm" section with use case → recommendation table
- Add "Choosing an API mode" subsection for sync/async/streaming/dictionary
- Update Table of Contents
Closes #234
* docs: document browser WASM requirements (SharedArrayBuffer, COOP/COEP) (#243)
- Add SharedArrayBuffer requirement warning with required HTTP headers
- Remove misleading "or import directly from a CDN" claim
- Link to MDN SharedArrayBuffer security requirements
Closes #231
* docs: add SSR framework integration guide (#244)
- Add "Framework Integration (SSR)" subsection under Browser Usage
- Include Next.js serverExternalPackages and Vite SSR external config
- Note automatic WASM fallback on client side
Closes #232
* docs: add fflate migration guide (#245)
- Add "From fflate" migration section with diff example
- Note comprs advantages over fflate (zstd, lz4, brotli, dictionaries, Web Streams)
Closes #237
* docs: add known limitations and performance notes (#246)
- Add "Notes" section with three admonitions covering:
- 256 MB default decompression limit with WithCapacity() workaround
- Small payload WASM overhead advisory
- Brotli decompression performance nuance vs node:zlib
Closes #238
* docs: add all undocumented async and capacity API variants to readme (#247)
- Add zstdDecompressWithDictWithCapacity to sync zstd dictionary table
- Add 9 missing async variants: WithCapacityAsync for all algorithms,
zstd dict async, zstdTrainDictionaryAsync, decompressAsync
- Reorder async table to group by algorithm with capacity/dict variants
Closes #233
* docs: add benchmark comparison SVG charts for cross-algorithm and brotli (#248)
- Add cross-algorithm compression chart (zstd/lz4/gzip/brotli at 10KB)
- Add brotli compression speedup chart (134.7x faster than node:zlib)
- Embed both charts alongside existing gzip chart in Benchmarks section
Closes #230
* docs: fold repetitive API tables into collapsible sections (#250)
- Wrap Dictionary API (zstd + brotli) in <details>
- Replace Async table header with pattern explanation + code example,
fold full function list into <details>
- Add Streaming import path + fold table into <details>
- Keep Node.js Transform code example visible, fold table into <details>
- One-shot core API tables remain visible as primary reference
Closes #249
* chore(deps): update taiki-e/install-action digest to 7627fb4 (#251)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.9 (#252)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update renovatebot/github-action action to v46.1.6 (#254)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency vite to v8.0.3 (#253)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update pnpm to v10.33.0 (#256)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.1 (#255)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency typescript to v6 (#257)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* feat: add interactive playground with GitHub Pages deployment (#259)
Add a browser-based playground for trying comprs compression live.
Includes four algorithms (zstd, gzip, brotli, lz4), level slider,
comparison table, file upload, and download. Deploys to GitHub Pages
via new CI workflow. Uses a JS mock for local dev without a WASM build.
Closes #258
* docs: improve playground visibility and add OGP/SEO meta tags (#261)
Add prominent text link below badges in README header and playground
as a bullet in "Why comprs?". Add Open Graph, Twitter Card, and
canonical URL meta tags to playground/index.html for social sharing.
Closes #260
* fix(playground): handle COI not established on first visit (#263)
On first visit, the COI service worker has not yet reloaded the page,
so crossOriginIsolated is false. Calling the WASM function immediately
throws because SharedArrayBuffer is unavailable. This showed a
misleading "unsupported browser" error even in modern browsers like
Vivaldi, Brave, or Arc.
- Check crossOriginIsolated before invoking WASM
- Show "Enabling security features... reloading" while the service
worker handles the reload
- After 3 s, show a manual Reload button if auto-reload did not happen
- Include the actual error message when WASM init itself fails
- Add .loading-reload-btn styles to style.css
Closes #262
* fix(playground): add Buffer polyfill for emnapi WASM compatibility (#265)
The napi-rs WASM runtime (emnapi) requires globalThis.Buffer when
creating memory views for function arguments/return values. Browsers
do not define Buffer natively, causing zstdCompress and other WASM
calls to throw immediately.
Add the 'buffer' npm package (v6 browser polyfill) to playground
dependencies and set globalThis.Buffer before any WASM calls.
Closes #264
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init (#267)
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init
Static imports are hoisted and evaluated before any module body code runs.
This caused globalThis.Buffer to be unset when comprs-wasm32-wasi evaluated
__emnapiInstantiateNapiModuleSync (which calls emnapi_create_memory_view).
Switch to a dynamic import so the Buffer polyfill assignment executes first.
Closes #266
* fix(playground): allow Vite dev server to serve WASM from parent directory
When comprs.wasi-browser.js is present in the repo root, the WASM file
is fetched at runtime via /@fs/ from outside the playground directory.
Add server.fs.allow so Vite does not block the request with 403.
Only applied when a local WASM build is detected (hasLocalWasm=true).
* fix(playground): fix comparison table label direction and badge layout (#271)
- Rename bar chart column 'Ratio (smaller = better)' to 'Savings'
and numeric column 'Ratio' to 'Saved' to accurately reflect that
the metric is compression savings % (larger = better)
- Change 'best ratio' badge to 'best' (shorter, clearer)
- Add 'fastest' badge to the fastest algorithm row; remove the
duplicate 'Fastest:' text from the footer note
- Change badge column from fixed 72px to auto to prevent overflow
Closes #270
* fix(playground): fix negative ratio text, CSS variable, download label, and file size alert (#273)
- Show "X% larger" instead of "-X% smaller" when compressed output exceeds input
- Fix reload button using undefined var(--accent); use var(--text)/var(--bg) instead
- Add file extension to download button (e.g. "Download .zst") via #download-ext span
- Replace alert() for oversized file with inline error message that auto-dismisses after 5s
Closes #272
* chore(deps): update codecov/codecov-action digest to 75cd116 (#268)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.2 (#269)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(playground): add FileReader error handler and textarea aria-label (#275)
- Add reader.onerror handler to show inline error when file reading fails
instead of silently doing nothing (e.g. on disk error or revoked access)
- Add aria-label to the input textarea for screen reader accessibility
Closes #274
* chore(deps): update dtolnay/rust-toolchain digest to 29eef33 (#276)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to c10b806 (#277)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/configure-pages action to v6 (#283)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/deploy-pages action to v5 (#284)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/upload-pages-artifact action to v4 (#285)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to d858f81 (#278)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust-dependencies to v3.8.4 (#281)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.2 (#280)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.10 (#279)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update codecov/codecov-action action to v6 (#286)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 (#287)
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2
Update @napi-rs/cli from v3.5.1 to v3.6.0 along with @emnapi/core
and @emnapi/runtime from v1.9.1 to v1.9.2 to resolve emnapi version
mismatch that caused WASM build failures.
* fix(deps): align emnapi transitive dependency to v1.9.2
Update the emnapi package (transitive via @napi-rs/cli) from v1.9.1
to v1.9.2, matching @emnapi/core and @emnapi/runtime versions.
* chore(deps): update renovatebot/github-action action to v46.1.7 (#292)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @types/node to v24.12.2 (#291)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(config): update biome.json schema to v2.4.10 (#294)
* chore(deps): update dependency @playwright/test to v1.59.1 (#293)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(core): remove unused wildcard import in crc test module (#295)
* fix(ci): update setup-deno and changesets/action to resolve Renovate lookup failures (#296)
Pin to specific version tags instead of major version floating tags,
which do not exist in these repositories. This allows Renovate to
resolve digests for tracking updates.
- denoland/setup-deno: v2 → v2.0.4
- changesets/action: v1 → v1.7.0
Closes #290
* chore(release): version packages to v0.4.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* refactor: rename npm package to @derodero24/comprs (#298)
Rename all npm packages from `comprs-*` to `@derodero24/comprs-*` to
bypass npm's typosquatting protection that blocked the `comprs` name.
- Root package: comprs → @derodero24/comprs
- 9 platform packages: comprs-{platform} → @derodero24/comprs-{platform}
- Regenerated napi-rs loader (index.js, browser.js) with new names
- Updated release workflow version checks
- Updated README install/import examples
- Updated playground npm link
Closes #297
* fix(playground): update vite alias for renamed WASM package
Update the vite resolve alias from 'comprs-wasm32-wasi' to
'@derodero24/comprs-wasm32-wasi' to match the package rename in #298.
* fix(ci): add publishConfig and permissions for scoped npm publish (#302)
- Add publishConfig.access: "public" to root and all platform
package.json files so napi prepublish can publish scoped packages
- Add contents: write permission to Publish job for GitHub Release
creation by napi prepublish
- Add changeset for v0.4.1 patch release
Closes #301
* chore(release): version packages to v0.4.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: update stale package references after rename to @derodero24/comprs (#307)
Update remaining references from old unscoped names to scoped names:
- browser-streaming.js: comprs-wasm32-wasi → @derodero24/comprs-wasm32-wasi
- playground/main.js: import('comprs') → import('@derodero24/comprs')
- playground/package.json: comprs → @derodero24/comprs dependency
- playground/vite.config.js: Vite alias keys updated
This fixes the GitHub Pages deployment failure caused by Vite being
unable to resolve the old unscoped import in browser-streaming.js.
Closes #306
* fix(playground): regenerate lockfile after package rename
The playground's pnpm-lock.yaml was not updated when the dependency
was renamed from comprs to @derodero24/comprs, causing CI to fail
with frozen-lockfile mismatch.
* docs: fix README package references, streaming note, and WASM size (#311)
* fix(docs): update remaining old package name references in README
- Deno import: npm:comprs → npm:@derodero24/comprs
- Subpath imports: comprs/streams → @derodero24/comprs/streams,
comprs/node → @derodero24/comprs/node
- SSR config: Next.js and Vite external package names
Closes #308
* docs: clarify browser streaming memory behavior
Add note explaining that browser WASM streaming buffers all data in
memory (unlike Node.js which streams incrementally). Also qualify the
"bounded memory" claim in the feature list to specify Node.js.
Closes #309
* docs: add concrete WASM binary size to README
Show approximate optimized WASM size (~2.0 MB raw, ~800 KB gzipped)
directly in README so users can evaluate without navigating to CI.
Closes #310
* fix: update npm badges and sub-package descriptions (#317)
* fix(docs): update npm badges to use scoped package name
The npm version and downloads badges were pointing to the old unscoped
package name "comprs" instead of the scoped "@derodero24/comprs".
Closes #312
* fix: update platform sub-package descriptions to include lz4
All 9 platform sub-packages under npm/ had descriptions listing only
"zstd, gzip, and brotli" but were missing "lz4" which was added to
the root package. Updated to match: "zstd, gzip, brotli, and lz4".
Closes #315
* fix: add missing ESM export and test coverage (#318)
* fix: add missing zstdDecompressWithDictWithCapacity to ESM and browser exports
Add zstdDecompressWithDictWithCapacity to index.mjs destructured exports,
browser-entry.js one-shot re-exports, and ESM smoke test assertions.
Closes #313
* test: add coverage for WithDictWithCapacity functions and Node.js Transform streams
Add tests for zstdDecompressWithDictWithCapacity, brotliDecompressWithDictWithCapacity,
brotliDecompressWithDictWithCapacityAsync, LZ4 Node.js Transform round-trip,
zstd dict Node.js Transform round-trip, and brotli dict Node.js Transform round-trip.
Closes #314
* feat(wasm): replace WASI WASM with wasm-bindgen for browser compatibility (#320)
* refactor(core): extract pure Rust logic into core-lib crate
Split crates/core into two crates:
- crates/core-lib: pure Rust compression logic (no FFI dependencies)
- crates/core: napi-rs thin wrappers delegating to core-lib
This prepares for adding a wasm-bindgen crate that shares the same
core logic, enabling browser usage without SharedArrayBuffer.
Part of #319
* feat(wasm): add wasm-bindgen browser binding crate
Create crates/wasm with wasm-bindgen wrappers over comprs-core,
targeting wasm32-unknown-unknown (no SharedArrayBuffer required).
Includes one-shot and streaming APIs for all algorithms:
zstd, gzip, deflate, brotli, lz4, auto-detect, and crc32.
Part of #319
* feat(wasm): integrate wasm-bindgen build into package distribution
- Add build:wasm-bindgen script using wasm-pack
- Update browser.js to use wasm-bindgen output
- Update browser-streaming.js imports
- Add wasm-bindgen files to npm package
- Update CI with wasm-bindgen build job
- Update release workflow for wasm-bindgen artifacts
- Update playground configuration
- Remove SharedArrayBuffer requirement from README
Part of #319
* fix(ci): resolve CI failures for wasm-bindgen migration
- Remove browser.js from napi generated file diff check (napi build
regenerates it, but we now maintain it manually for wasm-bindgen)
- Remove unused direct compression deps from napi crate (all logic
now lives in core-lib)
- Update e2e browser test to use wasm-bindgen output instead of
WASI browser entry
- Remove COOP/COEP headers from e2e vite config (no longer needed)
Part of #319
* fix(e2e): use manual WASM instantiation for wasm-bindgen in browser test
Vite 8 does not support the TC39 WebAssembly ESM integration proposal.
Import _bg.js directly and instantiate WASM binary manually, following
the same pattern used in rapid-fuzzy.
Also add lz4, crc32, and detectFormat tests to the browser e2e suite.
Part of #319
* fix(e2e): update browser test spec to match wasm-bindgen test suite
Replace streaming tests (removed with WASI migration) with lz4,
crc32, and detectFormat assertions.
Part of #319
* chore: add changeset for v1.0.0 major release (#322)
Closes #321
* chore(release): version packages to v1.0.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
derodero24
added a commit
that referenced
this pull request
Apr 4, 2026
* Set up napi-rs build pipeline (#9)
* chore(core): add Rust crate skeleton with napi-rs bindings
Minimal `zflate` crate with a single `version()` export to verify the
napi-rs build pipeline works end-to-end.
- crates/core/Cargo.toml: cdylib crate with napi-rs v3
- crates/core/build.rs: napi_build setup
- crates/core/src/lib.rs: version() function
* chore: add package.json with napi-rs configuration
Configure the npm package with 9 build targets (macOS, Linux, Windows,
WASM), dev dependencies for tooling (Biome, Vitest, TypeScript,
Changesets, CodSpeed), and build/test/lint scripts.
* chore: add build scripts and module entry points
- scripts/optimize-wasm.js: post-process WASM with wasm-opt
- index.mjs / index.d.mts: ESM entry points
- index.js / index.d.ts / browser.js: napi-rs generated files (force-tracked)
* Add CI workflow (#10)
* test: add initial test suite and ESM smoke test
- __test__/index.spec.ts: Vitest spec verifying version() export
- __test__/esm-import.mjs: ESM import smoke test for CI
* chore: update biome and tsconfig for test files
- biome.json: allow console in ESM smoke test
- tsconfig.json: format with Biome
* ci: add CI workflow
Full CI pipeline adapted from rapid-fuzzy:
- Lint (Biome, TypeScript, publint, generated file verification)
- Dependency audit (npm + cargo-deny)
- Rust lint (fmt + clippy) and test
- Cross-platform build (9 targets: macOS, Linux, Windows, WASM)
- Node.js test matrix (3 versions × 3 OSes)
- Coverage (Rust via cargo-llvm-cov + JS via Vitest, uploaded to Codecov)
* Add release and automation workflows (#11)
* ci: add release workflow with Cargo.toml version sync
Changesets-driven release flow:
- On develop push: create version PR, then release PR (develop → main)
- On main push: build all 9 targets and publish to npm with provenance
- scripts/sync-cargo-version.js: keep Cargo.toml version in sync with
package.json after changeset version bumps
* ci: add CodeQL security analysis
Analyze JavaScript/TypeScript and Rust code for security vulnerabilities.
Runs on PRs, develop pushes, and weekly schedule (Monday 6 AM UTC).
* ci: add CodSpeed benchmark workflow
Continuous performance tracking via CodSpeed. Runs JS benchmarks on
develop pushes and PRs when crate or bench files change.
Rust benchmarks will be added when the bench crate is introduced.
* ci: add self-hosted Renovate workflow
Automated dependency updates via self-hosted Renovate bot.
Runs weekly on Sunday 19:00 UTC (Monday 4:00 JST).
* test(bench): add initial benchmark file
Minimal benchmark to satisfy CodSpeed workflow. Real compression
benchmarks will be added alongside algorithm implementations.
* Implement zstd compression and decompression (#12)
* feat(zstd): implement zstd compression and decompression
Add zstdCompress() and zstdDecompress() functions powered by the zstd
C library via the zstd Rust crate.
- Compression levels 1-22 (default: 3)
- Auto-detection of frame content size for efficient decompression
- zstdDecompressWithCapacity() for data exceeding 256 MB default limit
- Rust unit tests for round-trip, empty data, large data, and levels
* test(zstd): add JS tests and benchmarks for zstd
- __test__/zstd.spec.ts: 11 tests covering round-trip, levels, errors
- __test__/esm-import.mjs: updated with zstd round-trip smoke test
- __test__/index.bench.ts: compress/decompress benchmarks at 150B/10KB/1MB
- index.mjs: export zstd functions
* chore: update napi-rs generated files and add changeset
- Regenerate index.js, index.d.ts, browser.js with zstd exports
- Add changeset for minor version bump
* ci: add WASI SDK setup for zstd WASM builds
The zstd crate compiles C code via zstd-sys, which requires a proper
WASI sysroot when targeting wasm32-wasip1-threads. Without it, clang
picks up system glibc headers and fails with missing
'bits/libc-header-start.h'.
Use bytecodealliance/actions/wasi-sdk/setup to install WASI SDK and
set CC, AR, CFLAGS, and WASI_SYSROOT environment variables for the
WASM build step in both CI and release workflows.
* Fix CodeRabbit review findings from #12 (#22)
* fix(zstd): validate capacity parameter in zstdDecompressWithCapacity
Reject negative, NaN, and Infinity values before casting f64 to usize.
Without validation, a negative f64 wraps to a very large usize, which
could cause memory exhaustion.
Closes #17
* fix(ci): use RUNNER_TEMP for WASI SDK installation
Replace /opt with $RUNNER_TEMP for WASI SDK extraction. /opt may not
be writable on all GitHub Actions runner configurations.
Also switch curl from -sL to -fsSL to fail on HTTP errors.
Closes #18
* docs: add SECURITY.md (#23)
* docs: add SECURITY.md
* docs: address review feedback on SECURITY.md
- Replace vague "extremely large size" with "massive size" for clarity
- Unify SLA day units by removing "business" qualifier from initial
assessment timeline for consistency
* docs: add CONTRIBUTING.md (#24)
* test(zstd): expand test coverage and benchmarks (#25)
* test(zstd): expand test coverage and benchmarks
Add comprehensive JS tests covering compression level boundaries (0,
negative, 22), decompressWithCapacity edge cases (insufficient, oversized,
invalid inputs), and round-trip verification with random data.
Expand benchmarks with random (incompressible) and realistic (JSON, text)
data variants alongside existing patterned data.
Update Rust doc comments to document negative compression levels and
level 0 behavior. Add Rust tests for boundary levels. Update changeset
to include all public API functions.
Closes #19
* chore: update napi-rs generated type definitions
Regenerate index.d.ts to reflect updated doc comments for
zstd_compress documenting negative levels and level 0 behavior.
* feat(zstd): add streaming compression/decompression API (#26)
Implement streaming zstd compression and decompression using the Web
Streams API (TransformStream). The Rust layer exposes ZstdCompressContext
and ZstdDecompressContext classes with transform/flush/finish methods
using zstd::stream::raw for stateful chunked processing.
The JS layer wraps these in TransformStream via createZstdCompressStream()
and createZstdDecompressStream(), available from both ESM and CJS entry
points. Streaming output is fully interoperable with one-shot APIs.
Add comprehensive tests covering chunked round-trips, various chunk
sizes, large data (1MB), random data, and one-shot/streaming interop.
Closes #5
* docs: add README and verify npm package readiness (#28)
* docs: add README and verify npm package readiness
Add comprehensive README with project description, installation guide,
quick-start examples (one-shot and streaming), full API reference,
supported algorithms table, platform compatibility matrix, and
benchmark results.
Verify npm package with publint (no errors). Package includes all
required files: index.js, index.mjs, index.d.ts, index.d.mts,
streams.js, streams.d.ts, browser.js, LICENSE, and README.md.
Closes #8
* docs(readme): address review feedback
- Add bun installation instructions
- Replace streaming example with self-contained runnable snippet
* fix(ci): resolve release workflow failures (#29)
* fix(ci): resolve release workflow failures
- Generate npm platform package directories for all 9 targets
- Add optionalDependencies to root package.json for platform binaries
- Fix changeset version command argument parsing in release workflow
by using a dedicated npm script instead of inline shell command
Closes #27
* fix(ci): remove optionalDependencies from committed package.json
The optionalDependencies for platform-specific packages are added
at publish time by the prepublishOnly hook (napi prepublish -t npm),
not in the committed source. Having them in package.json breaks
pnpm install --frozen-lockfile since the packages are not yet on npm.
* chore(release): version packages (#30)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(ci): regenerate napi bindings for v0.2.0 and fix version workflow (#34)
- Regenerate index.js with v0.2.0 version strings
- Update npm platform package versions to 0.2.0
- Add Rust toolchain to version job for napi build during version bump
- Add napi version, build, and format to changeset:version script
- Fix version test to read from package.json dynamically
* feat(gzip): add gzip and deflate compression support (#33)
Add gzip and raw deflate compression/decompression with one-shot and
streaming APIs. Includes interoperability tests with Node.js built-in
zlib module.
Closes #6
* feat(brotli): add brotli compression support (#32)
* feat(brotli): add brotli compression support
Add brotli compression/decompression with one-shot and streaming APIs.
Quality levels 0-11 (default: 6). Includes interoperability tests with
Node.js built-in zlib brotli module.
Closes #7
* test(brotli): add brotli compression benchmarks
Add brotli compress/decompress benchmarks matching the zstd benchmark
structure for patterned, random, and realistic data at various sizes.
* test(gzip): increase timeout for 1MB round-trip tests
Extend timeout from 5s to 30s for gzip and deflate 1MB data tests
to prevent CI failures on slower runners.
* test: increase timeout for all 1MB tests
Set 30s timeout on all 1MB data round-trip and streaming tests to
prevent failures on slower CI runners (Windows).
* docs(readme): update API reference for gzip and brotli (#36)
Add gzip, deflate, and brotli to the API reference tables. Update
supported algorithms table to reflect all three algorithms are now
available with both one-shot and streaming APIs.
* chore(deps): update taiki-e/install-action digest to c12d62a (#20)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(exports): add ./streams subpath export to package.json (#58)
Add a `./streams` subpath export so users can import streaming APIs
directly via `import { ... } from 'zflate/streams'`.
Closes #48
* fix(gzip): add compression level validation for gzip and deflate (#60)
Validate that the compression level parameter is within the valid range
(0-9) for gzip_compress, deflate_compress, GzipCompressContext, and
DeflateCompressContext. Return InvalidArg error if out of range, matching
the existing validation pattern in brotli_compress.
Closes #38
* fix(gzip): add decompression size limits to gzip and deflate (#64)
* fix(gzip): add decompression size limits to gzip and deflate
Add MAX_DECOMPRESSED_SIZE (256 MB) to gzip_decompress and
deflate_decompress to prevent zip bomb attacks. Both functions now use
chunked reading with size checking instead of unbounded read_to_end().
Add gzipDecompressWithCapacity and deflateDecompressWithCapacity for
explicit control over the maximum decompressed size, matching the
existing zstd and brotli patterns.
Closes #37
* chore(napi): regenerate bindings for new WithCapacity functions
* feat(api): accept Uint8Array input in all compression functions (#69)
Change all input parameters from Buffer to Either<Buffer, Uint8Array>
across all one-shot and streaming functions. This generates
Buffer | Uint8Array TypeScript types, enabling browser/Deno/Bun users
to pass Uint8Array directly without type errors.
Return types remain Buffer. No runtime behavior changes for existing
Buffer usage.
Closes #42
* docs: enhance README with examples, comparison table, and migration guides (#63)
* test: add cargo-fuzz targets for decompression fuzzing (#71)
* test: add edge case and error handling tests (#65)
* test: expand Node.js zlib interop tests to brotli and zstd (#67)
* ci: add hand-written JS files to workflow path filters (#56)
* ci(codeql): suppress rust/access-invalid-pointer false positives (#57)
* fix(dx): add ./streams subpath export to package.json (#59)
* ci: add cargo-audit to dependency audit pipeline (#61)
* feat(core): add unified auto-detect decompression API (#72)
Add decompress() function that auto-detects compression format from
magic bytes (zstd: 0xFD2FB528, gzip: 0x1F8B) and falls back to brotli
heuristic detection. Add detectFormat() for format identification
without decompression.
Supported formats: zstd, gzip, brotli. Raw deflate cannot be
auto-detected due to lack of magic bytes.
Closes #16
* ci(dx): add @arethetypeswrong/cli type validation (#62)
* perf(bench): add competitive benchmarks across all algorithms (#70)
* perf(wasm): add WASM binary size reporting to CI (#66)
* fix(exports): add missing ESM exports for new functions (#74)
Add decompress, detectFormat, gzipDecompressWithCapacity, and
deflateDecompressWithCapacity to index.mjs ESM export list. These
functions were available via CJS but missing from ESM imports.
Update ESM smoke test to cover all WithCapacity functions, decompress,
and detectFormat.
Closes #73
* feat(node): add Node.js stream.Transform compatibility layer (#76)
Add `zflate/node` subpath export with stream.Transform factories for all
compression algorithms (zstd, gzip, deflate, brotli), enabling use with
Node.js stream.pipeline() and pipe-based workflows.
Closes #43
* fix(brotli): use consistent error message for capacity exceeded (#78)
Change brotli_decompress_with_capacity error message from
"Destination buffer is too small" to "brotli decompress exceeded
maximum size of {} bytes" to match the pattern used by all other
decompress functions.
Closes #77
* fix(docs): remove merge conflict marker and stale src/ references (#80)
* feat(core): add async compression/decompression API (#82)
Add async versions of all one-shot compression/decompression functions
using napi-rs AsyncTask with the Task trait. This runs compression on
the libuv thread pool without blocking the Node.js event loop.
New functions: zstdCompressAsync, zstdDecompressAsync,
gzipCompressAsync, gzipDecompressAsync, deflateCompressAsync,
deflateDecompressAsync, brotliCompressAsync, brotliDecompressAsync.
Closes #40
* feat(zstd): add dictionary compression API (#83)
Add zstd dictionary training, compression, and decompression functions:
- zstdTrainDictionary: train a dictionary from sample data
- zstdCompressWithDict: compress with a pre-trained dictionary
- zstdDecompressWithDict: decompress data compressed with a dictionary
Enable the zdict_builder feature for the zstd crate to support
dictionary training. Include Rust unit tests and Vitest integration
tests verifying round-trip correctness and improved compression ratios
for small, similar data.
Closes #41
* docs: remove Cloudflare Workers from supported platforms (#91)
* ci: include Node.js WASM loader files in CI artifact (#90)
* test(wasm): add Node.js WASM compatibility test with native parity verification (#92)
* refactor(core): introduce thiserror for structured error types (#93)
Replace manual `Error::new(Status::..., ...)` calls with structured
`ZflateError` enum variants across all compression modules. Error
messages remain identical to preserve backward compatibility.
Closes #51
* test(deno): add Deno WASM smoke test (#96)
* test(bun): add Bun WASM smoke test (#95)
* test(browser): add Playwright browser WASM test (#94)
* ci(security): tighten cargo-deny bans and sources to deny mode (#108)
* ci: pin dtolnay/rust-toolchain to SHA digest (#107)
* fix(core): add decompression size limits to streaming contexts (#109)
Add cumulative output size tracking to all streaming decompression
contexts (ZstdDecompressContext, GzipDecompressContext,
DeflateDecompressContext, BrotliDecompressContext).
Each context now enforces MAX_DECOMPRESSED_SIZE (256 MB) across
transform() calls, preventing decompression bomb attacks that could
cause unbounded memory growth.
Closes #97
* feat(core): narrow detectFormat return type to string literal union (#111)
Use napi-rs string_enum to return CompressionFormat instead of plain
string, generating a const enum with 'zstd' | 'gzip' | 'brotli' |
'unknown' values in TypeScript.
Closes #103
* perf(core): optimize buffer allocation in compression and decompression (#110)
* perf(core): optimize buffer allocation in compression and decompression
Increase decompression read buffer from 4KB to 32KB in gzip, brotli,
and brotli_stream modules to reduce syscall overhead during chunked
reads. Pre-allocate compression output Vecs with input.len() capacity
for gzip and deflate encoders (both sync and async) to avoid repeated
reallocations during compression.
Closes #100
* revert: keep BUFFER_SIZE at 4096 to avoid decompress regression
The 32KB buffer caused 21 benchmark regressions in CodSpeed due to
increased stack zeroing cost. Revert to 4KB. The Vec::with_capacity
optimization for compression output is retained.
* perf(core): replace Vec clone and split_off with std::mem::take in streaming contexts (#112)
Use std::mem::take() instead of .get_ref().clone() + .get_mut().clear()
in brotli streaming contexts, and instead of .split_off(0) in gzip/deflate
streaming contexts. This avoids unnecessary memory allocation and copying
by moving ownership of the buffer contents in-place.
Closes #99
* fix(zstd): add compression level validation (#113)
Validate that the compression level is within the valid zstd range
(-131072 to 22) before passing it to the zstd library. This provides
clear error messages instead of opaque library errors.
Validation is added to:
- zstd_compress (sync)
- zstd_compress_async
- ZstdCompressContext::new (streaming)
- zstd_compress_with_dict
Closes #81
* feat(zstd): add streaming dictionary compression/decompression (#114)
Add ZstdCompressDictContext and ZstdDecompressDictContext classes that
accept a pre-trained dictionary for streaming compression/decompression.
Include Web Streams (createZstdCompressDictStream/createZstdDecompressDictStream)
and Node.js Transform (createZstdCompressDictTransform/createZstdDecompressDictTransform)
wrappers, type definitions, ESM exports, and streaming dict round-trip tests.
Closes #104
* perf(core): switch flate2 backend from miniz_oxide to zlib-rs (#115)
Replace the `rust_backend` (miniz_oxide) feature with `zlib-rs` for
2-3x faster gzip/deflate compression. zlib-rs is a pure-Rust
implementation that works on all targets including WASM, is ISRG
audited, and is on track to become flate2's default backend.
Relax compression level size ordering test since zlib-rs may produce
slightly different output sizes per level than miniz_oxide — round-trip
correctness is what matters.
Closes #98
* perf(zstd): enable multi-threaded compression for native targets (#116)
Add `zstdmt` feature for non-WASM targets via conditional dependency.
This enables zstd's internal thread pool for compression, improving
performance by 20-80% for large inputs on multi-core systems.
WASM targets are excluded because wasi-threads is experimental and
the interaction with napi-rs worker-based threading is untested.
Closes #102
* docs: comprehensive README update with missing APIs and SVG benchmark (#118)
* fix(docs): correct benchmark environment label to Apple M2 / Node.js v22 (#119)
* perf(wasm): add --converge flag to wasm-opt optimization (#122)
Re-run optimization passes until no further gains are achieved.
Increase timeout to 5 minutes to accommodate additional passes.
Closes #121
* perf(core): use 4x input size for decompression output pre-allocation (#123)
Compressed data typically decompresses to several times its original
size. Pre-allocating with input.len() alone leads to frequent
reallocations during decompression. Using input.len() * 4 as the
initial capacity provides a better heuristic while the existing
.min() caps continue to prevent excessive allocation.
Closes #120
* ci: migrate release workflow to npm trusted publishing (OIDC) (#68)
* ci: migrate release workflow to npm trusted publishing (OIDC)
Remove NPM_TOKEN/NODE_AUTH_TOKEN secret dependency and rely on
OIDC token exchange via id-token:write permission (already set).
Trusted publishing must be configured on npmjs.com for each package
before the next release.
* fix(ci): restore pinned SHA for dtolnay/rust-toolchain
Keep the pinned commit hash consistent with ci.yml and codspeed.yml
for supply chain security. The rust-toolchain change was out of scope
for the trusted publishing migration.
* fix(gzip): support concatenated gzip streams in decompression (#131)
Replace GzDecoder with MultiGzDecoder from flate2 to correctly handle
concatenated gzip members in both one-shot and streaming decompression.
* fix(streaming): add finish-guard state tracking for brotli and zstd compress contexts (#132)
* feat(core): add missing async variants for one-shot APIs (#133)
* feat(streaming): add auto-detect decompression streams (#145)
* docs: add troubleshooting section to CONTRIBUTING.md (#146)
* docs: add CODE_OF_CONDUCT.md (#149)
* ci: apply least-privilege permissions to workflows (#147)
* chore: improve package.json metadata and exports map (#148)
* ci: clean up CI workflow configuration (#150)
* feat(core): expose CRC32 utility function (#134)
* feat(core): expose CRC32 utility function
* test(crc32): use skipIf for conditional Node.js zlib interop test
* ci(testing): add coverage thresholds to vitest config (#151)
* ci: add aggregate gate job to CI workflow (#152)
* feat(streaming): add configurable maxOutputSize to decompression contexts (#153)
Add optional maxOutputSize parameter to all streaming decompression
context constructors (GzipDecompressContext, DeflateDecompressContext,
ZstdDecompressContext, BrotliDecompressContext, ZstdDecompressDictContext).
When not specified, the default 256 MB limit is used.
The parameter is also passed through all streaming factory functions
(createXxxDecompressStream, createXxxDecompressTransform, and
createDecompressStream/createDecompressTransform).
Closes #129
* ci: add environment protection for npm publish (#154)
* ci: add cargo-shear for unused dependency detection (#155)
* ci: add WASM binary size tracking on pull requests (#156)
* ci: add scripts/ to CI workflow path triggers (#159)
* fix(wasm): add JS-side streaming adapters for browser WASM environment (#158)
Replace native streaming contexts with JS-side adapters in the browser
entry point to work around WebAssembly.Memory growth invalidating
ArrayBuffer views. Adapters accumulate chunks and delegate to one-shot
APIs, maintaining the same transform()/flush()/finish() API surface.
- browser-streaming.js: adapter classes for all 10 streaming contexts
- browser-entry.js: re-exports WASM APIs with adapter overrides
- package.json: browser condition points to browser-entry.js
- e2e: streaming tests enabled for browser WASM
* feat(gzip): add header metadata support for compression and reading (#160)
Closes #130
* chore(renovate): run daily and remove PR concurrent limit (#164)
* chore(renovate): run daily and remove PR concurrent limit
Closes #163
* chore(renovate): disable lock file maintenance
* chore(deps): update codspeedhq/action digest to 1c8ae48 (#161)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dtolnay/rust-toolchain digest to 631a55b (#162)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to 3869755 (#165)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to b988c18 (#166)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(bench): exclude third-party benchmarks from CodSpeed tracking (#169)
* fix(bench): exclude third-party benchmarks from CodSpeed tracking
Split benchmark files into zflate-only (*.bench.ts) and comparison
(*.compare.bench.ts) variants. CodSpeed now only tracks zflate's own
benchmarks, eliminating false regressions from pako/fflate/node:zlib
that are caused by GitHub Actions CPU heterogeneity affecting Callgrind
instruction counting.
Closes #167
* fix(bench): use deterministic random data in CodSpeed benchmarks
Replace non-deterministic randomBytes() with a seeded LCG to ensure
reproducible benchmark inputs across runs, reducing CodSpeed instruction
count variance.
* feat(bench): migrate from CodSpeed to Bencher.dev for regression detection (#170)
Replace CodSpeed simulation mode (Callgrind instruction counting) with
Bencher.dev (statistical wall-clock analysis). CodSpeed's instruction
counting was fundamentally incompatible with this napi-rs project due to
GitHub Actions CPU heterogeneity and glibc's CPU-adaptive malloc.
Changes:
- Add Bencher workflow with t-test threshold for regression detection
- Add vitest-to-bmf.js conversion script for vitest bench output
- Remove CodSpeed workflow and @codspeed/vitest-plugin dependency
- Remove CodSpeed plugin from vitest config
Closes #168
* feat(bench): switch to CodSpeed Walltime mode for accurate benchmarks (#172)
Replace Bencher.dev with CodSpeed Walltime mode (Macro Runners) which
runs benchmarks on dedicated bare-metal hardware, eliminating GitHub
Actions CPU variance. CodSpeed also provides native vitest integration
via @codspeed/vitest-plugin, removing the need for conversion scripts.
Key change: `mode: walltime` instead of the previous `mode: simulation`
which used Callgrind instruction counting.
Closes #171
* refactor(bench): split benchmark files per algorithm for granular CodSpeed reporting (#174)
Split index.bench.ts (mixed zstd+brotli) into separate zstd.bench.ts
and brotli.bench.ts so CodSpeed PR reports only show benchmarks relevant
to the changed algorithm. Extract shared test fixtures into
bench-fixtures.ts and add realistic data benchmarks (JSON, text) to
gzip and deflate for consistent coverage across all four algorithms.
Closes #173
* perf(core): eliminate double-copy buffer pattern in decompression (#178)
* perf(core): eliminate double-copy buffer pattern in decompression
Replace the manual 4 KiB read loop (stack buffer → Vec copy) with
read_to_end + Take, which writes directly into Vec spare capacity.
This eliminates one full copy of every decompressed byte and uses
adaptive buffer sizing instead of fixed 4 KiB chunks.
Affected: gzip, deflate, brotli (sync + async, default + with-capacity)
Closes #175
* fix(core): use saturating_mul for init capacity and update size limit tests
Address CodeRabbit review feedback:
- Replace len() * 4 with len().saturating_mul(4) across all call sites
to prevent overflow on 32-bit targets (wasm32-wasip1-threads)
- Replace old chunk-loop size limit tests with Take+read_to_end tests
that exercise the actual decompression pattern (avoiding napi::Error
symbol resolution issues in standalone test binaries)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks (#179)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks
- Add bench:ci script that excludes comparison benchmarks
- Simplify CodSpeed workflow to use bench:ci instead of explicit file list
- Use glob pattern for bench file path triggers
- Add crates/bench workspace member with Criterion benchmarks
(gzip, deflate, brotli, zstd) using codspeed-criterion-compat
- Add rust-benchmarks job to CodSpeed workflow (simulation mode)
Closes #177
* fix(ci): add Cargo.toml, Cargo.lock, package.json to CodSpeed path triggers
Ensures CodSpeed benchmarks run when workspace config or dependency
lockfiles change, which could affect benchmark behavior.
* feat(core): add LZ4 frame compression support (#180)
* feat(core): add LZ4 frame compression support
Add LZ4 as a fourth compression algorithm using lz4_flex (pure Rust):
- Bulk API: lz4Compress, lz4Decompress, lz4DecompressWithCapacity
- Async API: lz4CompressAsync, lz4DecompressAsync, lz4DecompressWithCapacityAsync
- Streaming: Lz4CompressContext, Lz4DecompressContext
- Web Streams: createLz4CompressStream, createLz4DecompressStream
- Format detection: detectFormat returns "lz4" for LZ4 frame magic bytes
- Auto-decompress: decompress/decompressAsync and createDecompressStream
now handle LZ4 data automatically
Closes #176
* fix(core): add Read import to LZ4 test modules
Tests use read_to_end which requires the Read trait in scope.
* refactor(core): validate capacity as non-negative integer (#183)
* refactor(core): validate capacity as non-negative integer across all algorithms
Extract shared validate_capacity() that rejects NaN, Infinity, negative,
fractional values, and values exceeding usize::MAX. Applied to all 10
DecompressWithCapacity functions (sync + async × 5 algorithms).
Closes #181
* test: add regression tests for fractional and oversized capacity
* perf(lz4): implement incremental streaming for LZ4 compression (#184)
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with cursor-based incremental output in
Lz4CompressContext. The FrameEncoder now persists across transform()
calls, emitting compressed blocks as they become available instead of
buffering all input until finish().
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with FrameEncoder-based incremental output
in Lz4CompressContext. Compressed blocks are emitted as internal buffers
fill, and old bytes are drained to bound memory usage.
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* chore(deps): update taiki-e/install-action digest to 0620367 (#185)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust crate lz4_flex to 0.13 (#186)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore: rename package from zflate to comprs (#190)
* chore: rename package from zflate to comprs
Rename npm package, Rust crates, napi binary name, platform packages,
and all references from "zflate" to "comprs". The name "comprs" conveys
"compress" with a Rust flavor, is available on npm, and avoids the
typosquat protection that blocks "zflate".
Changes across 70 files:
- npm: comprs, comprs-darwin-arm64, comprs-wasm32-wasi, etc.
- Rust: comprs (core), comprs-bench
- napi binaryName: comprs (→ comprs.*.node, comprs.*.wasm)
- README, workflows, changesets, tests, benchmarks
GitHub repository URL (derodero24/zflate) is preserved.
Closes #189
* chore: update GitHub repo URLs to derodero24/comprs
Update all repository references to match the planned GitHub repo
rename from derodero24/zflate to derodero24/comprs.
* chore(core): rename ZflateError to ComprsError and update bench vars (#192)
Rename internal Rust error type `ZflateError` → `ComprsError` across
all source files and update benchmark variable names `*_ZFLATE` →
`*_COMPRS` for consistency with the package rename in #190.
Closes #191
* fix: add missing LZ4 docs, browser exports, Node.js transforms, and publish files (#194)
- Add LZ4 one-shot and streaming context exports to browser-entry.js
- Add Lz4CompressContext/Lz4DecompressContext adapters to browser-streaming.js
- Add createLz4CompressTransform/createLz4DecompressTransform to node.js
- Add LZ4 type declarations to node.d.ts
- Add 'lz4' case to createDecompressContext in node.js
- Add browser-entry.js and browser-streaming.js to package.json files array
- Document LZ4 API in README.md (one-shot, async, streaming, comparison)
- Fix streaming import path in README.md (comprs → comprs/streams)
- Update auto-detect descriptions to include LZ4
- Add "lz4" to package.json keywords and description
- Add LZ4 changeset for upcoming release
Closes #193
* fix: add missing LZ4 exports to ESM entry point and complete API docs (#196)
- Add LZ4 one-shot APIs, Context classes, and streaming functions to
index.mjs (ESM users could not access LZ4 at all)
- Add createLz4CompressStream/createLz4DecompressStream to index.mjs
- Add gzipCompressWithHeader and gzipReadHeader to browser-entry.js
- Document gzipCompressWithHeader, gzipReadHeader in README
- Document crc32 and version() utilities in README
- Document createDecompressStream and createDecompressTransform in README
Closes #195
* fix(docs): add missing lz4 to decompressAsync doc and clarify async section (#198)
The decompressAsync Rust doc comment listed only zstd, gzip, brotli as
supported formats, but the implementation also handles lz4. The README
async section claimed "all one-shot functions" have async variants, which
is inaccurate for utility functions like crc32, version, gzipReadHeader.
Closes #197
* chore(release): version packages (#35)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* perf(ci): remove JS walltime benchmarks from CodSpeed regression detection (#201)
JS walltime benchmarks on standard GitHub Actions runners produce 20-50%
variance due to CPU heterogeneity (AMD EPYC vs Intel Xeon assigned randomly)
and glibc's CPU-adaptive malloc. This caused CodSpeed Performance Analysis to
fail on nearly every PR since #174, including documentation-only changes.
Keep only Rust simulation benchmarks in CodSpeed, which use instruction
counting via Callgrind and produce <1% variance. Also narrow the workflow
path triggers to Rust-only files since JS changes no longer affect the
CodSpeed pipeline.
JS benchmarks remain available locally via `pnpm run bench`.
Closes #200
* ci(release): use granular access token for initial npm publish
The first publish requires an NPM_TOKEN since OIDC trusted publishing
only works for packages that already exist on the registry. After the
initial release, this will be switched back to OIDC.
* chore: add .mcp.json to .gitignore (#205)
Closes #203
* fix(core): use decompress_with_limit in async auto-detect decompression (#216)
Replace manual chunk-read loops with double-copy in DecompressTask for
gzip and brotli branches. Now uses crate::decompress_with_limit(),
matching the pattern already used by the LZ4 branch and per-algorithm
async tasks.
Closes #207
* ci: add browser files to CI path triggers (#217)
Closes #212
* fix(zstd): validate maxDictSize parameter in dictionary training (#219)
* test: update ESM smoke test to cover all exports (#215)
* Add LZ4 edge-case and streaming test coverage (#218)
* test(lz4): add edge-case and streaming test coverage
* test(lz4): fix empty buffer test to match actual lz4 behavior
LZ4 returns empty buffer for empty input (like zstd and deflate),
rather than throwing an error (like gzip and brotli).
* fix(zstd): enforce maxOutputSize in browser dict decompression (#221)
* fix(zstd): enforce maxOutputSize in browser dict decompression
Add zstd_decompress_with_dict_with_capacity Rust function and use it
in browser-streaming.js ZstdDecompressDictContext.flush() when
maxOutputSize is provided.
Closes #208
* chore: regenerate napi-rs binding files
* refactor(core): consolidate MAX_DECOMPRESSED_SIZE constant (#222)
Remove duplicated MAX_DECOMPRESSED_SIZE definitions from zstd, gzip,
brotli_impl, lz4, and detect modules. Change the lib.rs definition to
pub(crate) and reference it via crate::MAX_DECOMPRESSED_SIZE in all
modules.
* chore(release): version packages (#220)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* docs: add benchmark comparison results to README (#224)
Closes #213
* feat(brotli): add dictionary compression and decompression support (#225)
* docs: add brotli dictionary to README and update Node.js transforms (#228)
* docs: readme visual and structural overhaul (#239)
- Center-align hero section with title, tagline, and badges
- Add Node.js version and TypeScript badges
- Add Table of Contents for navigation
- Wrap benchmark tables in collapsible <details> sections
- Replace plain blockquote with GitHub Admonition (> [!TIP])
- Replace raw build target triples with human-readable table
Closes #229
* docs: reposition and enhance comparison table (#240)
- Move comparison table from bottom of README to right after "Why comprs?"
- Split "Streaming" into "Web Streams API" and "Node.js Transform" rows
- Replace ambiguous `✅*` with `⚠️ Experimental*` for node:zlib zstd
- Replace footnote `†` with inline "Chunked†" for pako streaming
- Update Table of Contents to reflect new section order
- Center-align feature columns for better scannability
Closes #236
* docs: improve quick start examples and add runtime-specific usage (#241)
- Condense one-shot examples to single primary zstd block + multi-algorithm block
- Replace impractical manual ReadableStream example with fetch-style streaming
- Add Node.js Transform stream example to Quick Start
- Add brief comment explaining async runs on libuv thread pool
- Consolidate compression levels into compact format
- Add Deno and Bun import examples
Closes #235
* docs: add algorithm selection guide (#242)
- Add "Choosing an Algorithm" section with use case → recommendation table
- Add "Choosing an API mode" subsection for sync/async/streaming/dictionary
- Update Table of Contents
Closes #234
* docs: document browser WASM requirements (SharedArrayBuffer, COOP/COEP) (#243)
- Add SharedArrayBuffer requirement warning with required HTTP headers
- Remove misleading "or import directly from a CDN" claim
- Link to MDN SharedArrayBuffer security requirements
Closes #231
* docs: add SSR framework integration guide (#244)
- Add "Framework Integration (SSR)" subsection under Browser Usage
- Include Next.js serverExternalPackages and Vite SSR external config
- Note automatic WASM fallback on client side
Closes #232
* docs: add fflate migration guide (#245)
- Add "From fflate" migration section with diff example
- Note comprs advantages over fflate (zstd, lz4, brotli, dictionaries, Web Streams)
Closes #237
* docs: add known limitations and performance notes (#246)
- Add "Notes" section with three admonitions covering:
- 256 MB default decompression limit with WithCapacity() workaround
- Small payload WASM overhead advisory
- Brotli decompression performance nuance vs node:zlib
Closes #238
* docs: add all undocumented async and capacity API variants to readme (#247)
- Add zstdDecompressWithDictWithCapacity to sync zstd dictionary table
- Add 9 missing async variants: WithCapacityAsync for all algorithms,
zstd dict async, zstdTrainDictionaryAsync, decompressAsync
- Reorder async table to group by algorithm with capacity/dict variants
Closes #233
* docs: add benchmark comparison SVG charts for cross-algorithm and brotli (#248)
- Add cross-algorithm compression chart (zstd/lz4/gzip/brotli at 10KB)
- Add brotli compression speedup chart (134.7x faster than node:zlib)
- Embed both charts alongside existing gzip chart in Benchmarks section
Closes #230
* docs: fold repetitive API tables into collapsible sections (#250)
- Wrap Dictionary API (zstd + brotli) in <details>
- Replace Async table header with pattern explanation + code example,
fold full function list into <details>
- Add Streaming import path + fold table into <details>
- Keep Node.js Transform code example visible, fold table into <details>
- One-shot core API tables remain visible as primary reference
Closes #249
* chore(deps): update taiki-e/install-action digest to 7627fb4 (#251)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.9 (#252)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update renovatebot/github-action action to v46.1.6 (#254)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency vite to v8.0.3 (#253)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update pnpm to v10.33.0 (#256)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.1 (#255)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency typescript to v6 (#257)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* feat: add interactive playground with GitHub Pages deployment (#259)
Add a browser-based playground for trying comprs compression live.
Includes four algorithms (zstd, gzip, brotli, lz4), level slider,
comparison table, file upload, and download. Deploys to GitHub Pages
via new CI workflow. Uses a JS mock for local dev without a WASM build.
Closes #258
* docs: improve playground visibility and add OGP/SEO meta tags (#261)
Add prominent text link below badges in README header and playground
as a bullet in "Why comprs?". Add Open Graph, Twitter Card, and
canonical URL meta tags to playground/index.html for social sharing.
Closes #260
* fix(playground): handle COI not established on first visit (#263)
On first visit, the COI service worker has not yet reloaded the page,
so crossOriginIsolated is false. Calling the WASM function immediately
throws because SharedArrayBuffer is unavailable. This showed a
misleading "unsupported browser" error even in modern browsers like
Vivaldi, Brave, or Arc.
- Check crossOriginIsolated before invoking WASM
- Show "Enabling security features... reloading" while the service
worker handles the reload
- After 3 s, show a manual Reload button if auto-reload did not happen
- Include the actual error message when WASM init itself fails
- Add .loading-reload-btn styles to style.css
Closes #262
* fix(playground): add Buffer polyfill for emnapi WASM compatibility (#265)
The napi-rs WASM runtime (emnapi) requires globalThis.Buffer when
creating memory views for function arguments/return values. Browsers
do not define Buffer natively, causing zstdCompress and other WASM
calls to throw immediately.
Add the 'buffer' npm package (v6 browser polyfill) to playground
dependencies and set globalThis.Buffer before any WASM calls.
Closes #264
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init (#267)
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init
Static imports are hoisted and evaluated before any module body code runs.
This caused globalThis.Buffer to be unset when comprs-wasm32-wasi evaluated
__emnapiInstantiateNapiModuleSync (which calls emnapi_create_memory_view).
Switch to a dynamic import so the Buffer polyfill assignment executes first.
Closes #266
* fix(playground): allow Vite dev server to serve WASM from parent directory
When comprs.wasi-browser.js is present in the repo root, the WASM file
is fetched at runtime via /@fs/ from outside the playground directory.
Add server.fs.allow so Vite does not block the request with 403.
Only applied when a local WASM build is detected (hasLocalWasm=true).
* fix(playground): fix comparison table label direction and badge layout (#271)
- Rename bar chart column 'Ratio (smaller = better)' to 'Savings'
and numeric column 'Ratio' to 'Saved' to accurately reflect that
the metric is compression savings % (larger = better)
- Change 'best ratio' badge to 'best' (shorter, clearer)
- Add 'fastest' badge to the fastest algorithm row; remove the
duplicate 'Fastest:' text from the footer note
- Change badge column from fixed 72px to auto to prevent overflow
Closes #270
* fix(playground): fix negative ratio text, CSS variable, download label, and file size alert (#273)
- Show "X% larger" instead of "-X% smaller" when compressed output exceeds input
- Fix reload button using undefined var(--accent); use var(--text)/var(--bg) instead
- Add file extension to download button (e.g. "Download .zst") via #download-ext span
- Replace alert() for oversized file with inline error message that auto-dismisses after 5s
Closes #272
* chore(deps): update codecov/codecov-action digest to 75cd116 (#268)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.2 (#269)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(playground): add FileReader error handler and textarea aria-label (#275)
- Add reader.onerror handler to show inline error when file reading fails
instead of silently doing nothing (e.g. on disk error or revoked access)
- Add aria-label to the input textarea for screen reader accessibility
Closes #274
* chore(deps): update dtolnay/rust-toolchain digest to 29eef33 (#276)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to c10b806 (#277)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/configure-pages action to v6 (#283)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/deploy-pages action to v5 (#284)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/upload-pages-artifact action to v4 (#285)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to d858f81 (#278)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust-dependencies to v3.8.4 (#281)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.2 (#280)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.10 (#279)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update codecov/codecov-action action to v6 (#286)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 (#287)
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2
Update @napi-rs/cli from v3.5.1 to v3.6.0 along with @emnapi/core
and @emnapi/runtime from v1.9.1 to v1.9.2 to resolve emnapi version
mismatch that caused WASM build failures.
* fix(deps): align emnapi transitive dependency to v1.9.2
Update the emnapi package (transitive via @napi-rs/cli) from v1.9.1
to v1.9.2, matching @emnapi/core and @emnapi/runtime versions.
* chore(deps): update renovatebot/github-action action to v46.1.7 (#292)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @types/node to v24.12.2 (#291)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(config): update biome.json schema to v2.4.10 (#294)
* chore(deps): update dependency @playwright/test to v1.59.1 (#293)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(core): remove unused wildcard import in crc test module (#295)
* fix(ci): update setup-deno and changesets/action to resolve Renovate lookup failures (#296)
Pin to specific version tags instead of major version floating tags,
which do not exist in these repositories. This allows Renovate to
resolve digests for tracking updates.
- denoland/setup-deno: v2 → v2.0.4
- changesets/action: v1 → v1.7.0
Closes #290
* chore(release): version packages to v0.4.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* refactor: rename npm package to @derodero24/comprs (#298)
Rename all npm packages from `comprs-*` to `@derodero24/comprs-*` to
bypass npm's typosquatting protection that blocked the `comprs` name.
- Root package: comprs → @derodero24/comprs
- 9 platform packages: comprs-{platform} → @derodero24/comprs-{platform}
- Regenerated napi-rs loader (index.js, browser.js) with new names
- Updated release workflow version checks
- Updated README install/import examples
- Updated playground npm link
Closes #297
* fix(playground): update vite alias for renamed WASM package
Update the vite resolve alias from 'comprs-wasm32-wasi' to
'@derodero24/comprs-wasm32-wasi' to match the package rename in #298.
* fix(ci): add publishConfig and permissions for scoped npm publish (#302)
- Add publishConfig.access: "public" to root and all platform
package.json files so napi prepublish can publish scoped packages
- Add contents: write permission to Publish job for GitHub Release
creation by napi prepublish
- Add changeset for v0.4.1 patch release
Closes #301
* chore(release): version packages to v0.4.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: update stale package references after rename to @derodero24/comprs (#307)
Update remaining references from old unscoped names to scoped names:
- browser-streaming.js: comprs-wasm32-wasi → @derodero24/comprs-wasm32-wasi
- playground/main.js: import('comprs') → import('@derodero24/comprs')
- playground/package.json: comprs → @derodero24/comprs dependency
- playground/vite.config.js: Vite alias keys updated
This fixes the GitHub Pages deployment failure caused by Vite being
unable to resolve the old unscoped import in browser-streaming.js.
Closes #306
* fix(playground): regenerate lockfile after package rename
The playground's pnpm-lock.yaml was not updated when the dependency
was renamed from comprs to @derodero24/comprs, causing CI to fail
with frozen-lockfile mismatch.
* docs: fix README package references, streaming note, and WASM size (#311)
* fix(docs): update remaining old package name references in README
- Deno import: npm:comprs → npm:@derodero24/comprs
- Subpath imports: comprs/streams → @derodero24/comprs/streams,
comprs/node → @derodero24/comprs/node
- SSR config: Next.js and Vite external package names
Closes #308
* docs: clarify browser streaming memory behavior
Add note explaining that browser WASM streaming buffers all data in
memory (unlike Node.js which streams incrementally). Also qualify the
"bounded memory" claim in the feature list to specify Node.js.
Closes #309
* docs: add concrete WASM binary size to README
Show approximate optimized WASM size (~2.0 MB raw, ~800 KB gzipped)
directly in README so users can evaluate without navigating to CI.
Closes #310
* fix: update npm badges and sub-package descriptions (#317)
* fix(docs): update npm badges to use scoped package name
The npm version and downloads badges were pointing to the old unscoped
package name "comprs" instead of the scoped "@derodero24/comprs".
Closes #312
* fix: update platform sub-package descriptions to include lz4
All 9 platform sub-packages under npm/ had descriptions listing only
"zstd, gzip, and brotli" but were missing "lz4" which was added to
the root package. Updated to match: "zstd, gzip, brotli, and lz4".
Closes #315
* fix: add missing ESM export and test coverage (#318)
* fix: add missing zstdDecompressWithDictWithCapacity to ESM and browser exports
Add zstdDecompressWithDictWithCapacity to index.mjs destructured exports,
browser-entry.js one-shot re-exports, and ESM smoke test assertions.
Closes #313
* test: add coverage for WithDictWithCapacity functions and Node.js Transform streams
Add tests for zstdDecompressWithDictWithCapacity, brotliDecompressWithDictWithCapacity,
brotliDecompressWithDictWithCapacityAsync, LZ4 Node.js Transform round-trip,
zstd dict Node.js Transform round-trip, and brotli dict Node.js Transform round-trip.
Closes #314
* feat(wasm): replace WASI WASM with wasm-bindgen for browser compatibility (#320)
* refactor(core): extract pure Rust logic into core-lib crate
Split crates/core into two crates:
- crates/core-lib: pure Rust compression logic (no FFI dependencies)
- crates/core: napi-rs thin wrappers delegating to core-lib
This prepares for adding a wasm-bindgen crate that shares the same
core logic, enabling browser usage without SharedArrayBuffer.
Part of #319
* feat(wasm): add wasm-bindgen browser binding crate
Create crates/wasm with wasm-bindgen wrappers over comprs-core,
targeting wasm32-unknown-unknown (no SharedArrayBuffer required).
Includes one-shot and streaming APIs for all algorithms:
zstd, gzip, deflate, brotli, lz4, auto-detect, and crc32.
Part of #319
* feat(wasm): integrate wasm-bindgen build into package distribution
- Add build:wasm-bindgen script using wasm-pack
- Update browser.js to use wasm-bindgen output
- Update browser-streaming.js imports
- Add wasm-bindgen files to npm package
- Update CI with wasm-bindgen build job
- Update release workflow for wasm-bindgen artifacts
- Update playground configuration
- Remove SharedArrayBuffer requirement from README
Part of #319
* fix(ci): resolve CI failures for wasm-bindgen migration
- Remove browser.js from napi generated file diff check (napi build
regenerates it, but we now maintain it manually for wasm-bindgen)
- Remove unused direct compression deps from napi crate (all logic
now lives in core-lib)
- Update e2e browser test to use wasm-bindgen output instead of
WASI browser entry
- Remove COOP/COEP headers from e2e vite config (no longer needed)
Part of #319
* fix(e2e): use manual WASM instantiation for wasm-bindgen in browser test
Vite 8 does not support the TC39 WebAssembly ESM integration proposal.
Import _bg.js directly and instantiate WASM binary manually, following
the same pattern used in rapid-fuzzy.
Also add lz4, crc32, and detectFormat tests to the browser e2e suite.
Part of #319
* fix(e2e): update browser test spec to match wasm-bindgen test suite
Replace streaming tests (removed with WASI migration) with lz4,
crc32, and detectFormat assertions.
Part of #319
* chore: add changeset for v1.0.0 major release (#322)
Closes #321
* chore(release): version packages to v1.0.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: address post-release bugs in v1.0.0 (#327)
- Add missing zstdDecompressWithDictWithCapacityAsync to index.mjs
- Sync crates/core-lib and crates/wasm version to 1.0.0
- Fix sync-cargo-version.js to cover all three Cargo.toml files
- Add missing async functions to README API list
- Add missing ESM smoke test assertion
Closes #326
* chore: add changeset for v1.0.1 patch release (#328)
Covers the bug fixes from #327:
- Missing zstdDecompressWithDictWithCapacityAsync ESM export
- Version mismatch in core-lib and wasm Cargo.toml
- Missing async functions in README API list
* chore(release): version packages to v1.0.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
derodero24
added a commit
that referenced
this pull request
Apr 4, 2026
* Set up napi-rs build pipeline (#9)
* chore(core): add Rust crate skeleton with napi-rs bindings
Minimal `zflate` crate with a single `version()` export to verify the
napi-rs build pipeline works end-to-end.
- crates/core/Cargo.toml: cdylib crate with napi-rs v3
- crates/core/build.rs: napi_build setup
- crates/core/src/lib.rs: version() function
* chore: add package.json with napi-rs configuration
Configure the npm package with 9 build targets (macOS, Linux, Windows,
WASM), dev dependencies for tooling (Biome, Vitest, TypeScript,
Changesets, CodSpeed), and build/test/lint scripts.
* chore: add build scripts and module entry points
- scripts/optimize-wasm.js: post-process WASM with wasm-opt
- index.mjs / index.d.mts: ESM entry points
- index.js / index.d.ts / browser.js: napi-rs generated files (force-tracked)
* Add CI workflow (#10)
* test: add initial test suite and ESM smoke test
- __test__/index.spec.ts: Vitest spec verifying version() export
- __test__/esm-import.mjs: ESM import smoke test for CI
* chore: update biome and tsconfig for test files
- biome.json: allow console in ESM smoke test
- tsconfig.json: format with Biome
* ci: add CI workflow
Full CI pipeline adapted from rapid-fuzzy:
- Lint (Biome, TypeScript, publint, generated file verification)
- Dependency audit (npm + cargo-deny)
- Rust lint (fmt + clippy) and test
- Cross-platform build (9 targets: macOS, Linux, Windows, WASM)
- Node.js test matrix (3 versions × 3 OSes)
- Coverage (Rust via cargo-llvm-cov + JS via Vitest, uploaded to Codecov)
* Add release and automation workflows (#11)
* ci: add release workflow with Cargo.toml version sync
Changesets-driven release flow:
- On develop push: create version PR, then release PR (develop → main)
- On main push: build all 9 targets and publish to npm with provenance
- scripts/sync-cargo-version.js: keep Cargo.toml version in sync with
package.json after changeset version bumps
* ci: add CodeQL security analysis
Analyze JavaScript/TypeScript and Rust code for security vulnerabilities.
Runs on PRs, develop pushes, and weekly schedule (Monday 6 AM UTC).
* ci: add CodSpeed benchmark workflow
Continuous performance tracking via CodSpeed. Runs JS benchmarks on
develop pushes and PRs when crate or bench files change.
Rust benchmarks will be added when the bench crate is introduced.
* ci: add self-hosted Renovate workflow
Automated dependency updates via self-hosted Renovate bot.
Runs weekly on Sunday 19:00 UTC (Monday 4:00 JST).
* test(bench): add initial benchmark file
Minimal benchmark to satisfy CodSpeed workflow. Real compression
benchmarks will be added alongside algorithm implementations.
* Implement zstd compression and decompression (#12)
* feat(zstd): implement zstd compression and decompression
Add zstdCompress() and zstdDecompress() functions powered by the zstd
C library via the zstd Rust crate.
- Compression levels 1-22 (default: 3)
- Auto-detection of frame content size for efficient decompression
- zstdDecompressWithCapacity() for data exceeding 256 MB default limit
- Rust unit tests for round-trip, empty data, large data, and levels
* test(zstd): add JS tests and benchmarks for zstd
- __test__/zstd.spec.ts: 11 tests covering round-trip, levels, errors
- __test__/esm-import.mjs: updated with zstd round-trip smoke test
- __test__/index.bench.ts: compress/decompress benchmarks at 150B/10KB/1MB
- index.mjs: export zstd functions
* chore: update napi-rs generated files and add changeset
- Regenerate index.js, index.d.ts, browser.js with zstd exports
- Add changeset for minor version bump
* ci: add WASI SDK setup for zstd WASM builds
The zstd crate compiles C code via zstd-sys, which requires a proper
WASI sysroot when targeting wasm32-wasip1-threads. Without it, clang
picks up system glibc headers and fails with missing
'bits/libc-header-start.h'.
Use bytecodealliance/actions/wasi-sdk/setup to install WASI SDK and
set CC, AR, CFLAGS, and WASI_SYSROOT environment variables for the
WASM build step in both CI and release workflows.
* Fix CodeRabbit review findings from #12 (#22)
* fix(zstd): validate capacity parameter in zstdDecompressWithCapacity
Reject negative, NaN, and Infinity values before casting f64 to usize.
Without validation, a negative f64 wraps to a very large usize, which
could cause memory exhaustion.
Closes #17
* fix(ci): use RUNNER_TEMP for WASI SDK installation
Replace /opt with $RUNNER_TEMP for WASI SDK extraction. /opt may not
be writable on all GitHub Actions runner configurations.
Also switch curl from -sL to -fsSL to fail on HTTP errors.
Closes #18
* docs: add SECURITY.md (#23)
* docs: add SECURITY.md
* docs: address review feedback on SECURITY.md
- Replace vague "extremely large size" with "massive size" for clarity
- Unify SLA day units by removing "business" qualifier from initial
assessment timeline for consistency
* docs: add CONTRIBUTING.md (#24)
* test(zstd): expand test coverage and benchmarks (#25)
* test(zstd): expand test coverage and benchmarks
Add comprehensive JS tests covering compression level boundaries (0,
negative, 22), decompressWithCapacity edge cases (insufficient, oversized,
invalid inputs), and round-trip verification with random data.
Expand benchmarks with random (incompressible) and realistic (JSON, text)
data variants alongside existing patterned data.
Update Rust doc comments to document negative compression levels and
level 0 behavior. Add Rust tests for boundary levels. Update changeset
to include all public API functions.
Closes #19
* chore: update napi-rs generated type definitions
Regenerate index.d.ts to reflect updated doc comments for
zstd_compress documenting negative levels and level 0 behavior.
* feat(zstd): add streaming compression/decompression API (#26)
Implement streaming zstd compression and decompression using the Web
Streams API (TransformStream). The Rust layer exposes ZstdCompressContext
and ZstdDecompressContext classes with transform/flush/finish methods
using zstd::stream::raw for stateful chunked processing.
The JS layer wraps these in TransformStream via createZstdCompressStream()
and createZstdDecompressStream(), available from both ESM and CJS entry
points. Streaming output is fully interoperable with one-shot APIs.
Add comprehensive tests covering chunked round-trips, various chunk
sizes, large data (1MB), random data, and one-shot/streaming interop.
Closes #5
* docs: add README and verify npm package readiness (#28)
* docs: add README and verify npm package readiness
Add comprehensive README with project description, installation guide,
quick-start examples (one-shot and streaming), full API reference,
supported algorithms table, platform compatibility matrix, and
benchmark results.
Verify npm package with publint (no errors). Package includes all
required files: index.js, index.mjs, index.d.ts, index.d.mts,
streams.js, streams.d.ts, browser.js, LICENSE, and README.md.
Closes #8
* docs(readme): address review feedback
- Add bun installation instructions
- Replace streaming example with self-contained runnable snippet
* fix(ci): resolve release workflow failures (#29)
* fix(ci): resolve release workflow failures
- Generate npm platform package directories for all 9 targets
- Add optionalDependencies to root package.json for platform binaries
- Fix changeset version command argument parsing in release workflow
by using a dedicated npm script instead of inline shell command
Closes #27
* fix(ci): remove optionalDependencies from committed package.json
The optionalDependencies for platform-specific packages are added
at publish time by the prepublishOnly hook (napi prepublish -t npm),
not in the committed source. Having them in package.json breaks
pnpm install --frozen-lockfile since the packages are not yet on npm.
* chore(release): version packages (#30)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(ci): regenerate napi bindings for v0.2.0 and fix version workflow (#34)
- Regenerate index.js with v0.2.0 version strings
- Update npm platform package versions to 0.2.0
- Add Rust toolchain to version job for napi build during version bump
- Add napi version, build, and format to changeset:version script
- Fix version test to read from package.json dynamically
* feat(gzip): add gzip and deflate compression support (#33)
Add gzip and raw deflate compression/decompression with one-shot and
streaming APIs. Includes interoperability tests with Node.js built-in
zlib module.
Closes #6
* feat(brotli): add brotli compression support (#32)
* feat(brotli): add brotli compression support
Add brotli compression/decompression with one-shot and streaming APIs.
Quality levels 0-11 (default: 6). Includes interoperability tests with
Node.js built-in zlib brotli module.
Closes #7
* test(brotli): add brotli compression benchmarks
Add brotli compress/decompress benchmarks matching the zstd benchmark
structure for patterned, random, and realistic data at various sizes.
* test(gzip): increase timeout for 1MB round-trip tests
Extend timeout from 5s to 30s for gzip and deflate 1MB data tests
to prevent CI failures on slower runners.
* test: increase timeout for all 1MB tests
Set 30s timeout on all 1MB data round-trip and streaming tests to
prevent failures on slower CI runners (Windows).
* docs(readme): update API reference for gzip and brotli (#36)
Add gzip, deflate, and brotli to the API reference tables. Update
supported algorithms table to reflect all three algorithms are now
available with both one-shot and streaming APIs.
* chore(deps): update taiki-e/install-action digest to c12d62a (#20)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(exports): add ./streams subpath export to package.json (#58)
Add a `./streams` subpath export so users can import streaming APIs
directly via `import { ... } from 'zflate/streams'`.
Closes #48
* fix(gzip): add compression level validation for gzip and deflate (#60)
Validate that the compression level parameter is within the valid range
(0-9) for gzip_compress, deflate_compress, GzipCompressContext, and
DeflateCompressContext. Return InvalidArg error if out of range, matching
the existing validation pattern in brotli_compress.
Closes #38
* fix(gzip): add decompression size limits to gzip and deflate (#64)
* fix(gzip): add decompression size limits to gzip and deflate
Add MAX_DECOMPRESSED_SIZE (256 MB) to gzip_decompress and
deflate_decompress to prevent zip bomb attacks. Both functions now use
chunked reading with size checking instead of unbounded read_to_end().
Add gzipDecompressWithCapacity and deflateDecompressWithCapacity for
explicit control over the maximum decompressed size, matching the
existing zstd and brotli patterns.
Closes #37
* chore(napi): regenerate bindings for new WithCapacity functions
* feat(api): accept Uint8Array input in all compression functions (#69)
Change all input parameters from Buffer to Either<Buffer, Uint8Array>
across all one-shot and streaming functions. This generates
Buffer | Uint8Array TypeScript types, enabling browser/Deno/Bun users
to pass Uint8Array directly without type errors.
Return types remain Buffer. No runtime behavior changes for existing
Buffer usage.
Closes #42
* docs: enhance README with examples, comparison table, and migration guides (#63)
* test: add cargo-fuzz targets for decompression fuzzing (#71)
* test: add edge case and error handling tests (#65)
* test: expand Node.js zlib interop tests to brotli and zstd (#67)
* ci: add hand-written JS files to workflow path filters (#56)
* ci(codeql): suppress rust/access-invalid-pointer false positives (#57)
* fix(dx): add ./streams subpath export to package.json (#59)
* ci: add cargo-audit to dependency audit pipeline (#61)
* feat(core): add unified auto-detect decompression API (#72)
Add decompress() function that auto-detects compression format from
magic bytes (zstd: 0xFD2FB528, gzip: 0x1F8B) and falls back to brotli
heuristic detection. Add detectFormat() for format identification
without decompression.
Supported formats: zstd, gzip, brotli. Raw deflate cannot be
auto-detected due to lack of magic bytes.
Closes #16
* ci(dx): add @arethetypeswrong/cli type validation (#62)
* perf(bench): add competitive benchmarks across all algorithms (#70)
* perf(wasm): add WASM binary size reporting to CI (#66)
* fix(exports): add missing ESM exports for new functions (#74)
Add decompress, detectFormat, gzipDecompressWithCapacity, and
deflateDecompressWithCapacity to index.mjs ESM export list. These
functions were available via CJS but missing from ESM imports.
Update ESM smoke test to cover all WithCapacity functions, decompress,
and detectFormat.
Closes #73
* feat(node): add Node.js stream.Transform compatibility layer (#76)
Add `zflate/node` subpath export with stream.Transform factories for all
compression algorithms (zstd, gzip, deflate, brotli), enabling use with
Node.js stream.pipeline() and pipe-based workflows.
Closes #43
* fix(brotli): use consistent error message for capacity exceeded (#78)
Change brotli_decompress_with_capacity error message from
"Destination buffer is too small" to "brotli decompress exceeded
maximum size of {} bytes" to match the pattern used by all other
decompress functions.
Closes #77
* fix(docs): remove merge conflict marker and stale src/ references (#80)
* feat(core): add async compression/decompression API (#82)
Add async versions of all one-shot compression/decompression functions
using napi-rs AsyncTask with the Task trait. This runs compression on
the libuv thread pool without blocking the Node.js event loop.
New functions: zstdCompressAsync, zstdDecompressAsync,
gzipCompressAsync, gzipDecompressAsync, deflateCompressAsync,
deflateDecompressAsync, brotliCompressAsync, brotliDecompressAsync.
Closes #40
* feat(zstd): add dictionary compression API (#83)
Add zstd dictionary training, compression, and decompression functions:
- zstdTrainDictionary: train a dictionary from sample data
- zstdCompressWithDict: compress with a pre-trained dictionary
- zstdDecompressWithDict: decompress data compressed with a dictionary
Enable the zdict_builder feature for the zstd crate to support
dictionary training. Include Rust unit tests and Vitest integration
tests verifying round-trip correctness and improved compression ratios
for small, similar data.
Closes #41
* docs: remove Cloudflare Workers from supported platforms (#91)
* ci: include Node.js WASM loader files in CI artifact (#90)
* test(wasm): add Node.js WASM compatibility test with native parity verification (#92)
* refactor(core): introduce thiserror for structured error types (#93)
Replace manual `Error::new(Status::..., ...)` calls with structured
`ZflateError` enum variants across all compression modules. Error
messages remain identical to preserve backward compatibility.
Closes #51
* test(deno): add Deno WASM smoke test (#96)
* test(bun): add Bun WASM smoke test (#95)
* test(browser): add Playwright browser WASM test (#94)
* ci(security): tighten cargo-deny bans and sources to deny mode (#108)
* ci: pin dtolnay/rust-toolchain to SHA digest (#107)
* fix(core): add decompression size limits to streaming contexts (#109)
Add cumulative output size tracking to all streaming decompression
contexts (ZstdDecompressContext, GzipDecompressContext,
DeflateDecompressContext, BrotliDecompressContext).
Each context now enforces MAX_DECOMPRESSED_SIZE (256 MB) across
transform() calls, preventing decompression bomb attacks that could
cause unbounded memory growth.
Closes #97
* feat(core): narrow detectFormat return type to string literal union (#111)
Use napi-rs string_enum to return CompressionFormat instead of plain
string, generating a const enum with 'zstd' | 'gzip' | 'brotli' |
'unknown' values in TypeScript.
Closes #103
* perf(core): optimize buffer allocation in compression and decompression (#110)
* perf(core): optimize buffer allocation in compression and decompression
Increase decompression read buffer from 4KB to 32KB in gzip, brotli,
and brotli_stream modules to reduce syscall overhead during chunked
reads. Pre-allocate compression output Vecs with input.len() capacity
for gzip and deflate encoders (both sync and async) to avoid repeated
reallocations during compression.
Closes #100
* revert: keep BUFFER_SIZE at 4096 to avoid decompress regression
The 32KB buffer caused 21 benchmark regressions in CodSpeed due to
increased stack zeroing cost. Revert to 4KB. The Vec::with_capacity
optimization for compression output is retained.
* perf(core): replace Vec clone and split_off with std::mem::take in streaming contexts (#112)
Use std::mem::take() instead of .get_ref().clone() + .get_mut().clear()
in brotli streaming contexts, and instead of .split_off(0) in gzip/deflate
streaming contexts. This avoids unnecessary memory allocation and copying
by moving ownership of the buffer contents in-place.
Closes #99
* fix(zstd): add compression level validation (#113)
Validate that the compression level is within the valid zstd range
(-131072 to 22) before passing it to the zstd library. This provides
clear error messages instead of opaque library errors.
Validation is added to:
- zstd_compress (sync)
- zstd_compress_async
- ZstdCompressContext::new (streaming)
- zstd_compress_with_dict
Closes #81
* feat(zstd): add streaming dictionary compression/decompression (#114)
Add ZstdCompressDictContext and ZstdDecompressDictContext classes that
accept a pre-trained dictionary for streaming compression/decompression.
Include Web Streams (createZstdCompressDictStream/createZstdDecompressDictStream)
and Node.js Transform (createZstdCompressDictTransform/createZstdDecompressDictTransform)
wrappers, type definitions, ESM exports, and streaming dict round-trip tests.
Closes #104
* perf(core): switch flate2 backend from miniz_oxide to zlib-rs (#115)
Replace the `rust_backend` (miniz_oxide) feature with `zlib-rs` for
2-3x faster gzip/deflate compression. zlib-rs is a pure-Rust
implementation that works on all targets including WASM, is ISRG
audited, and is on track to become flate2's default backend.
Relax compression level size ordering test since zlib-rs may produce
slightly different output sizes per level than miniz_oxide — round-trip
correctness is what matters.
Closes #98
* perf(zstd): enable multi-threaded compression for native targets (#116)
Add `zstdmt` feature for non-WASM targets via conditional dependency.
This enables zstd's internal thread pool for compression, improving
performance by 20-80% for large inputs on multi-core systems.
WASM targets are excluded because wasi-threads is experimental and
the interaction with napi-rs worker-based threading is untested.
Closes #102
* docs: comprehensive README update with missing APIs and SVG benchmark (#118)
* fix(docs): correct benchmark environment label to Apple M2 / Node.js v22 (#119)
* perf(wasm): add --converge flag to wasm-opt optimization (#122)
Re-run optimization passes until no further gains are achieved.
Increase timeout to 5 minutes to accommodate additional passes.
Closes #121
* perf(core): use 4x input size for decompression output pre-allocation (#123)
Compressed data typically decompresses to several times its original
size. Pre-allocating with input.len() alone leads to frequent
reallocations during decompression. Using input.len() * 4 as the
initial capacity provides a better heuristic while the existing
.min() caps continue to prevent excessive allocation.
Closes #120
* ci: migrate release workflow to npm trusted publishing (OIDC) (#68)
* ci: migrate release workflow to npm trusted publishing (OIDC)
Remove NPM_TOKEN/NODE_AUTH_TOKEN secret dependency and rely on
OIDC token exchange via id-token:write permission (already set).
Trusted publishing must be configured on npmjs.com for each package
before the next release.
* fix(ci): restore pinned SHA for dtolnay/rust-toolchain
Keep the pinned commit hash consistent with ci.yml and codspeed.yml
for supply chain security. The rust-toolchain change was out of scope
for the trusted publishing migration.
* fix(gzip): support concatenated gzip streams in decompression (#131)
Replace GzDecoder with MultiGzDecoder from flate2 to correctly handle
concatenated gzip members in both one-shot and streaming decompression.
* fix(streaming): add finish-guard state tracking for brotli and zstd compress contexts (#132)
* feat(core): add missing async variants for one-shot APIs (#133)
* feat(streaming): add auto-detect decompression streams (#145)
* docs: add troubleshooting section to CONTRIBUTING.md (#146)
* docs: add CODE_OF_CONDUCT.md (#149)
* ci: apply least-privilege permissions to workflows (#147)
* chore: improve package.json metadata and exports map (#148)
* ci: clean up CI workflow configuration (#150)
* feat(core): expose CRC32 utility function (#134)
* feat(core): expose CRC32 utility function
* test(crc32): use skipIf for conditional Node.js zlib interop test
* ci(testing): add coverage thresholds to vitest config (#151)
* ci: add aggregate gate job to CI workflow (#152)
* feat(streaming): add configurable maxOutputSize to decompression contexts (#153)
Add optional maxOutputSize parameter to all streaming decompression
context constructors (GzipDecompressContext, DeflateDecompressContext,
ZstdDecompressContext, BrotliDecompressContext, ZstdDecompressDictContext).
When not specified, the default 256 MB limit is used.
The parameter is also passed through all streaming factory functions
(createXxxDecompressStream, createXxxDecompressTransform, and
createDecompressStream/createDecompressTransform).
Closes #129
* ci: add environment protection for npm publish (#154)
* ci: add cargo-shear for unused dependency detection (#155)
* ci: add WASM binary size tracking on pull requests (#156)
* ci: add scripts/ to CI workflow path triggers (#159)
* fix(wasm): add JS-side streaming adapters for browser WASM environment (#158)
Replace native streaming contexts with JS-side adapters in the browser
entry point to work around WebAssembly.Memory growth invalidating
ArrayBuffer views. Adapters accumulate chunks and delegate to one-shot
APIs, maintaining the same transform()/flush()/finish() API surface.
- browser-streaming.js: adapter classes for all 10 streaming contexts
- browser-entry.js: re-exports WASM APIs with adapter overrides
- package.json: browser condition points to browser-entry.js
- e2e: streaming tests enabled for browser WASM
* feat(gzip): add header metadata support for compression and reading (#160)
Closes #130
* chore(renovate): run daily and remove PR concurrent limit (#164)
* chore(renovate): run daily and remove PR concurrent limit
Closes #163
* chore(renovate): disable lock file maintenance
* chore(deps): update codspeedhq/action digest to 1c8ae48 (#161)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dtolnay/rust-toolchain digest to 631a55b (#162)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to 3869755 (#165)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to b988c18 (#166)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(bench): exclude third-party benchmarks from CodSpeed tracking (#169)
* fix(bench): exclude third-party benchmarks from CodSpeed tracking
Split benchmark files into zflate-only (*.bench.ts) and comparison
(*.compare.bench.ts) variants. CodSpeed now only tracks zflate's own
benchmarks, eliminating false regressions from pako/fflate/node:zlib
that are caused by GitHub Actions CPU heterogeneity affecting Callgrind
instruction counting.
Closes #167
* fix(bench): use deterministic random data in CodSpeed benchmarks
Replace non-deterministic randomBytes() with a seeded LCG to ensure
reproducible benchmark inputs across runs, reducing CodSpeed instruction
count variance.
* feat(bench): migrate from CodSpeed to Bencher.dev for regression detection (#170)
Replace CodSpeed simulation mode (Callgrind instruction counting) with
Bencher.dev (statistical wall-clock analysis). CodSpeed's instruction
counting was fundamentally incompatible with this napi-rs project due to
GitHub Actions CPU heterogeneity and glibc's CPU-adaptive malloc.
Changes:
- Add Bencher workflow with t-test threshold for regression detection
- Add vitest-to-bmf.js conversion script for vitest bench output
- Remove CodSpeed workflow and @codspeed/vitest-plugin dependency
- Remove CodSpeed plugin from vitest config
Closes #168
* feat(bench): switch to CodSpeed Walltime mode for accurate benchmarks (#172)
Replace Bencher.dev with CodSpeed Walltime mode (Macro Runners) which
runs benchmarks on dedicated bare-metal hardware, eliminating GitHub
Actions CPU variance. CodSpeed also provides native vitest integration
via @codspeed/vitest-plugin, removing the need for conversion scripts.
Key change: `mode: walltime` instead of the previous `mode: simulation`
which used Callgrind instruction counting.
Closes #171
* refactor(bench): split benchmark files per algorithm for granular CodSpeed reporting (#174)
Split index.bench.ts (mixed zstd+brotli) into separate zstd.bench.ts
and brotli.bench.ts so CodSpeed PR reports only show benchmarks relevant
to the changed algorithm. Extract shared test fixtures into
bench-fixtures.ts and add realistic data benchmarks (JSON, text) to
gzip and deflate for consistent coverage across all four algorithms.
Closes #173
* perf(core): eliminate double-copy buffer pattern in decompression (#178)
* perf(core): eliminate double-copy buffer pattern in decompression
Replace the manual 4 KiB read loop (stack buffer → Vec copy) with
read_to_end + Take, which writes directly into Vec spare capacity.
This eliminates one full copy of every decompressed byte and uses
adaptive buffer sizing instead of fixed 4 KiB chunks.
Affected: gzip, deflate, brotli (sync + async, default + with-capacity)
Closes #175
* fix(core): use saturating_mul for init capacity and update size limit tests
Address CodeRabbit review feedback:
- Replace len() * 4 with len().saturating_mul(4) across all call sites
to prevent overflow on 32-bit targets (wasm32-wasip1-threads)
- Replace old chunk-loop size limit tests with Take+read_to_end tests
that exercise the actual decompression pattern (avoiding napi::Error
symbol resolution issues in standalone test binaries)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks (#179)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks
- Add bench:ci script that excludes comparison benchmarks
- Simplify CodSpeed workflow to use bench:ci instead of explicit file list
- Use glob pattern for bench file path triggers
- Add crates/bench workspace member with Criterion benchmarks
(gzip, deflate, brotli, zstd) using codspeed-criterion-compat
- Add rust-benchmarks job to CodSpeed workflow (simulation mode)
Closes #177
* fix(ci): add Cargo.toml, Cargo.lock, package.json to CodSpeed path triggers
Ensures CodSpeed benchmarks run when workspace config or dependency
lockfiles change, which could affect benchmark behavior.
* feat(core): add LZ4 frame compression support (#180)
* feat(core): add LZ4 frame compression support
Add LZ4 as a fourth compression algorithm using lz4_flex (pure Rust):
- Bulk API: lz4Compress, lz4Decompress, lz4DecompressWithCapacity
- Async API: lz4CompressAsync, lz4DecompressAsync, lz4DecompressWithCapacityAsync
- Streaming: Lz4CompressContext, Lz4DecompressContext
- Web Streams: createLz4CompressStream, createLz4DecompressStream
- Format detection: detectFormat returns "lz4" for LZ4 frame magic bytes
- Auto-decompress: decompress/decompressAsync and createDecompressStream
now handle LZ4 data automatically
Closes #176
* fix(core): add Read import to LZ4 test modules
Tests use read_to_end which requires the Read trait in scope.
* refactor(core): validate capacity as non-negative integer (#183)
* refactor(core): validate capacity as non-negative integer across all algorithms
Extract shared validate_capacity() that rejects NaN, Infinity, negative,
fractional values, and values exceeding usize::MAX. Applied to all 10
DecompressWithCapacity functions (sync + async × 5 algorithms).
Closes #181
* test: add regression tests for fractional and oversized capacity
* perf(lz4): implement incremental streaming for LZ4 compression (#184)
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with cursor-based incremental output in
Lz4CompressContext. The FrameEncoder now persists across transform()
calls, emitting compressed blocks as they become available instead of
buffering all input until finish().
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with FrameEncoder-based incremental output
in Lz4CompressContext. Compressed blocks are emitted as internal buffers
fill, and old bytes are drained to bound memory usage.
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* chore(deps): update taiki-e/install-action digest to 0620367 (#185)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust crate lz4_flex to 0.13 (#186)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore: rename package from zflate to comprs (#190)
* chore: rename package from zflate to comprs
Rename npm package, Rust crates, napi binary name, platform packages,
and all references from "zflate" to "comprs". The name "comprs" conveys
"compress" with a Rust flavor, is available on npm, and avoids the
typosquat protection that blocks "zflate".
Changes across 70 files:
- npm: comprs, comprs-darwin-arm64, comprs-wasm32-wasi, etc.
- Rust: comprs (core), comprs-bench
- napi binaryName: comprs (→ comprs.*.node, comprs.*.wasm)
- README, workflows, changesets, tests, benchmarks
GitHub repository URL (derodero24/zflate) is preserved.
Closes #189
* chore: update GitHub repo URLs to derodero24/comprs
Update all repository references to match the planned GitHub repo
rename from derodero24/zflate to derodero24/comprs.
* chore(core): rename ZflateError to ComprsError and update bench vars (#192)
Rename internal Rust error type `ZflateError` → `ComprsError` across
all source files and update benchmark variable names `*_ZFLATE` →
`*_COMPRS` for consistency with the package rename in #190.
Closes #191
* fix: add missing LZ4 docs, browser exports, Node.js transforms, and publish files (#194)
- Add LZ4 one-shot and streaming context exports to browser-entry.js
- Add Lz4CompressContext/Lz4DecompressContext adapters to browser-streaming.js
- Add createLz4CompressTransform/createLz4DecompressTransform to node.js
- Add LZ4 type declarations to node.d.ts
- Add 'lz4' case to createDecompressContext in node.js
- Add browser-entry.js and browser-streaming.js to package.json files array
- Document LZ4 API in README.md (one-shot, async, streaming, comparison)
- Fix streaming import path in README.md (comprs → comprs/streams)
- Update auto-detect descriptions to include LZ4
- Add "lz4" to package.json keywords and description
- Add LZ4 changeset for upcoming release
Closes #193
* fix: add missing LZ4 exports to ESM entry point and complete API docs (#196)
- Add LZ4 one-shot APIs, Context classes, and streaming functions to
index.mjs (ESM users could not access LZ4 at all)
- Add createLz4CompressStream/createLz4DecompressStream to index.mjs
- Add gzipCompressWithHeader and gzipReadHeader to browser-entry.js
- Document gzipCompressWithHeader, gzipReadHeader in README
- Document crc32 and version() utilities in README
- Document createDecompressStream and createDecompressTransform in README
Closes #195
* fix(docs): add missing lz4 to decompressAsync doc and clarify async section (#198)
The decompressAsync Rust doc comment listed only zstd, gzip, brotli as
supported formats, but the implementation also handles lz4. The README
async section claimed "all one-shot functions" have async variants, which
is inaccurate for utility functions like crc32, version, gzipReadHeader.
Closes #197
* chore(release): version packages (#35)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* perf(ci): remove JS walltime benchmarks from CodSpeed regression detection (#201)
JS walltime benchmarks on standard GitHub Actions runners produce 20-50%
variance due to CPU heterogeneity (AMD EPYC vs Intel Xeon assigned randomly)
and glibc's CPU-adaptive malloc. This caused CodSpeed Performance Analysis to
fail on nearly every PR since #174, including documentation-only changes.
Keep only Rust simulation benchmarks in CodSpeed, which use instruction
counting via Callgrind and produce <1% variance. Also narrow the workflow
path triggers to Rust-only files since JS changes no longer affect the
CodSpeed pipeline.
JS benchmarks remain available locally via `pnpm run bench`.
Closes #200
* ci(release): use granular access token for initial npm publish
The first publish requires an NPM_TOKEN since OIDC trusted publishing
only works for packages that already exist on the registry. After the
initial release, this will be switched back to OIDC.
* chore: add .mcp.json to .gitignore (#205)
Closes #203
* fix(core): use decompress_with_limit in async auto-detect decompression (#216)
Replace manual chunk-read loops with double-copy in DecompressTask for
gzip and brotli branches. Now uses crate::decompress_with_limit(),
matching the pattern already used by the LZ4 branch and per-algorithm
async tasks.
Closes #207
* ci: add browser files to CI path triggers (#217)
Closes #212
* fix(zstd): validate maxDictSize parameter in dictionary training (#219)
* test: update ESM smoke test to cover all exports (#215)
* Add LZ4 edge-case and streaming test coverage (#218)
* test(lz4): add edge-case and streaming test coverage
* test(lz4): fix empty buffer test to match actual lz4 behavior
LZ4 returns empty buffer for empty input (like zstd and deflate),
rather than throwing an error (like gzip and brotli).
* fix(zstd): enforce maxOutputSize in browser dict decompression (#221)
* fix(zstd): enforce maxOutputSize in browser dict decompression
Add zstd_decompress_with_dict_with_capacity Rust function and use it
in browser-streaming.js ZstdDecompressDictContext.flush() when
maxOutputSize is provided.
Closes #208
* chore: regenerate napi-rs binding files
* refactor(core): consolidate MAX_DECOMPRESSED_SIZE constant (#222)
Remove duplicated MAX_DECOMPRESSED_SIZE definitions from zstd, gzip,
brotli_impl, lz4, and detect modules. Change the lib.rs definition to
pub(crate) and reference it via crate::MAX_DECOMPRESSED_SIZE in all
modules.
* chore(release): version packages (#220)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* docs: add benchmark comparison results to README (#224)
Closes #213
* feat(brotli): add dictionary compression and decompression support (#225)
* docs: add brotli dictionary to README and update Node.js transforms (#228)
* docs: readme visual and structural overhaul (#239)
- Center-align hero section with title, tagline, and badges
- Add Node.js version and TypeScript badges
- Add Table of Contents for navigation
- Wrap benchmark tables in collapsible <details> sections
- Replace plain blockquote with GitHub Admonition (> [!TIP])
- Replace raw build target triples with human-readable table
Closes #229
* docs: reposition and enhance comparison table (#240)
- Move comparison table from bottom of README to right after "Why comprs?"
- Split "Streaming" into "Web Streams API" and "Node.js Transform" rows
- Replace ambiguous `✅*` with `⚠️ Experimental*` for node:zlib zstd
- Replace footnote `†` with inline "Chunked†" for pako streaming
- Update Table of Contents to reflect new section order
- Center-align feature columns for better scannability
Closes #236
* docs: improve quick start examples and add runtime-specific usage (#241)
- Condense one-shot examples to single primary zstd block + multi-algorithm block
- Replace impractical manual ReadableStream example with fetch-style streaming
- Add Node.js Transform stream example to Quick Start
- Add brief comment explaining async runs on libuv thread pool
- Consolidate compression levels into compact format
- Add Deno and Bun import examples
Closes #235
* docs: add algorithm selection guide (#242)
- Add "Choosing an Algorithm" section with use case → recommendation table
- Add "Choosing an API mode" subsection for sync/async/streaming/dictionary
- Update Table of Contents
Closes #234
* docs: document browser WASM requirements (SharedArrayBuffer, COOP/COEP) (#243)
- Add SharedArrayBuffer requirement warning with required HTTP headers
- Remove misleading "or import directly from a CDN" claim
- Link to MDN SharedArrayBuffer security requirements
Closes #231
* docs: add SSR framework integration guide (#244)
- Add "Framework Integration (SSR)" subsection under Browser Usage
- Include Next.js serverExternalPackages and Vite SSR external config
- Note automatic WASM fallback on client side
Closes #232
* docs: add fflate migration guide (#245)
- Add "From fflate" migration section with diff example
- Note comprs advantages over fflate (zstd, lz4, brotli, dictionaries, Web Streams)
Closes #237
* docs: add known limitations and performance notes (#246)
- Add "Notes" section with three admonitions covering:
- 256 MB default decompression limit with WithCapacity() workaround
- Small payload WASM overhead advisory
- Brotli decompression performance nuance vs node:zlib
Closes #238
* docs: add all undocumented async and capacity API variants to readme (#247)
- Add zstdDecompressWithDictWithCapacity to sync zstd dictionary table
- Add 9 missing async variants: WithCapacityAsync for all algorithms,
zstd dict async, zstdTrainDictionaryAsync, decompressAsync
- Reorder async table to group by algorithm with capacity/dict variants
Closes #233
* docs: add benchmark comparison SVG charts for cross-algorithm and brotli (#248)
- Add cross-algorithm compression chart (zstd/lz4/gzip/brotli at 10KB)
- Add brotli compression speedup chart (134.7x faster than node:zlib)
- Embed both charts alongside existing gzip chart in Benchmarks section
Closes #230
* docs: fold repetitive API tables into collapsible sections (#250)
- Wrap Dictionary API (zstd + brotli) in <details>
- Replace Async table header with pattern explanation + code example,
fold full function list into <details>
- Add Streaming import path + fold table into <details>
- Keep Node.js Transform code example visible, fold table into <details>
- One-shot core API tables remain visible as primary reference
Closes #249
* chore(deps): update taiki-e/install-action digest to 7627fb4 (#251)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.9 (#252)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update renovatebot/github-action action to v46.1.6 (#254)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency vite to v8.0.3 (#253)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update pnpm to v10.33.0 (#256)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.1 (#255)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency typescript to v6 (#257)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* feat: add interactive playground with GitHub Pages deployment (#259)
Add a browser-based playground for trying comprs compression live.
Includes four algorithms (zstd, gzip, brotli, lz4), level slider,
comparison table, file upload, and download. Deploys to GitHub Pages
via new CI workflow. Uses a JS mock for local dev without a WASM build.
Closes #258
* docs: improve playground visibility and add OGP/SEO meta tags (#261)
Add prominent text link below badges in README header and playground
as a bullet in "Why comprs?". Add Open Graph, Twitter Card, and
canonical URL meta tags to playground/index.html for social sharing.
Closes #260
* fix(playground): handle COI not established on first visit (#263)
On first visit, the COI service worker has not yet reloaded the page,
so crossOriginIsolated is false. Calling the WASM function immediately
throws because SharedArrayBuffer is unavailable. This showed a
misleading "unsupported browser" error even in modern browsers like
Vivaldi, Brave, or Arc.
- Check crossOriginIsolated before invoking WASM
- Show "Enabling security features... reloading" while the service
worker handles the reload
- After 3 s, show a manual Reload button if auto-reload did not happen
- Include the actual error message when WASM init itself fails
- Add .loading-reload-btn styles to style.css
Closes #262
* fix(playground): add Buffer polyfill for emnapi WASM compatibility (#265)
The napi-rs WASM runtime (emnapi) requires globalThis.Buffer when
creating memory views for function arguments/return values. Browsers
do not define Buffer natively, causing zstdCompress and other WASM
calls to throw immediately.
Add the 'buffer' npm package (v6 browser polyfill) to playground
dependencies and set globalThis.Buffer before any WASM calls.
Closes #264
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init (#267)
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init
Static imports are hoisted and evaluated before any module body code runs.
This caused globalThis.Buffer to be unset when comprs-wasm32-wasi evaluated
__emnapiInstantiateNapiModuleSync (which calls emnapi_create_memory_view).
Switch to a dynamic import so the Buffer polyfill assignment executes first.
Closes #266
* fix(playground): allow Vite dev server to serve WASM from parent directory
When comprs.wasi-browser.js is present in the repo root, the WASM file
is fetched at runtime via /@fs/ from outside the playground directory.
Add server.fs.allow so Vite does not block the request with 403.
Only applied when a local WASM build is detected (hasLocalWasm=true).
* fix(playground): fix comparison table label direction and badge layout (#271)
- Rename bar chart column 'Ratio (smaller = better)' to 'Savings'
and numeric column 'Ratio' to 'Saved' to accurately reflect that
the metric is compression savings % (larger = better)
- Change 'best ratio' badge to 'best' (shorter, clearer)
- Add 'fastest' badge to the fastest algorithm row; remove the
duplicate 'Fastest:' text from the footer note
- Change badge column from fixed 72px to auto to prevent overflow
Closes #270
* fix(playground): fix negative ratio text, CSS variable, download label, and file size alert (#273)
- Show "X% larger" instead of "-X% smaller" when compressed output exceeds input
- Fix reload button using undefined var(--accent); use var(--text)/var(--bg) instead
- Add file extension to download button (e.g. "Download .zst") via #download-ext span
- Replace alert() for oversized file with inline error message that auto-dismisses after 5s
Closes #272
* chore(deps): update codecov/codecov-action digest to 75cd116 (#268)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.2 (#269)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(playground): add FileReader error handler and textarea aria-label (#275)
- Add reader.onerror handler to show inline error when file reading fails
instead of silently doing nothing (e.g. on disk error or revoked access)
- Add aria-label to the input textarea for screen reader accessibility
Closes #274
* chore(deps): update dtolnay/rust-toolchain digest to 29eef33 (#276)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to c10b806 (#277)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/configure-pages action to v6 (#283)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/deploy-pages action to v5 (#284)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/upload-pages-artifact action to v4 (#285)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to d858f81 (#278)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust-dependencies to v3.8.4 (#281)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.2 (#280)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.10 (#279)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update codecov/codecov-action action to v6 (#286)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 (#287)
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2
Update @napi-rs/cli from v3.5.1 to v3.6.0 along with @emnapi/core
and @emnapi/runtime from v1.9.1 to v1.9.2 to resolve emnapi version
mismatch that caused WASM build failures.
* fix(deps): align emnapi transitive dependency to v1.9.2
Update the emnapi package (transitive via @napi-rs/cli) from v1.9.1
to v1.9.2, matching @emnapi/core and @emnapi/runtime versions.
* chore(deps): update renovatebot/github-action action to v46.1.7 (#292)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @types/node to v24.12.2 (#291)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(config): update biome.json schema to v2.4.10 (#294)
* chore(deps): update dependency @playwright/test to v1.59.1 (#293)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(core): remove unused wildcard import in crc test module (#295)
* fix(ci): update setup-deno and changesets/action to resolve Renovate lookup failures (#296)
Pin to specific version tags instead of major version floating tags,
which do not exist in these repositories. This allows Renovate to
resolve digests for tracking updates.
- denoland/setup-deno: v2 → v2.0.4
- changesets/action: v1 → v1.7.0
Closes #290
* chore(release): version packages to v0.4.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* refactor: rename npm package to @derodero24/comprs (#298)
Rename all npm packages from `comprs-*` to `@derodero24/comprs-*` to
bypass npm's typosquatting protection that blocked the `comprs` name.
- Root package: comprs → @derodero24/comprs
- 9 platform packages: comprs-{platform} → @derodero24/comprs-{platform}
- Regenerated napi-rs loader (index.js, browser.js) with new names
- Updated release workflow version checks
- Updated README install/import examples
- Updated playground npm link
Closes #297
* fix(playground): update vite alias for renamed WASM package
Update the vite resolve alias from 'comprs-wasm32-wasi' to
'@derodero24/comprs-wasm32-wasi' to match the package rename in #298.
* fix(ci): add publishConfig and permissions for scoped npm publish (#302)
- Add publishConfig.access: "public" to root and all platform
package.json files so napi prepublish can publish scoped packages
- Add contents: write permission to Publish job for GitHub Release
creation by napi prepublish
- Add changeset for v0.4.1 patch release
Closes #301
* chore(release): version packages to v0.4.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: update stale package references after rename to @derodero24/comprs (#307)
Update remaining references from old unscoped names to scoped names:
- browser-streaming.js: comprs-wasm32-wasi → @derodero24/comprs-wasm32-wasi
- playground/main.js: import('comprs') → import('@derodero24/comprs')
- playground/package.json: comprs → @derodero24/comprs dependency
- playground/vite.config.js: Vite alias keys updated
This fixes the GitHub Pages deployment failure caused by Vite being
unable to resolve the old unscoped import in browser-streaming.js.
Closes #306
* fix(playground): regenerate lockfile after package rename
The playground's pnpm-lock.yaml was not updated when the dependency
was renamed from comprs to @derodero24/comprs, causing CI to fail
with frozen-lockfile mismatch.
* docs: fix README package references, streaming note, and WASM size (#311)
* fix(docs): update remaining old package name references in README
- Deno import: npm:comprs → npm:@derodero24/comprs
- Subpath imports: comprs/streams → @derodero24/comprs/streams,
comprs/node → @derodero24/comprs/node
- SSR config: Next.js and Vite external package names
Closes #308
* docs: clarify browser streaming memory behavior
Add note explaining that browser WASM streaming buffers all data in
memory (unlike Node.js which streams incrementally). Also qualify the
"bounded memory" claim in the feature list to specify Node.js.
Closes #309
* docs: add concrete WASM binary size to README
Show approximate optimized WASM size (~2.0 MB raw, ~800 KB gzipped)
directly in README so users can evaluate without navigating to CI.
Closes #310
* fix: update npm badges and sub-package descriptions (#317)
* fix(docs): update npm badges to use scoped package name
The npm version and downloads badges were pointing to the old unscoped
package name "comprs" instead of the scoped "@derodero24/comprs".
Closes #312
* fix: update platform sub-package descriptions to include lz4
All 9 platform sub-packages under npm/ had descriptions listing only
"zstd, gzip, and brotli" but were missing "lz4" which was added to
the root package. Updated to match: "zstd, gzip, brotli, and lz4".
Closes #315
* fix: add missing ESM export and test coverage (#318)
* fix: add missing zstdDecompressWithDictWithCapacity to ESM and browser exports
Add zstdDecompressWithDictWithCapacity to index.mjs destructured exports,
browser-entry.js one-shot re-exports, and ESM smoke test assertions.
Closes #313
* test: add coverage for WithDictWithCapacity functions and Node.js Transform streams
Add tests for zstdDecompressWithDictWithCapacity, brotliDecompressWithDictWithCapacity,
brotliDecompressWithDictWithCapacityAsync, LZ4 Node.js Transform round-trip,
zstd dict Node.js Transform round-trip, and brotli dict Node.js Transform round-trip.
Closes #314
* feat(wasm): replace WASI WASM with wasm-bindgen for browser compatibility (#320)
* refactor(core): extract pure Rust logic into core-lib crate
Split crates/core into two crates:
- crates/core-lib: pure Rust compression logic (no FFI dependencies)
- crates/core: napi-rs thin wrappers delegating to core-lib
This prepares for adding a wasm-bindgen crate that shares the same
core logic, enabling browser usage without SharedArrayBuffer.
Part of #319
* feat(wasm): add wasm-bindgen browser binding crate
Create crates/wasm with wasm-bindgen wrappers over comprs-core,
targeting wasm32-unknown-unknown (no SharedArrayBuffer required).
Includes one-shot and streaming APIs for all algorithms:
zstd, gzip, deflate, brotli, lz4, auto-detect, and crc32.
Part of #319
* feat(wasm): integrate wasm-bindgen build into package distribution
- Add build:wasm-bindgen script using wasm-pack
- Update browser.js to use wasm-bindgen output
- Update browser-streaming.js imports
- Add wasm-bindgen files to npm package
- Update CI with wasm-bindgen build job
- Update release workflow for wasm-bindgen artifacts
- Update playground configuration
- Remove SharedArrayBuffer requirement from README
Part of #319
* fix(ci): resolve CI failures for wasm-bindgen migration
- Remove browser.js from napi generated file diff check (napi build
regenerates it, but we now maintain it manually for wasm-bindgen)
- Remove unused direct compression deps from napi crate (all logic
now lives in core-lib)
- Update e2e browser test to use wasm-bindgen output instead of
WASI browser entry
- Remove COOP/COEP headers from e2e vite config (no longer needed)
Part of #319
* fix(e2e): use manual WASM instantiation for wasm-bindgen in browser test
Vite 8 does not support the TC39 WebAssembly ESM integration proposal.
Import _bg.js directly and instantiate WASM binary manually, following
the same pattern used in rapid-fuzzy.
Also add lz4, crc32, and detectFormat tests to the browser e2e suite.
Part of #319
* fix(e2e): update browser test spec to match wasm-bindgen test suite
Replace streaming tests (removed with WASI migration) with lz4,
crc32, and detectFormat assertions.
Part of #319
* chore: add changeset for v1.0.0 major release (#322)
Closes #321
* chore(release): version packages to v1.0.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: address post-release bugs in v1.0.0 (#327)
- Add missing zstdDecompressWithDictWithCapacityAsync to index.mjs
- Sync crates/core-lib and crates/wasm version to 1.0.0
- Fix sync-cargo-version.js to cover all three Cargo.toml files
- Add missing async functions to README API list
- Add missing ESM smoke test assertion
Closes #326
* chore: add changeset for v1.0.1 patch release (#328)
Covers the bug fixes from #327:
- Missing zstdDecompressWithDictWithCapacityAsync ESM export
- Version mismatch in core-lib and wasm Cargo.toml
- Missing async functions in README API list
* chore(release): version packages to v1.0.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(playground): fix GitHub Pages deployment with wasm-bindgen (#333)
Root cause: napi build/version overwrites browser.js to point at WASI
WASM, and Vite 8 rejects ESM WASM imports from wasm-pack bundler output.
Fixes:
- Add scripts/patch-browser-js.js to restore browser.js after napi
overwrites it (runs in changeset:version and build:wasm-bindgen)
- Add playground/wasm-loader.js for manual WASM instantiation
(Vite 8 ESM WASM workaround, same pattern as rapid-fuzzy)
- Update playground vite.config.js to use wasm-loader
- Remove obsolete Buffer polyfill from playground/main.js
Closes #332
* chore: improve npm discoverability and update CONTRIBUTING.md (#334)
- Add keywords: compress, wasm-bindgen, native, web-streams
- Update CONTRIBUTING.md project structure for three-crate architecture
- Fix dead "GitHub Discussion" reference (Discussions is disabled)
* perf: v1.1.0 performance improvements (#348)
* perf(gzip): use ISIZE field for decompression buffer pre-allocation
Read the gzip footer's ISIZE field (last 4 bytes, little-endian uint32)
to pre-allocate the exact output buffer size instead of using a fixed
4x heuristic. This eliminates Vec re-allocations for most decompression
workloads.
Falls back to 4x heuristic when ISIZE is 0 (empty data or >4GB wrapping)
or when input is too small to be valid gzip.
Closes #335
* perf(zstd): enable multi-threaded compression (zstdmt)
Add zstdmt feature flag to comprs-core, enabled by default in the
napi crate (Node.js) but not in the wasm crate (single-threaded).
This allows zstd to use multiple threads for compression of large data,
providing 2-4x speedup on multi-core systems.
Closes #336
* perf(streaming): reuse internal buffers instead of allocating per transform
Add a reusable output_buf field to all zstd streaming context structs.
Vec::resize reuses existing capacity after the first call, eliminating
the 128KB allocation + zeroing overhead on subsequent transform calls.
The gzip, brotli, and lz4 streaming contexts delegate buffering to
their underlying encoder/decoder inner Vec<u8> (drained via mem::take),
so they do not have the same per-call allocation pattern and are
unaffected.
Closes #337
* perf(zstd): add comparison benchmark against node:zlib
Add zstd.compare.bench.ts comparing comprs zstd performance against
Node.js 22+ built-in zlib.zstdCompressSync/zstdDecompressSync.
Closes #338
* perf(streaming): add Transform stream throughput benchmarks
Add streaming.compare.bench.ts measuring Node.js Transform stream
throughput for comprs vs node:zlib, with various chunk sizes.
Closes #340
* chore: add changeset for v1.1.0 performance improvements (#349)
Covers #335 (gzip ISIZE), #336 (zstdmt), #337 (streaming buffer reuse).
* chore(release): version packages to v1.1.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
derodero24
added a commit
that referenced
this pull request
Apr 12, 2026
* Set up napi-rs build pipeline (#9)
* chore(core): add Rust crate skeleton with napi-rs bindings
Minimal `zflate` crate with a single `version()` export to verify the
napi-rs build pipeline works end-to-end.
- crates/core/Cargo.toml: cdylib crate with napi-rs v3
- crates/core/build.rs: napi_build setup
- crates/core/src/lib.rs: version() function
* chore: add package.json with napi-rs configuration
Configure the npm package with 9 build targets (macOS, Linux, Windows,
WASM), dev dependencies for tooling (Biome, Vitest, TypeScript,
Changesets, CodSpeed), and build/test/lint scripts.
* chore: add build scripts and module entry points
- scripts/optimize-wasm.js: post-process WASM with wasm-opt
- index.mjs / index.d.mts: ESM entry points
- index.js / index.d.ts / browser.js: napi-rs generated files (force-tracked)
* Add CI workflow (#10)
* test: add initial test suite and ESM smoke test
- __test__/index.spec.ts: Vitest spec verifying version() export
- __test__/esm-import.mjs: ESM import smoke test for CI
* chore: update biome and tsconfig for test files
- biome.json: allow console in ESM smoke test
- tsconfig.json: format with Biome
* ci: add CI workflow
Full CI pipeline adapted from rapid-fuzzy:
- Lint (Biome, TypeScript, publint, generated file verification)
- Dependency audit (npm + cargo-deny)
- Rust lint (fmt + clippy) and test
- Cross-platform build (9 targets: macOS, Linux, Windows, WASM)
- Node.js test matrix (3 versions × 3 OSes)
- Coverage (Rust via cargo-llvm-cov + JS via Vitest, uploaded to Codecov)
* Add release and automation workflows (#11)
* ci: add release workflow with Cargo.toml version sync
Changesets-driven release flow:
- On develop push: create version PR, then release PR (develop → main)
- On main push: build all 9 targets and publish to npm with provenance
- scripts/sync-cargo-version.js: keep Cargo.toml version in sync with
package.json after changeset version bumps
* ci: add CodeQL security analysis
Analyze JavaScript/TypeScript and Rust code for security vulnerabilities.
Runs on PRs, develop pushes, and weekly schedule (Monday 6 AM UTC).
* ci: add CodSpeed benchmark workflow
Continuous performance tracking via CodSpeed. Runs JS benchmarks on
develop pushes and PRs when crate or bench files change.
Rust benchmarks will be added when the bench crate is introduced.
* ci: add self-hosted Renovate workflow
Automated dependency updates via self-hosted Renovate bot.
Runs weekly on Sunday 19:00 UTC (Monday 4:00 JST).
* test(bench): add initial benchmark file
Minimal benchmark to satisfy CodSpeed workflow. Real compression
benchmarks will be added alongside algorithm implementations.
* Implement zstd compression and decompression (#12)
* feat(zstd): implement zstd compression and decompression
Add zstdCompress() and zstdDecompress() functions powered by the zstd
C library via the zstd Rust crate.
- Compression levels 1-22 (default: 3)
- Auto-detection of frame content size for efficient decompression
- zstdDecompressWithCapacity() for data exceeding 256 MB default limit
- Rust unit tests for round-trip, empty data, large data, and levels
* test(zstd): add JS tests and benchmarks for zstd
- __test__/zstd.spec.ts: 11 tests covering round-trip, levels, errors
- __test__/esm-import.mjs: updated with zstd round-trip smoke test
- __test__/index.bench.ts: compress/decompress benchmarks at 150B/10KB/1MB
- index.mjs: export zstd functions
* chore: update napi-rs generated files and add changeset
- Regenerate index.js, index.d.ts, browser.js with zstd exports
- Add changeset for minor version bump
* ci: add WASI SDK setup for zstd WASM builds
The zstd crate compiles C code via zstd-sys, which requires a proper
WASI sysroot when targeting wasm32-wasip1-threads. Without it, clang
picks up system glibc headers and fails with missing
'bits/libc-header-start.h'.
Use bytecodealliance/actions/wasi-sdk/setup to install WASI SDK and
set CC, AR, CFLAGS, and WASI_SYSROOT environment variables for the
WASM build step in both CI and release workflows.
* Fix CodeRabbit review findings from #12 (#22)
* fix(zstd): validate capacity parameter in zstdDecompressWithCapacity
Reject negative, NaN, and Infinity values before casting f64 to usize.
Without validation, a negative f64 wraps to a very large usize, which
could cause memory exhaustion.
Closes #17
* fix(ci): use RUNNER_TEMP for WASI SDK installation
Replace /opt with $RUNNER_TEMP for WASI SDK extraction. /opt may not
be writable on all GitHub Actions runner configurations.
Also switch curl from -sL to -fsSL to fail on HTTP errors.
Closes #18
* docs: add SECURITY.md (#23)
* docs: add SECURITY.md
* docs: address review feedback on SECURITY.md
- Replace vague "extremely large size" with "massive size" for clarity
- Unify SLA day units by removing "business" qualifier from initial
assessment timeline for consistency
* docs: add CONTRIBUTING.md (#24)
* test(zstd): expand test coverage and benchmarks (#25)
* test(zstd): expand test coverage and benchmarks
Add comprehensive JS tests covering compression level boundaries (0,
negative, 22), decompressWithCapacity edge cases (insufficient, oversized,
invalid inputs), and round-trip verification with random data.
Expand benchmarks with random (incompressible) and realistic (JSON, text)
data variants alongside existing patterned data.
Update Rust doc comments to document negative compression levels and
level 0 behavior. Add Rust tests for boundary levels. Update changeset
to include all public API functions.
Closes #19
* chore: update napi-rs generated type definitions
Regenerate index.d.ts to reflect updated doc comments for
zstd_compress documenting negative levels and level 0 behavior.
* feat(zstd): add streaming compression/decompression API (#26)
Implement streaming zstd compression and decompression using the Web
Streams API (TransformStream). The Rust layer exposes ZstdCompressContext
and ZstdDecompressContext classes with transform/flush/finish methods
using zstd::stream::raw for stateful chunked processing.
The JS layer wraps these in TransformStream via createZstdCompressStream()
and createZstdDecompressStream(), available from both ESM and CJS entry
points. Streaming output is fully interoperable with one-shot APIs.
Add comprehensive tests covering chunked round-trips, various chunk
sizes, large data (1MB), random data, and one-shot/streaming interop.
Closes #5
* docs: add README and verify npm package readiness (#28)
* docs: add README and verify npm package readiness
Add comprehensive README with project description, installation guide,
quick-start examples (one-shot and streaming), full API reference,
supported algorithms table, platform compatibility matrix, and
benchmark results.
Verify npm package with publint (no errors). Package includes all
required files: index.js, index.mjs, index.d.ts, index.d.mts,
streams.js, streams.d.ts, browser.js, LICENSE, and README.md.
Closes #8
* docs(readme): address review feedback
- Add bun installation instructions
- Replace streaming example with self-contained runnable snippet
* fix(ci): resolve release workflow failures (#29)
* fix(ci): resolve release workflow failures
- Generate npm platform package directories for all 9 targets
- Add optionalDependencies to root package.json for platform binaries
- Fix changeset version command argument parsing in release workflow
by using a dedicated npm script instead of inline shell command
Closes #27
* fix(ci): remove optionalDependencies from committed package.json
The optionalDependencies for platform-specific packages are added
at publish time by the prepublishOnly hook (napi prepublish -t npm),
not in the committed source. Having them in package.json breaks
pnpm install --frozen-lockfile since the packages are not yet on npm.
* chore(release): version packages (#30)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(ci): regenerate napi bindings for v0.2.0 and fix version workflow (#34)
- Regenerate index.js with v0.2.0 version strings
- Update npm platform package versions to 0.2.0
- Add Rust toolchain to version job for napi build during version bump
- Add napi version, build, and format to changeset:version script
- Fix version test to read from package.json dynamically
* feat(gzip): add gzip and deflate compression support (#33)
Add gzip and raw deflate compression/decompression with one-shot and
streaming APIs. Includes interoperability tests with Node.js built-in
zlib module.
Closes #6
* feat(brotli): add brotli compression support (#32)
* feat(brotli): add brotli compression support
Add brotli compression/decompression with one-shot and streaming APIs.
Quality levels 0-11 (default: 6). Includes interoperability tests with
Node.js built-in zlib brotli module.
Closes #7
* test(brotli): add brotli compression benchmarks
Add brotli compress/decompress benchmarks matching the zstd benchmark
structure for patterned, random, and realistic data at various sizes.
* test(gzip): increase timeout for 1MB round-trip tests
Extend timeout from 5s to 30s for gzip and deflate 1MB data tests
to prevent CI failures on slower runners.
* test: increase timeout for all 1MB tests
Set 30s timeout on all 1MB data round-trip and streaming tests to
prevent failures on slower CI runners (Windows).
* docs(readme): update API reference for gzip and brotli (#36)
Add gzip, deflate, and brotli to the API reference tables. Update
supported algorithms table to reflect all three algorithms are now
available with both one-shot and streaming APIs.
* chore(deps): update taiki-e/install-action digest to c12d62a (#20)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(exports): add ./streams subpath export to package.json (#58)
Add a `./streams` subpath export so users can import streaming APIs
directly via `import { ... } from 'zflate/streams'`.
Closes #48
* fix(gzip): add compression level validation for gzip and deflate (#60)
Validate that the compression level parameter is within the valid range
(0-9) for gzip_compress, deflate_compress, GzipCompressContext, and
DeflateCompressContext. Return InvalidArg error if out of range, matching
the existing validation pattern in brotli_compress.
Closes #38
* fix(gzip): add decompression size limits to gzip and deflate (#64)
* fix(gzip): add decompression size limits to gzip and deflate
Add MAX_DECOMPRESSED_SIZE (256 MB) to gzip_decompress and
deflate_decompress to prevent zip bomb attacks. Both functions now use
chunked reading with size checking instead of unbounded read_to_end().
Add gzipDecompressWithCapacity and deflateDecompressWithCapacity for
explicit control over the maximum decompressed size, matching the
existing zstd and brotli patterns.
Closes #37
* chore(napi): regenerate bindings for new WithCapacity functions
* feat(api): accept Uint8Array input in all compression functions (#69)
Change all input parameters from Buffer to Either<Buffer, Uint8Array>
across all one-shot and streaming functions. This generates
Buffer | Uint8Array TypeScript types, enabling browser/Deno/Bun users
to pass Uint8Array directly without type errors.
Return types remain Buffer. No runtime behavior changes for existing
Buffer usage.
Closes #42
* docs: enhance README with examples, comparison table, and migration guides (#63)
* test: add cargo-fuzz targets for decompression fuzzing (#71)
* test: add edge case and error handling tests (#65)
* test: expand Node.js zlib interop tests to brotli and zstd (#67)
* ci: add hand-written JS files to workflow path filters (#56)
* ci(codeql): suppress rust/access-invalid-pointer false positives (#57)
* fix(dx): add ./streams subpath export to package.json (#59)
* ci: add cargo-audit to dependency audit pipeline (#61)
* feat(core): add unified auto-detect decompression API (#72)
Add decompress() function that auto-detects compression format from
magic bytes (zstd: 0xFD2FB528, gzip: 0x1F8B) and falls back to brotli
heuristic detection. Add detectFormat() for format identification
without decompression.
Supported formats: zstd, gzip, brotli. Raw deflate cannot be
auto-detected due to lack of magic bytes.
Closes #16
* ci(dx): add @arethetypeswrong/cli type validation (#62)
* perf(bench): add competitive benchmarks across all algorithms (#70)
* perf(wasm): add WASM binary size reporting to CI (#66)
* fix(exports): add missing ESM exports for new functions (#74)
Add decompress, detectFormat, gzipDecompressWithCapacity, and
deflateDecompressWithCapacity to index.mjs ESM export list. These
functions were available via CJS but missing from ESM imports.
Update ESM smoke test to cover all WithCapacity functions, decompress,
and detectFormat.
Closes #73
* feat(node): add Node.js stream.Transform compatibility layer (#76)
Add `zflate/node` subpath export with stream.Transform factories for all
compression algorithms (zstd, gzip, deflate, brotli), enabling use with
Node.js stream.pipeline() and pipe-based workflows.
Closes #43
* fix(brotli): use consistent error message for capacity exceeded (#78)
Change brotli_decompress_with_capacity error message from
"Destination buffer is too small" to "brotli decompress exceeded
maximum size of {} bytes" to match the pattern used by all other
decompress functions.
Closes #77
* fix(docs): remove merge conflict marker and stale src/ references (#80)
* feat(core): add async compression/decompression API (#82)
Add async versions of all one-shot compression/decompression functions
using napi-rs AsyncTask with the Task trait. This runs compression on
the libuv thread pool without blocking the Node.js event loop.
New functions: zstdCompressAsync, zstdDecompressAsync,
gzipCompressAsync, gzipDecompressAsync, deflateCompressAsync,
deflateDecompressAsync, brotliCompressAsync, brotliDecompressAsync.
Closes #40
* feat(zstd): add dictionary compression API (#83)
Add zstd dictionary training, compression, and decompression functions:
- zstdTrainDictionary: train a dictionary from sample data
- zstdCompressWithDict: compress with a pre-trained dictionary
- zstdDecompressWithDict: decompress data compressed with a dictionary
Enable the zdict_builder feature for the zstd crate to support
dictionary training. Include Rust unit tests and Vitest integration
tests verifying round-trip correctness and improved compression ratios
for small, similar data.
Closes #41
* docs: remove Cloudflare Workers from supported platforms (#91)
* ci: include Node.js WASM loader files in CI artifact (#90)
* test(wasm): add Node.js WASM compatibility test with native parity verification (#92)
* refactor(core): introduce thiserror for structured error types (#93)
Replace manual `Error::new(Status::..., ...)` calls with structured
`ZflateError` enum variants across all compression modules. Error
messages remain identical to preserve backward compatibility.
Closes #51
* test(deno): add Deno WASM smoke test (#96)
* test(bun): add Bun WASM smoke test (#95)
* test(browser): add Playwright browser WASM test (#94)
* ci(security): tighten cargo-deny bans and sources to deny mode (#108)
* ci: pin dtolnay/rust-toolchain to SHA digest (#107)
* fix(core): add decompression size limits to streaming contexts (#109)
Add cumulative output size tracking to all streaming decompression
contexts (ZstdDecompressContext, GzipDecompressContext,
DeflateDecompressContext, BrotliDecompressContext).
Each context now enforces MAX_DECOMPRESSED_SIZE (256 MB) across
transform() calls, preventing decompression bomb attacks that could
cause unbounded memory growth.
Closes #97
* feat(core): narrow detectFormat return type to string literal union (#111)
Use napi-rs string_enum to return CompressionFormat instead of plain
string, generating a const enum with 'zstd' | 'gzip' | 'brotli' |
'unknown' values in TypeScript.
Closes #103
* perf(core): optimize buffer allocation in compression and decompression (#110)
* perf(core): optimize buffer allocation in compression and decompression
Increase decompression read buffer from 4KB to 32KB in gzip, brotli,
and brotli_stream modules to reduce syscall overhead during chunked
reads. Pre-allocate compression output Vecs with input.len() capacity
for gzip and deflate encoders (both sync and async) to avoid repeated
reallocations during compression.
Closes #100
* revert: keep BUFFER_SIZE at 4096 to avoid decompress regression
The 32KB buffer caused 21 benchmark regressions in CodSpeed due to
increased stack zeroing cost. Revert to 4KB. The Vec::with_capacity
optimization for compression output is retained.
* perf(core): replace Vec clone and split_off with std::mem::take in streaming contexts (#112)
Use std::mem::take() instead of .get_ref().clone() + .get_mut().clear()
in brotli streaming contexts, and instead of .split_off(0) in gzip/deflate
streaming contexts. This avoids unnecessary memory allocation and copying
by moving ownership of the buffer contents in-place.
Closes #99
* fix(zstd): add compression level validation (#113)
Validate that the compression level is within the valid zstd range
(-131072 to 22) before passing it to the zstd library. This provides
clear error messages instead of opaque library errors.
Validation is added to:
- zstd_compress (sync)
- zstd_compress_async
- ZstdCompressContext::new (streaming)
- zstd_compress_with_dict
Closes #81
* feat(zstd): add streaming dictionary compression/decompression (#114)
Add ZstdCompressDictContext and ZstdDecompressDictContext classes that
accept a pre-trained dictionary for streaming compression/decompression.
Include Web Streams (createZstdCompressDictStream/createZstdDecompressDictStream)
and Node.js Transform (createZstdCompressDictTransform/createZstdDecompressDictTransform)
wrappers, type definitions, ESM exports, and streaming dict round-trip tests.
Closes #104
* perf(core): switch flate2 backend from miniz_oxide to zlib-rs (#115)
Replace the `rust_backend` (miniz_oxide) feature with `zlib-rs` for
2-3x faster gzip/deflate compression. zlib-rs is a pure-Rust
implementation that works on all targets including WASM, is ISRG
audited, and is on track to become flate2's default backend.
Relax compression level size ordering test since zlib-rs may produce
slightly different output sizes per level than miniz_oxide — round-trip
correctness is what matters.
Closes #98
* perf(zstd): enable multi-threaded compression for native targets (#116)
Add `zstdmt` feature for non-WASM targets via conditional dependency.
This enables zstd's internal thread pool for compression, improving
performance by 20-80% for large inputs on multi-core systems.
WASM targets are excluded because wasi-threads is experimental and
the interaction with napi-rs worker-based threading is untested.
Closes #102
* docs: comprehensive README update with missing APIs and SVG benchmark (#118)
* fix(docs): correct benchmark environment label to Apple M2 / Node.js v22 (#119)
* perf(wasm): add --converge flag to wasm-opt optimization (#122)
Re-run optimization passes until no further gains are achieved.
Increase timeout to 5 minutes to accommodate additional passes.
Closes #121
* perf(core): use 4x input size for decompression output pre-allocation (#123)
Compressed data typically decompresses to several times its original
size. Pre-allocating with input.len() alone leads to frequent
reallocations during decompression. Using input.len() * 4 as the
initial capacity provides a better heuristic while the existing
.min() caps continue to prevent excessive allocation.
Closes #120
* ci: migrate release workflow to npm trusted publishing (OIDC) (#68)
* ci: migrate release workflow to npm trusted publishing (OIDC)
Remove NPM_TOKEN/NODE_AUTH_TOKEN secret dependency and rely on
OIDC token exchange via id-token:write permission (already set).
Trusted publishing must be configured on npmjs.com for each package
before the next release.
* fix(ci): restore pinned SHA for dtolnay/rust-toolchain
Keep the pinned commit hash consistent with ci.yml and codspeed.yml
for supply chain security. The rust-toolchain change was out of scope
for the trusted publishing migration.
* fix(gzip): support concatenated gzip streams in decompression (#131)
Replace GzDecoder with MultiGzDecoder from flate2 to correctly handle
concatenated gzip members in both one-shot and streaming decompression.
* fix(streaming): add finish-guard state tracking for brotli and zstd compress contexts (#132)
* feat(core): add missing async variants for one-shot APIs (#133)
* feat(streaming): add auto-detect decompression streams (#145)
* docs: add troubleshooting section to CONTRIBUTING.md (#146)
* docs: add CODE_OF_CONDUCT.md (#149)
* ci: apply least-privilege permissions to workflows (#147)
* chore: improve package.json metadata and exports map (#148)
* ci: clean up CI workflow configuration (#150)
* feat(core): expose CRC32 utility function (#134)
* feat(core): expose CRC32 utility function
* test(crc32): use skipIf for conditional Node.js zlib interop test
* ci(testing): add coverage thresholds to vitest config (#151)
* ci: add aggregate gate job to CI workflow (#152)
* feat(streaming): add configurable maxOutputSize to decompression contexts (#153)
Add optional maxOutputSize parameter to all streaming decompression
context constructors (GzipDecompressContext, DeflateDecompressContext,
ZstdDecompressContext, BrotliDecompressContext, ZstdDecompressDictContext).
When not specified, the default 256 MB limit is used.
The parameter is also passed through all streaming factory functions
(createXxxDecompressStream, createXxxDecompressTransform, and
createDecompressStream/createDecompressTransform).
Closes #129
* ci: add environment protection for npm publish (#154)
* ci: add cargo-shear for unused dependency detection (#155)
* ci: add WASM binary size tracking on pull requests (#156)
* ci: add scripts/ to CI workflow path triggers (#159)
* fix(wasm): add JS-side streaming adapters for browser WASM environment (#158)
Replace native streaming contexts with JS-side adapters in the browser
entry point to work around WebAssembly.Memory growth invalidating
ArrayBuffer views. Adapters accumulate chunks and delegate to one-shot
APIs, maintaining the same transform()/flush()/finish() API surface.
- browser-streaming.js: adapter classes for all 10 streaming contexts
- browser-entry.js: re-exports WASM APIs with adapter overrides
- package.json: browser condition points to browser-entry.js
- e2e: streaming tests enabled for browser WASM
* feat(gzip): add header metadata support for compression and reading (#160)
Closes #130
* chore(renovate): run daily and remove PR concurrent limit (#164)
* chore(renovate): run daily and remove PR concurrent limit
Closes #163
* chore(renovate): disable lock file maintenance
* chore(deps): update codspeedhq/action digest to 1c8ae48 (#161)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dtolnay/rust-toolchain digest to 631a55b (#162)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to 3869755 (#165)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to b988c18 (#166)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(bench): exclude third-party benchmarks from CodSpeed tracking (#169)
* fix(bench): exclude third-party benchmarks from CodSpeed tracking
Split benchmark files into zflate-only (*.bench.ts) and comparison
(*.compare.bench.ts) variants. CodSpeed now only tracks zflate's own
benchmarks, eliminating false regressions from pako/fflate/node:zlib
that are caused by GitHub Actions CPU heterogeneity affecting Callgrind
instruction counting.
Closes #167
* fix(bench): use deterministic random data in CodSpeed benchmarks
Replace non-deterministic randomBytes() with a seeded LCG to ensure
reproducible benchmark inputs across runs, reducing CodSpeed instruction
count variance.
* feat(bench): migrate from CodSpeed to Bencher.dev for regression detection (#170)
Replace CodSpeed simulation mode (Callgrind instruction counting) with
Bencher.dev (statistical wall-clock analysis). CodSpeed's instruction
counting was fundamentally incompatible with this napi-rs project due to
GitHub Actions CPU heterogeneity and glibc's CPU-adaptive malloc.
Changes:
- Add Bencher workflow with t-test threshold for regression detection
- Add vitest-to-bmf.js conversion script for vitest bench output
- Remove CodSpeed workflow and @codspeed/vitest-plugin dependency
- Remove CodSpeed plugin from vitest config
Closes #168
* feat(bench): switch to CodSpeed Walltime mode for accurate benchmarks (#172)
Replace Bencher.dev with CodSpeed Walltime mode (Macro Runners) which
runs benchmarks on dedicated bare-metal hardware, eliminating GitHub
Actions CPU variance. CodSpeed also provides native vitest integration
via @codspeed/vitest-plugin, removing the need for conversion scripts.
Key change: `mode: walltime` instead of the previous `mode: simulation`
which used Callgrind instruction counting.
Closes #171
* refactor(bench): split benchmark files per algorithm for granular CodSpeed reporting (#174)
Split index.bench.ts (mixed zstd+brotli) into separate zstd.bench.ts
and brotli.bench.ts so CodSpeed PR reports only show benchmarks relevant
to the changed algorithm. Extract shared test fixtures into
bench-fixtures.ts and add realistic data benchmarks (JSON, text) to
gzip and deflate for consistent coverage across all four algorithms.
Closes #173
* perf(core): eliminate double-copy buffer pattern in decompression (#178)
* perf(core): eliminate double-copy buffer pattern in decompression
Replace the manual 4 KiB read loop (stack buffer → Vec copy) with
read_to_end + Take, which writes directly into Vec spare capacity.
This eliminates one full copy of every decompressed byte and uses
adaptive buffer sizing instead of fixed 4 KiB chunks.
Affected: gzip, deflate, brotli (sync + async, default + with-capacity)
Closes #175
* fix(core): use saturating_mul for init capacity and update size limit tests
Address CodeRabbit review feedback:
- Replace len() * 4 with len().saturating_mul(4) across all call sites
to prevent overflow on 32-bit targets (wasm32-wasip1-threads)
- Replace old chunk-loop size limit tests with Take+read_to_end tests
that exercise the actual decompression pattern (avoiding napi::Error
symbol resolution issues in standalone test binaries)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks (#179)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks
- Add bench:ci script that excludes comparison benchmarks
- Simplify CodSpeed workflow to use bench:ci instead of explicit file list
- Use glob pattern for bench file path triggers
- Add crates/bench workspace member with Criterion benchmarks
(gzip, deflate, brotli, zstd) using codspeed-criterion-compat
- Add rust-benchmarks job to CodSpeed workflow (simulation mode)
Closes #177
* fix(ci): add Cargo.toml, Cargo.lock, package.json to CodSpeed path triggers
Ensures CodSpeed benchmarks run when workspace config or dependency
lockfiles change, which could affect benchmark behavior.
* feat(core): add LZ4 frame compression support (#180)
* feat(core): add LZ4 frame compression support
Add LZ4 as a fourth compression algorithm using lz4_flex (pure Rust):
- Bulk API: lz4Compress, lz4Decompress, lz4DecompressWithCapacity
- Async API: lz4CompressAsync, lz4DecompressAsync, lz4DecompressWithCapacityAsync
- Streaming: Lz4CompressContext, Lz4DecompressContext
- Web Streams: createLz4CompressStream, createLz4DecompressStream
- Format detection: detectFormat returns "lz4" for LZ4 frame magic bytes
- Auto-decompress: decompress/decompressAsync and createDecompressStream
now handle LZ4 data automatically
Closes #176
* fix(core): add Read import to LZ4 test modules
Tests use read_to_end which requires the Read trait in scope.
* refactor(core): validate capacity as non-negative integer (#183)
* refactor(core): validate capacity as non-negative integer across all algorithms
Extract shared validate_capacity() that rejects NaN, Infinity, negative,
fractional values, and values exceeding usize::MAX. Applied to all 10
DecompressWithCapacity functions (sync + async × 5 algorithms).
Closes #181
* test: add regression tests for fractional and oversized capacity
* perf(lz4): implement incremental streaming for LZ4 compression (#184)
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with cursor-based incremental output in
Lz4CompressContext. The FrameEncoder now persists across transform()
calls, emitting compressed blocks as they become available instead of
buffering all input until finish().
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with FrameEncoder-based incremental output
in Lz4CompressContext. Compressed blocks are emitted as internal buffers
fill, and old bytes are drained to bound memory usage.
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* chore(deps): update taiki-e/install-action digest to 0620367 (#185)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust crate lz4_flex to 0.13 (#186)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore: rename package from zflate to comprs (#190)
* chore: rename package from zflate to comprs
Rename npm package, Rust crates, napi binary name, platform packages,
and all references from "zflate" to "comprs". The name "comprs" conveys
"compress" with a Rust flavor, is available on npm, and avoids the
typosquat protection that blocks "zflate".
Changes across 70 files:
- npm: comprs, comprs-darwin-arm64, comprs-wasm32-wasi, etc.
- Rust: comprs (core), comprs-bench
- napi binaryName: comprs (→ comprs.*.node, comprs.*.wasm)
- README, workflows, changesets, tests, benchmarks
GitHub repository URL (derodero24/zflate) is preserved.
Closes #189
* chore: update GitHub repo URLs to derodero24/comprs
Update all repository references to match the planned GitHub repo
rename from derodero24/zflate to derodero24/comprs.
* chore(core): rename ZflateError to ComprsError and update bench vars (#192)
Rename internal Rust error type `ZflateError` → `ComprsError` across
all source files and update benchmark variable names `*_ZFLATE` →
`*_COMPRS` for consistency with the package rename in #190.
Closes #191
* fix: add missing LZ4 docs, browser exports, Node.js transforms, and publish files (#194)
- Add LZ4 one-shot and streaming context exports to browser-entry.js
- Add Lz4CompressContext/Lz4DecompressContext adapters to browser-streaming.js
- Add createLz4CompressTransform/createLz4DecompressTransform to node.js
- Add LZ4 type declarations to node.d.ts
- Add 'lz4' case to createDecompressContext in node.js
- Add browser-entry.js and browser-streaming.js to package.json files array
- Document LZ4 API in README.md (one-shot, async, streaming, comparison)
- Fix streaming import path in README.md (comprs → comprs/streams)
- Update auto-detect descriptions to include LZ4
- Add "lz4" to package.json keywords and description
- Add LZ4 changeset for upcoming release
Closes #193
* fix: add missing LZ4 exports to ESM entry point and complete API docs (#196)
- Add LZ4 one-shot APIs, Context classes, and streaming functions to
index.mjs (ESM users could not access LZ4 at all)
- Add createLz4CompressStream/createLz4DecompressStream to index.mjs
- Add gzipCompressWithHeader and gzipReadHeader to browser-entry.js
- Document gzipCompressWithHeader, gzipReadHeader in README
- Document crc32 and version() utilities in README
- Document createDecompressStream and createDecompressTransform in README
Closes #195
* fix(docs): add missing lz4 to decompressAsync doc and clarify async section (#198)
The decompressAsync Rust doc comment listed only zstd, gzip, brotli as
supported formats, but the implementation also handles lz4. The README
async section claimed "all one-shot functions" have async variants, which
is inaccurate for utility functions like crc32, version, gzipReadHeader.
Closes #197
* chore(release): version packages (#35)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* perf(ci): remove JS walltime benchmarks from CodSpeed regression detection (#201)
JS walltime benchmarks on standard GitHub Actions runners produce 20-50%
variance due to CPU heterogeneity (AMD EPYC vs Intel Xeon assigned randomly)
and glibc's CPU-adaptive malloc. This caused CodSpeed Performance Analysis to
fail on nearly every PR since #174, including documentation-only changes.
Keep only Rust simulation benchmarks in CodSpeed, which use instruction
counting via Callgrind and produce <1% variance. Also narrow the workflow
path triggers to Rust-only files since JS changes no longer affect the
CodSpeed pipeline.
JS benchmarks remain available locally via `pnpm run bench`.
Closes #200
* ci(release): use granular access token for initial npm publish
The first publish requires an NPM_TOKEN since OIDC trusted publishing
only works for packages that already exist on the registry. After the
initial release, this will be switched back to OIDC.
* chore: add .mcp.json to .gitignore (#205)
Closes #203
* fix(core): use decompress_with_limit in async auto-detect decompression (#216)
Replace manual chunk-read loops with double-copy in DecompressTask for
gzip and brotli branches. Now uses crate::decompress_with_limit(),
matching the pattern already used by the LZ4 branch and per-algorithm
async tasks.
Closes #207
* ci: add browser files to CI path triggers (#217)
Closes #212
* fix(zstd): validate maxDictSize parameter in dictionary training (#219)
* test: update ESM smoke test to cover all exports (#215)
* Add LZ4 edge-case and streaming test coverage (#218)
* test(lz4): add edge-case and streaming test coverage
* test(lz4): fix empty buffer test to match actual lz4 behavior
LZ4 returns empty buffer for empty input (like zstd and deflate),
rather than throwing an error (like gzip and brotli).
* fix(zstd): enforce maxOutputSize in browser dict decompression (#221)
* fix(zstd): enforce maxOutputSize in browser dict decompression
Add zstd_decompress_with_dict_with_capacity Rust function and use it
in browser-streaming.js ZstdDecompressDictContext.flush() when
maxOutputSize is provided.
Closes #208
* chore: regenerate napi-rs binding files
* refactor(core): consolidate MAX_DECOMPRESSED_SIZE constant (#222)
Remove duplicated MAX_DECOMPRESSED_SIZE definitions from zstd, gzip,
brotli_impl, lz4, and detect modules. Change the lib.rs definition to
pub(crate) and reference it via crate::MAX_DECOMPRESSED_SIZE in all
modules.
* chore(release): version packages (#220)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* docs: add benchmark comparison results to README (#224)
Closes #213
* feat(brotli): add dictionary compression and decompression support (#225)
* docs: add brotli dictionary to README and update Node.js transforms (#228)
* docs: readme visual and structural overhaul (#239)
- Center-align hero section with title, tagline, and badges
- Add Node.js version and TypeScript badges
- Add Table of Contents for navigation
- Wrap benchmark tables in collapsible <details> sections
- Replace plain blockquote with GitHub Admonition (> [!TIP])
- Replace raw build target triples with human-readable table
Closes #229
* docs: reposition and enhance comparison table (#240)
- Move comparison table from bottom of README to right after "Why comprs?"
- Split "Streaming" into "Web Streams API" and "Node.js Transform" rows
- Replace ambiguous `✅*` with `⚠️ Experimental*` for node:zlib zstd
- Replace footnote `†` with inline "Chunked†" for pako streaming
- Update Table of Contents to reflect new section order
- Center-align feature columns for better scannability
Closes #236
* docs: improve quick start examples and add runtime-specific usage (#241)
- Condense one-shot examples to single primary zstd block + multi-algorithm block
- Replace impractical manual ReadableStream example with fetch-style streaming
- Add Node.js Transform stream example to Quick Start
- Add brief comment explaining async runs on libuv thread pool
- Consolidate compression levels into compact format
- Add Deno and Bun import examples
Closes #235
* docs: add algorithm selection guide (#242)
- Add "Choosing an Algorithm" section with use case → recommendation table
- Add "Choosing an API mode" subsection for sync/async/streaming/dictionary
- Update Table of Contents
Closes #234
* docs: document browser WASM requirements (SharedArrayBuffer, COOP/COEP) (#243)
- Add SharedArrayBuffer requirement warning with required HTTP headers
- Remove misleading "or import directly from a CDN" claim
- Link to MDN SharedArrayBuffer security requirements
Closes #231
* docs: add SSR framework integration guide (#244)
- Add "Framework Integration (SSR)" subsection under Browser Usage
- Include Next.js serverExternalPackages and Vite SSR external config
- Note automatic WASM fallback on client side
Closes #232
* docs: add fflate migration guide (#245)
- Add "From fflate" migration section with diff example
- Note comprs advantages over fflate (zstd, lz4, brotli, dictionaries, Web Streams)
Closes #237
* docs: add known limitations and performance notes (#246)
- Add "Notes" section with three admonitions covering:
- 256 MB default decompression limit with WithCapacity() workaround
- Small payload WASM overhead advisory
- Brotli decompression performance nuance vs node:zlib
Closes #238
* docs: add all undocumented async and capacity API variants to readme (#247)
- Add zstdDecompressWithDictWithCapacity to sync zstd dictionary table
- Add 9 missing async variants: WithCapacityAsync for all algorithms,
zstd dict async, zstdTrainDictionaryAsync, decompressAsync
- Reorder async table to group by algorithm with capacity/dict variants
Closes #233
* docs: add benchmark comparison SVG charts for cross-algorithm and brotli (#248)
- Add cross-algorithm compression chart (zstd/lz4/gzip/brotli at 10KB)
- Add brotli compression speedup chart (134.7x faster than node:zlib)
- Embed both charts alongside existing gzip chart in Benchmarks section
Closes #230
* docs: fold repetitive API tables into collapsible sections (#250)
- Wrap Dictionary API (zstd + brotli) in <details>
- Replace Async table header with pattern explanation + code example,
fold full function list into <details>
- Add Streaming import path + fold table into <details>
- Keep Node.js Transform code example visible, fold table into <details>
- One-shot core API tables remain visible as primary reference
Closes #249
* chore(deps): update taiki-e/install-action digest to 7627fb4 (#251)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.9 (#252)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update renovatebot/github-action action to v46.1.6 (#254)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency vite to v8.0.3 (#253)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update pnpm to v10.33.0 (#256)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.1 (#255)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency typescript to v6 (#257)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* feat: add interactive playground with GitHub Pages deployment (#259)
Add a browser-based playground for trying comprs compression live.
Includes four algorithms (zstd, gzip, brotli, lz4), level slider,
comparison table, file upload, and download. Deploys to GitHub Pages
via new CI workflow. Uses a JS mock for local dev without a WASM build.
Closes #258
* docs: improve playground visibility and add OGP/SEO meta tags (#261)
Add prominent text link below badges in README header and playground
as a bullet in "Why comprs?". Add Open Graph, Twitter Card, and
canonical URL meta tags to playground/index.html for social sharing.
Closes #260
* fix(playground): handle COI not established on first visit (#263)
On first visit, the COI service worker has not yet reloaded the page,
so crossOriginIsolated is false. Calling the WASM function immediately
throws because SharedArrayBuffer is unavailable. This showed a
misleading "unsupported browser" error even in modern browsers like
Vivaldi, Brave, or Arc.
- Check crossOriginIsolated before invoking WASM
- Show "Enabling security features... reloading" while the service
worker handles the reload
- After 3 s, show a manual Reload button if auto-reload did not happen
- Include the actual error message when WASM init itself fails
- Add .loading-reload-btn styles to style.css
Closes #262
* fix(playground): add Buffer polyfill for emnapi WASM compatibility (#265)
The napi-rs WASM runtime (emnapi) requires globalThis.Buffer when
creating memory views for function arguments/return values. Browsers
do not define Buffer natively, causing zstdCompress and other WASM
calls to throw immediately.
Add the 'buffer' npm package (v6 browser polyfill) to playground
dependencies and set globalThis.Buffer before any WASM calls.
Closes #264
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init (#267)
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init
Static imports are hoisted and evaluated before any module body code runs.
This caused globalThis.Buffer to be unset when comprs-wasm32-wasi evaluated
__emnapiInstantiateNapiModuleSync (which calls emnapi_create_memory_view).
Switch to a dynamic import so the Buffer polyfill assignment executes first.
Closes #266
* fix(playground): allow Vite dev server to serve WASM from parent directory
When comprs.wasi-browser.js is present in the repo root, the WASM file
is fetched at runtime via /@fs/ from outside the playground directory.
Add server.fs.allow so Vite does not block the request with 403.
Only applied when a local WASM build is detected (hasLocalWasm=true).
* fix(playground): fix comparison table label direction and badge layout (#271)
- Rename bar chart column 'Ratio (smaller = better)' to 'Savings'
and numeric column 'Ratio' to 'Saved' to accurately reflect that
the metric is compression savings % (larger = better)
- Change 'best ratio' badge to 'best' (shorter, clearer)
- Add 'fastest' badge to the fastest algorithm row; remove the
duplicate 'Fastest:' text from the footer note
- Change badge column from fixed 72px to auto to prevent overflow
Closes #270
* fix(playground): fix negative ratio text, CSS variable, download label, and file size alert (#273)
- Show "X% larger" instead of "-X% smaller" when compressed output exceeds input
- Fix reload button using undefined var(--accent); use var(--text)/var(--bg) instead
- Add file extension to download button (e.g. "Download .zst") via #download-ext span
- Replace alert() for oversized file with inline error message that auto-dismisses after 5s
Closes #272
* chore(deps): update codecov/codecov-action digest to 75cd116 (#268)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.2 (#269)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(playground): add FileReader error handler and textarea aria-label (#275)
- Add reader.onerror handler to show inline error when file reading fails
instead of silently doing nothing (e.g. on disk error or revoked access)
- Add aria-label to the input textarea for screen reader accessibility
Closes #274
* chore(deps): update dtolnay/rust-toolchain digest to 29eef33 (#276)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to c10b806 (#277)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/configure-pages action to v6 (#283)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/deploy-pages action to v5 (#284)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/upload-pages-artifact action to v4 (#285)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to d858f81 (#278)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust-dependencies to v3.8.4 (#281)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.2 (#280)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.10 (#279)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update codecov/codecov-action action to v6 (#286)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 (#287)
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2
Update @napi-rs/cli from v3.5.1 to v3.6.0 along with @emnapi/core
and @emnapi/runtime from v1.9.1 to v1.9.2 to resolve emnapi version
mismatch that caused WASM build failures.
* fix(deps): align emnapi transitive dependency to v1.9.2
Update the emnapi package (transitive via @napi-rs/cli) from v1.9.1
to v1.9.2, matching @emnapi/core and @emnapi/runtime versions.
* chore(deps): update renovatebot/github-action action to v46.1.7 (#292)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @types/node to v24.12.2 (#291)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(config): update biome.json schema to v2.4.10 (#294)
* chore(deps): update dependency @playwright/test to v1.59.1 (#293)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(core): remove unused wildcard import in crc test module (#295)
* fix(ci): update setup-deno and changesets/action to resolve Renovate lookup failures (#296)
Pin to specific version tags instead of major version floating tags,
which do not exist in these repositories. This allows Renovate to
resolve digests for tracking updates.
- denoland/setup-deno: v2 → v2.0.4
- changesets/action: v1 → v1.7.0
Closes #290
* chore(release): version packages to v0.4.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* refactor: rename npm package to @derodero24/comprs (#298)
Rename all npm packages from `comprs-*` to `@derodero24/comprs-*` to
bypass npm's typosquatting protection that blocked the `comprs` name.
- Root package: comprs → @derodero24/comprs
- 9 platform packages: comprs-{platform} → @derodero24/comprs-{platform}
- Regenerated napi-rs loader (index.js, browser.js) with new names
- Updated release workflow version checks
- Updated README install/import examples
- Updated playground npm link
Closes #297
* fix(playground): update vite alias for renamed WASM package
Update the vite resolve alias from 'comprs-wasm32-wasi' to
'@derodero24/comprs-wasm32-wasi' to match the package rename in #298.
* fix(ci): add publishConfig and permissions for scoped npm publish (#302)
- Add publishConfig.access: "public" to root and all platform
package.json files so napi prepublish can publish scoped packages
- Add contents: write permission to Publish job for GitHub Release
creation by napi prepublish
- Add changeset for v0.4.1 patch release
Closes #301
* chore(release): version packages to v0.4.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: update stale package references after rename to @derodero24/comprs (#307)
Update remaining references from old unscoped names to scoped names:
- browser-streaming.js: comprs-wasm32-wasi → @derodero24/comprs-wasm32-wasi
- playground/main.js: import('comprs') → import('@derodero24/comprs')
- playground/package.json: comprs → @derodero24/comprs dependency
- playground/vite.config.js: Vite alias keys updated
This fixes the GitHub Pages deployment failure caused by Vite being
unable to resolve the old unscoped import in browser-streaming.js.
Closes #306
* fix(playground): regenerate lockfile after package rename
The playground's pnpm-lock.yaml was not updated when the dependency
was renamed from comprs to @derodero24/comprs, causing CI to fail
with frozen-lockfile mismatch.
* docs: fix README package references, streaming note, and WASM size (#311)
* fix(docs): update remaining old package name references in README
- Deno import: npm:comprs → npm:@derodero24/comprs
- Subpath imports: comprs/streams → @derodero24/comprs/streams,
comprs/node → @derodero24/comprs/node
- SSR config: Next.js and Vite external package names
Closes #308
* docs: clarify browser streaming memory behavior
Add note explaining that browser WASM streaming buffers all data in
memory (unlike Node.js which streams incrementally). Also qualify the
"bounded memory" claim in the feature list to specify Node.js.
Closes #309
* docs: add concrete WASM binary size to README
Show approximate optimized WASM size (~2.0 MB raw, ~800 KB gzipped)
directly in README so users can evaluate without navigating to CI.
Closes #310
* fix: update npm badges and sub-package descriptions (#317)
* fix(docs): update npm badges to use scoped package name
The npm version and downloads badges were pointing to the old unscoped
package name "comprs" instead of the scoped "@derodero24/comprs".
Closes #312
* fix: update platform sub-package descriptions to include lz4
All 9 platform sub-packages under npm/ had descriptions listing only
"zstd, gzip, and brotli" but were missing "lz4" which was added to
the root package. Updated to match: "zstd, gzip, brotli, and lz4".
Closes #315
* fix: add missing ESM export and test coverage (#318)
* fix: add missing zstdDecompressWithDictWithCapacity to ESM and browser exports
Add zstdDecompressWithDictWithCapacity to index.mjs destructured exports,
browser-entry.js one-shot re-exports, and ESM smoke test assertions.
Closes #313
* test: add coverage for WithDictWithCapacity functions and Node.js Transform streams
Add tests for zstdDecompressWithDictWithCapacity, brotliDecompressWithDictWithCapacity,
brotliDecompressWithDictWithCapacityAsync, LZ4 Node.js Transform round-trip,
zstd dict Node.js Transform round-trip, and brotli dict Node.js Transform round-trip.
Closes #314
* feat(wasm): replace WASI WASM with wasm-bindgen for browser compatibility (#320)
* refactor(core): extract pure Rust logic into core-lib crate
Split crates/core into two crates:
- crates/core-lib: pure Rust compression logic (no FFI dependencies)
- crates/core: napi-rs thin wrappers delegating to core-lib
This prepares for adding a wasm-bindgen crate that shares the same
core logic, enabling browser usage without SharedArrayBuffer.
Part of #319
* feat(wasm): add wasm-bindgen browser binding crate
Create crates/wasm with wasm-bindgen wrappers over comprs-core,
targeting wasm32-unknown-unknown (no SharedArrayBuffer required).
Includes one-shot and streaming APIs for all algorithms:
zstd, gzip, deflate, brotli, lz4, auto-detect, and crc32.
Part of #319
* feat(wasm): integrate wasm-bindgen build into package distribution
- Add build:wasm-bindgen script using wasm-pack
- Update browser.js to use wasm-bindgen output
- Update browser-streaming.js imports
- Add wasm-bindgen files to npm package
- Update CI with wasm-bindgen build job
- Update release workflow for wasm-bindgen artifacts
- Update playground configuration
- Remove SharedArrayBuffer requirement from README
Part of #319
* fix(ci): resolve CI failures for wasm-bindgen migration
- Remove browser.js from napi generated file diff check (napi build
regenerates it, but we now maintain it manually for wasm-bindgen)
- Remove unused direct compression deps from napi crate (all logic
now lives in core-lib)
- Update e2e browser test to use wasm-bindgen output instead of
WASI browser entry
- Remove COOP/COEP headers from e2e vite config (no longer needed)
Part of #319
* fix(e2e): use manual WASM instantiation for wasm-bindgen in browser test
Vite 8 does not support the TC39 WebAssembly ESM integration proposal.
Import _bg.js directly and instantiate WASM binary manually, following
the same pattern used in rapid-fuzzy.
Also add lz4, crc32, and detectFormat tests to the browser e2e suite.
Part of #319
* fix(e2e): update browser test spec to match wasm-bindgen test suite
Replace streaming tests (removed with WASI migration) with lz4,
crc32, and detectFormat assertions.
Part of #319
* chore: add changeset for v1.0.0 major release (#322)
Closes #321
* chore(release): version packages to v1.0.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: address post-release bugs in v1.0.0 (#327)
- Add missing zstdDecompressWithDictWithCapacityAsync to index.mjs
- Sync crates/core-lib and crates/wasm version to 1.0.0
- Fix sync-cargo-version.js to cover all three Cargo.toml files
- Add missing async functions to README API list
- Add missing ESM smoke test assertion
Closes #326
* chore: add changeset for v1.0.1 patch release (#328)
Covers the bug fixes from #327:
- Missing zstdDecompressWithDictWithCapacityAsync ESM export
- Version mismatch in core-lib and wasm Cargo.toml
- Missing async functions in README API list
* chore(release): version packages to v1.0.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(playground): fix GitHub Pages deployment with wasm-bindgen (#333)
Root cause: napi build/version overwrites browser.js to point at WASI
WASM, and Vite 8 rejects ESM WASM imports from wasm-pack bundler output.
Fixes:
- Add scripts/patch-browser-js.js to restore browser.js after napi
overwrites it (runs in changeset:version and build:wasm-bindgen)
- Add playground/wasm-loader.js for manual WASM instantiation
(Vite 8 ESM WASM workaround, same pattern as rapid-fuzzy)
- Update playground vite.config.js to use wasm-loader
- Remove obsolete Buffer polyfill from playground/main.js
Closes #332
* chore: improve npm discoverability and update CONTRIBUTING.md (#334)
- Add keywords: compress, wasm-bindgen, native, web-streams
- Update CONTRIBUTING.md project structure for three-crate architecture
- Fix dead "GitHub Discussion" reference (Discussions is disabled)
* perf: v1.1.0 performance improvements (#348)
* perf(gzip): use ISIZE field for decompression buffer pre-allocation
Read the gzip footer's ISIZE field (last 4 bytes, little-endian uint32)
to pre-allocate the exact output buffer size instead of using a fixed
4x heuristic. This eliminates Vec re-allocations for most decompression
workloads.
Falls back to 4x heuristic when ISIZE is 0 (empty data or >4GB wrapping)
or when input is too small to be valid gzip.
Closes #335
* perf(zstd): enable multi-threaded compression (zstdmt)
Add zstdmt feature flag to comprs-core, enabled by default in the
napi crate (Node.js) but not in the wasm crate (single-threaded).
This allows zstd to use multiple threads for compression of large data,
providing 2-4x speedup on multi-core systems.
Closes #336
* perf(streaming): reuse internal buffers instead of allocating per transform
Add a reusable output_buf field to all zstd streaming context structs.
Vec::resize reuses existing capacity after the first call, eliminating
the 128KB allocation + zeroing overhead on subsequent transform calls.
The gzip, brotli, and lz4 streaming contexts delegate buffering to
their underlying encoder/decoder inner Vec<u8> (drained via mem::take),
so they do not have the same per-call allocation pattern and are
unaffected.
Closes #337
* perf(zstd): add comparison benchmark against node:zlib
Add zstd.compare.bench.ts comparing comprs zstd performance against
Node.js 22+ built-in zlib.zstdCompressSync/zstdDecompressSync.
Closes #338
* perf(streaming): add Transform stream throughput benchmarks
Add streaming.compare.bench.ts measuring Node.js Transform stream
throughput for comprs vs node:zlib, with various chunk sizes.
Closes #340
* chore: add changeset for v1.1.0 performance improvements (#349)
Covers #335 (gzip ISIZE), #336 (zstdmt), #337 (streaming buffer reuse).
* chore(release): version packages to v1.1.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* chore(deps): update taiki-e/install-action digest to 7a562df (#353)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update embarkstudios/cargo-deny-action digest to 175dc7f (#354)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to 97a5807 (#355)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update renovatebot/github-action action to v46.1.8 (#361)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency lefthook to v2.1.5 (#359)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.11 (#356)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency vite to v8.0.8 (#360)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/cli to v3.6.1 (#357)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.3 (#358)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.4 (#362)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): set minimumReleaseAge to 3 days in Renovate (#363)
Prevent adopting newly released packages that may be yanked or have
undiscovered issues. This applies globally to npm, Cargo, and GitHub
Actions dependencies.
* feat(middleware): add Express HTTP compression middleware (#364)
* feat(middleware): add Express HTTP compression middleware
Add @derodero24/comprs-middleware — the first Express middleware with
zstd support. Includes Accept-Encoding negotiation, configurable
algorithm priority, threshold-based skip, and Content-Type filtering.
- Set up pnpm workspace for multi-package monorepo
- Implement negotiation, compression, and middleware modules
- Add 31 unit and integration tests
- Update CI to lint/test the middleware package
- Update release workflow for multi-package publishing
Closes #343
* fix(middleware): address CodeRabbit review feedback
- Make isCompressibleType return false for missing Content-Type
- Add stream error handler to prevent unhandled exceptions
- Remove wildcard fallback that could override explicit rejections
- Separate middleware publish job for independent releases
- Limit middleware CI tests to ubuntu/node24 only
- Narrow biome noBarrelFile override to middleware package
- Refactor test helper to support HEAD requests
- Add HEAD request skip test
- Remove "first" claim from README descriptions
* fix(middleware): fix description, docs, and add typescript devDep
- Remove "First" claim from package.json description
- Update filter JSDoc to reflect Content-Type skip behavior
- Add typescript to devDependencies for standalone builds
- Fix filter default description in README table
* fix(middleware): update lockfile for typescript devDependency
* fix(middleware): normalize write/end args and preserve callbacks
Properly handle all overload forms of res.write() and res.end():
- Normalize chunk, encoding, and callback from variadic args
- Pass callbacks through to compressStream.write/end
- Handle res.end(callback) without treating function as chunk body
- Use null-check (!=) instead of truthy for chunk to handle empty strings
* chore(release): version packages (#365)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(ci): check all workspace packages for unpublished versions (#367)
* fix(ci): check all workspace packages for unpublished versions
The release PR creation step only checked the core package version,
preventing middleware-only releases from triggering a develop→main PR.
Now checks both @derodero24/comprs and @derodero24/comprs-middleware
versions against npm, creating a release PR if any package is unpublished.
Closes #366
* fix(ci): read package names from manifest and handle npm errors
- Read package names from package.json instead of hardcoding
- Distinguish E404 (unpublished) from network/auth errors
- Fail the job on unexpected npm view errors instead of silently skipping
* feat(middleware): add Fastify and Hono support via subpath exports (#370)
* feat(middleware): add Fastify and Hono support via subpath exports
Restructure @derodero24/comprs-middleware from Express-only to
multi-framework with subpath exports:
- ./express — Express/Connect middleware (refactored from root export)
- ./fastify — Fastify plugin (onSend hook with stream compression)
- ./hono — Hono middleware (sync compression via Web Standards API)
- . — shared utilities (negotiate, types)
Extract framework-agnostic logic into shared.ts. Add 21 new tests
for Fastify (10) and Hono (11), total 53 tests across 4 test files.
Closes #369
* fix(middleware): address CodeRabbit review feedback
- Handle Vary: * without appending Accept-Encoding (RFC 7231)
- Extract shouldSkip helper in Hono adapter to reduce complexity
- Add Content-Length early threshold check in Hono to avoid body read
- Log compression pipeline errors in Fastify (skip premature close)
- Add stream compression behavior comment in Fastify
- Add identity-only Accept-Encoding test for Fastify
- Fix README import examples to use separate blocks per framework
* fix(middleware): case-insensitive Cache-Control check, Hono error handling
- Normalize Cache-Control to lowercase before checking no-transform (all adapters)
- Set Vary header before HEAD early return in Hono
- Wrap Hono body read + compression in try/catch for graceful fallback
* fix(ci): use manifest name and E404 check in publish-middleware job (#373)
Read package name from package.json instead of hardcoding.
Distinguish E404 (unpublished) from network/auth errors to prevent
silent failures during middleware publishing.
Closes #372
* chore(release): version packages (#371)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
---------
Co-authored-by: Shintaro Okada <38486312+derodero24@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
Co-authored-by: derodero24 <shintaro@payn.io>
derodero24
added a commit
that referenced
this pull request
May 4, 2026
* Set up napi-rs build pipeline (#9)
* chore(core): add Rust crate skeleton with napi-rs bindings
Minimal `zflate` crate with a single `version()` export to verify the
napi-rs build pipeline works end-to-end.
- crates/core/Cargo.toml: cdylib crate with napi-rs v3
- crates/core/build.rs: napi_build setup
- crates/core/src/lib.rs: version() function
* chore: add package.json with napi-rs configuration
Configure the npm package with 9 build targets (macOS, Linux, Windows,
WASM), dev dependencies for tooling (Biome, Vitest, TypeScript,
Changesets, CodSpeed), and build/test/lint scripts.
* chore: add build scripts and module entry points
- scripts/optimize-wasm.js: post-process WASM with wasm-opt
- index.mjs / index.d.mts: ESM entry points
- index.js / index.d.ts / browser.js: napi-rs generated files (force-tracked)
* Add CI workflow (#10)
* test: add initial test suite and ESM smoke test
- __test__/index.spec.ts: Vitest spec verifying version() export
- __test__/esm-import.mjs: ESM import smoke test for CI
* chore: update biome and tsconfig for test files
- biome.json: allow console in ESM smoke test
- tsconfig.json: format with Biome
* ci: add CI workflow
Full CI pipeline adapted from rapid-fuzzy:
- Lint (Biome, TypeScript, publint, generated file verification)
- Dependency audit (npm + cargo-deny)
- Rust lint (fmt + clippy) and test
- Cross-platform build (9 targets: macOS, Linux, Windows, WASM)
- Node.js test matrix (3 versions × 3 OSes)
- Coverage (Rust via cargo-llvm-cov + JS via Vitest, uploaded to Codecov)
* Add release and automation workflows (#11)
* ci: add release workflow with Cargo.toml version sync
Changesets-driven release flow:
- On develop push: create version PR, then release PR (develop → main)
- On main push: build all 9 targets and publish to npm with provenance
- scripts/sync-cargo-version.js: keep Cargo.toml version in sync with
package.json after changeset version bumps
* ci: add CodeQL security analysis
Analyze JavaScript/TypeScript and Rust code for security vulnerabilities.
Runs on PRs, develop pushes, and weekly schedule (Monday 6 AM UTC).
* ci: add CodSpeed benchmark workflow
Continuous performance tracking via CodSpeed. Runs JS benchmarks on
develop pushes and PRs when crate or bench files change.
Rust benchmarks will be added when the bench crate is introduced.
* ci: add self-hosted Renovate workflow
Automated dependency updates via self-hosted Renovate bot.
Runs weekly on Sunday 19:00 UTC (Monday 4:00 JST).
* test(bench): add initial benchmark file
Minimal benchmark to satisfy CodSpeed workflow. Real compression
benchmarks will be added alongside algorithm implementations.
* Implement zstd compression and decompression (#12)
* feat(zstd): implement zstd compression and decompression
Add zstdCompress() and zstdDecompress() functions powered by the zstd
C library via the zstd Rust crate.
- Compression levels 1-22 (default: 3)
- Auto-detection of frame content size for efficient decompression
- zstdDecompressWithCapacity() for data exceeding 256 MB default limit
- Rust unit tests for round-trip, empty data, large data, and levels
* test(zstd): add JS tests and benchmarks for zstd
- __test__/zstd.spec.ts: 11 tests covering round-trip, levels, errors
- __test__/esm-import.mjs: updated with zstd round-trip smoke test
- __test__/index.bench.ts: compress/decompress benchmarks at 150B/10KB/1MB
- index.mjs: export zstd functions
* chore: update napi-rs generated files and add changeset
- Regenerate index.js, index.d.ts, browser.js with zstd exports
- Add changeset for minor version bump
* ci: add WASI SDK setup for zstd WASM builds
The zstd crate compiles C code via zstd-sys, which requires a proper
WASI sysroot when targeting wasm32-wasip1-threads. Without it, clang
picks up system glibc headers and fails with missing
'bits/libc-header-start.h'.
Use bytecodealliance/actions/wasi-sdk/setup to install WASI SDK and
set CC, AR, CFLAGS, and WASI_SYSROOT environment variables for the
WASM build step in both CI and release workflows.
* Fix CodeRabbit review findings from #12 (#22)
* fix(zstd): validate capacity parameter in zstdDecompressWithCapacity
Reject negative, NaN, and Infinity values before casting f64 to usize.
Without validation, a negative f64 wraps to a very large usize, which
could cause memory exhaustion.
Closes #17
* fix(ci): use RUNNER_TEMP for WASI SDK installation
Replace /opt with $RUNNER_TEMP for WASI SDK extraction. /opt may not
be writable on all GitHub Actions runner configurations.
Also switch curl from -sL to -fsSL to fail on HTTP errors.
Closes #18
* docs: add SECURITY.md (#23)
* docs: add SECURITY.md
* docs: address review feedback on SECURITY.md
- Replace vague "extremely large size" with "massive size" for clarity
- Unify SLA day units by removing "business" qualifier from initial
assessment timeline for consistency
* docs: add CONTRIBUTING.md (#24)
* test(zstd): expand test coverage and benchmarks (#25)
* test(zstd): expand test coverage and benchmarks
Add comprehensive JS tests covering compression level boundaries (0,
negative, 22), decompressWithCapacity edge cases (insufficient, oversized,
invalid inputs), and round-trip verification with random data.
Expand benchmarks with random (incompressible) and realistic (JSON, text)
data variants alongside existing patterned data.
Update Rust doc comments to document negative compression levels and
level 0 behavior. Add Rust tests for boundary levels. Update changeset
to include all public API functions.
Closes #19
* chore: update napi-rs generated type definitions
Regenerate index.d.ts to reflect updated doc comments for
zstd_compress documenting negative levels and level 0 behavior.
* feat(zstd): add streaming compression/decompression API (#26)
Implement streaming zstd compression and decompression using the Web
Streams API (TransformStream). The Rust layer exposes ZstdCompressContext
and ZstdDecompressContext classes with transform/flush/finish methods
using zstd::stream::raw for stateful chunked processing.
The JS layer wraps these in TransformStream via createZstdCompressStream()
and createZstdDecompressStream(), available from both ESM and CJS entry
points. Streaming output is fully interoperable with one-shot APIs.
Add comprehensive tests covering chunked round-trips, various chunk
sizes, large data (1MB), random data, and one-shot/streaming interop.
Closes #5
* docs: add README and verify npm package readiness (#28)
* docs: add README and verify npm package readiness
Add comprehensive README with project description, installation guide,
quick-start examples (one-shot and streaming), full API reference,
supported algorithms table, platform compatibility matrix, and
benchmark results.
Verify npm package with publint (no errors). Package includes all
required files: index.js, index.mjs, index.d.ts, index.d.mts,
streams.js, streams.d.ts, browser.js, LICENSE, and README.md.
Closes #8
* docs(readme): address review feedback
- Add bun installation instructions
- Replace streaming example with self-contained runnable snippet
* fix(ci): resolve release workflow failures (#29)
* fix(ci): resolve release workflow failures
- Generate npm platform package directories for all 9 targets
- Add optionalDependencies to root package.json for platform binaries
- Fix changeset version command argument parsing in release workflow
by using a dedicated npm script instead of inline shell command
Closes #27
* fix(ci): remove optionalDependencies from committed package.json
The optionalDependencies for platform-specific packages are added
at publish time by the prepublishOnly hook (napi prepublish -t npm),
not in the committed source. Having them in package.json breaks
pnpm install --frozen-lockfile since the packages are not yet on npm.
* chore(release): version packages (#30)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(ci): regenerate napi bindings for v0.2.0 and fix version workflow (#34)
- Regenerate index.js with v0.2.0 version strings
- Update npm platform package versions to 0.2.0
- Add Rust toolchain to version job for napi build during version bump
- Add napi version, build, and format to changeset:version script
- Fix version test to read from package.json dynamically
* feat(gzip): add gzip and deflate compression support (#33)
Add gzip and raw deflate compression/decompression with one-shot and
streaming APIs. Includes interoperability tests with Node.js built-in
zlib module.
Closes #6
* feat(brotli): add brotli compression support (#32)
* feat(brotli): add brotli compression support
Add brotli compression/decompression with one-shot and streaming APIs.
Quality levels 0-11 (default: 6). Includes interoperability tests with
Node.js built-in zlib brotli module.
Closes #7
* test(brotli): add brotli compression benchmarks
Add brotli compress/decompress benchmarks matching the zstd benchmark
structure for patterned, random, and realistic data at various sizes.
* test(gzip): increase timeout for 1MB round-trip tests
Extend timeout from 5s to 30s for gzip and deflate 1MB data tests
to prevent CI failures on slower runners.
* test: increase timeout for all 1MB tests
Set 30s timeout on all 1MB data round-trip and streaming tests to
prevent failures on slower CI runners (Windows).
* docs(readme): update API reference for gzip and brotli (#36)
Add gzip, deflate, and brotli to the API reference tables. Update
supported algorithms table to reflect all three algorithms are now
available with both one-shot and streaming APIs.
* chore(deps): update taiki-e/install-action digest to c12d62a (#20)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(exports): add ./streams subpath export to package.json (#58)
Add a `./streams` subpath export so users can import streaming APIs
directly via `import { ... } from 'zflate/streams'`.
Closes #48
* fix(gzip): add compression level validation for gzip and deflate (#60)
Validate that the compression level parameter is within the valid range
(0-9) for gzip_compress, deflate_compress, GzipCompressContext, and
DeflateCompressContext. Return InvalidArg error if out of range, matching
the existing validation pattern in brotli_compress.
Closes #38
* fix(gzip): add decompression size limits to gzip and deflate (#64)
* fix(gzip): add decompression size limits to gzip and deflate
Add MAX_DECOMPRESSED_SIZE (256 MB) to gzip_decompress and
deflate_decompress to prevent zip bomb attacks. Both functions now use
chunked reading with size checking instead of unbounded read_to_end().
Add gzipDecompressWithCapacity and deflateDecompressWithCapacity for
explicit control over the maximum decompressed size, matching the
existing zstd and brotli patterns.
Closes #37
* chore(napi): regenerate bindings for new WithCapacity functions
* feat(api): accept Uint8Array input in all compression functions (#69)
Change all input parameters from Buffer to Either<Buffer, Uint8Array>
across all one-shot and streaming functions. This generates
Buffer | Uint8Array TypeScript types, enabling browser/Deno/Bun users
to pass Uint8Array directly without type errors.
Return types remain Buffer. No runtime behavior changes for existing
Buffer usage.
Closes #42
* docs: enhance README with examples, comparison table, and migration guides (#63)
* test: add cargo-fuzz targets for decompression fuzzing (#71)
* test: add edge case and error handling tests (#65)
* test: expand Node.js zlib interop tests to brotli and zstd (#67)
* ci: add hand-written JS files to workflow path filters (#56)
* ci(codeql): suppress rust/access-invalid-pointer false positives (#57)
* fix(dx): add ./streams subpath export to package.json (#59)
* ci: add cargo-audit to dependency audit pipeline (#61)
* feat(core): add unified auto-detect decompression API (#72)
Add decompress() function that auto-detects compression format from
magic bytes (zstd: 0xFD2FB528, gzip: 0x1F8B) and falls back to brotli
heuristic detection. Add detectFormat() for format identification
without decompression.
Supported formats: zstd, gzip, brotli. Raw deflate cannot be
auto-detected due to lack of magic bytes.
Closes #16
* ci(dx): add @arethetypeswrong/cli type validation (#62)
* perf(bench): add competitive benchmarks across all algorithms (#70)
* perf(wasm): add WASM binary size reporting to CI (#66)
* fix(exports): add missing ESM exports for new functions (#74)
Add decompress, detectFormat, gzipDecompressWithCapacity, and
deflateDecompressWithCapacity to index.mjs ESM export list. These
functions were available via CJS but missing from ESM imports.
Update ESM smoke test to cover all WithCapacity functions, decompress,
and detectFormat.
Closes #73
* feat(node): add Node.js stream.Transform compatibility layer (#76)
Add `zflate/node` subpath export with stream.Transform factories for all
compression algorithms (zstd, gzip, deflate, brotli), enabling use with
Node.js stream.pipeline() and pipe-based workflows.
Closes #43
* fix(brotli): use consistent error message for capacity exceeded (#78)
Change brotli_decompress_with_capacity error message from
"Destination buffer is too small" to "brotli decompress exceeded
maximum size of {} bytes" to match the pattern used by all other
decompress functions.
Closes #77
* fix(docs): remove merge conflict marker and stale src/ references (#80)
* feat(core): add async compression/decompression API (#82)
Add async versions of all one-shot compression/decompression functions
using napi-rs AsyncTask with the Task trait. This runs compression on
the libuv thread pool without blocking the Node.js event loop.
New functions: zstdCompressAsync, zstdDecompressAsync,
gzipCompressAsync, gzipDecompressAsync, deflateCompressAsync,
deflateDecompressAsync, brotliCompressAsync, brotliDecompressAsync.
Closes #40
* feat(zstd): add dictionary compression API (#83)
Add zstd dictionary training, compression, and decompression functions:
- zstdTrainDictionary: train a dictionary from sample data
- zstdCompressWithDict: compress with a pre-trained dictionary
- zstdDecompressWithDict: decompress data compressed with a dictionary
Enable the zdict_builder feature for the zstd crate to support
dictionary training. Include Rust unit tests and Vitest integration
tests verifying round-trip correctness and improved compression ratios
for small, similar data.
Closes #41
* docs: remove Cloudflare Workers from supported platforms (#91)
* ci: include Node.js WASM loader files in CI artifact (#90)
* test(wasm): add Node.js WASM compatibility test with native parity verification (#92)
* refactor(core): introduce thiserror for structured error types (#93)
Replace manual `Error::new(Status::..., ...)` calls with structured
`ZflateError` enum variants across all compression modules. Error
messages remain identical to preserve backward compatibility.
Closes #51
* test(deno): add Deno WASM smoke test (#96)
* test(bun): add Bun WASM smoke test (#95)
* test(browser): add Playwright browser WASM test (#94)
* ci(security): tighten cargo-deny bans and sources to deny mode (#108)
* ci: pin dtolnay/rust-toolchain to SHA digest (#107)
* fix(core): add decompression size limits to streaming contexts (#109)
Add cumulative output size tracking to all streaming decompression
contexts (ZstdDecompressContext, GzipDecompressContext,
DeflateDecompressContext, BrotliDecompressContext).
Each context now enforces MAX_DECOMPRESSED_SIZE (256 MB) across
transform() calls, preventing decompression bomb attacks that could
cause unbounded memory growth.
Closes #97
* feat(core): narrow detectFormat return type to string literal union (#111)
Use napi-rs string_enum to return CompressionFormat instead of plain
string, generating a const enum with 'zstd' | 'gzip' | 'brotli' |
'unknown' values in TypeScript.
Closes #103
* perf(core): optimize buffer allocation in compression and decompression (#110)
* perf(core): optimize buffer allocation in compression and decompression
Increase decompression read buffer from 4KB to 32KB in gzip, brotli,
and brotli_stream modules to reduce syscall overhead during chunked
reads. Pre-allocate compression output Vecs with input.len() capacity
for gzip and deflate encoders (both sync and async) to avoid repeated
reallocations during compression.
Closes #100
* revert: keep BUFFER_SIZE at 4096 to avoid decompress regression
The 32KB buffer caused 21 benchmark regressions in CodSpeed due to
increased stack zeroing cost. Revert to 4KB. The Vec::with_capacity
optimization for compression output is retained.
* perf(core): replace Vec clone and split_off with std::mem::take in streaming contexts (#112)
Use std::mem::take() instead of .get_ref().clone() + .get_mut().clear()
in brotli streaming contexts, and instead of .split_off(0) in gzip/deflate
streaming contexts. This avoids unnecessary memory allocation and copying
by moving ownership of the buffer contents in-place.
Closes #99
* fix(zstd): add compression level validation (#113)
Validate that the compression level is within the valid zstd range
(-131072 to 22) before passing it to the zstd library. This provides
clear error messages instead of opaque library errors.
Validation is added to:
- zstd_compress (sync)
- zstd_compress_async
- ZstdCompressContext::new (streaming)
- zstd_compress_with_dict
Closes #81
* feat(zstd): add streaming dictionary compression/decompression (#114)
Add ZstdCompressDictContext and ZstdDecompressDictContext classes that
accept a pre-trained dictionary for streaming compression/decompression.
Include Web Streams (createZstdCompressDictStream/createZstdDecompressDictStream)
and Node.js Transform (createZstdCompressDictTransform/createZstdDecompressDictTransform)
wrappers, type definitions, ESM exports, and streaming dict round-trip tests.
Closes #104
* perf(core): switch flate2 backend from miniz_oxide to zlib-rs (#115)
Replace the `rust_backend` (miniz_oxide) feature with `zlib-rs` for
2-3x faster gzip/deflate compression. zlib-rs is a pure-Rust
implementation that works on all targets including WASM, is ISRG
audited, and is on track to become flate2's default backend.
Relax compression level size ordering test since zlib-rs may produce
slightly different output sizes per level than miniz_oxide — round-trip
correctness is what matters.
Closes #98
* perf(zstd): enable multi-threaded compression for native targets (#116)
Add `zstdmt` feature for non-WASM targets via conditional dependency.
This enables zstd's internal thread pool for compression, improving
performance by 20-80% for large inputs on multi-core systems.
WASM targets are excluded because wasi-threads is experimental and
the interaction with napi-rs worker-based threading is untested.
Closes #102
* docs: comprehensive README update with missing APIs and SVG benchmark (#118)
* fix(docs): correct benchmark environment label to Apple M2 / Node.js v22 (#119)
* perf(wasm): add --converge flag to wasm-opt optimization (#122)
Re-run optimization passes until no further gains are achieved.
Increase timeout to 5 minutes to accommodate additional passes.
Closes #121
* perf(core): use 4x input size for decompression output pre-allocation (#123)
Compressed data typically decompresses to several times its original
size. Pre-allocating with input.len() alone leads to frequent
reallocations during decompression. Using input.len() * 4 as the
initial capacity provides a better heuristic while the existing
.min() caps continue to prevent excessive allocation.
Closes #120
* ci: migrate release workflow to npm trusted publishing (OIDC) (#68)
* ci: migrate release workflow to npm trusted publishing (OIDC)
Remove NPM_TOKEN/NODE_AUTH_TOKEN secret dependency and rely on
OIDC token exchange via id-token:write permission (already set).
Trusted publishing must be configured on npmjs.com for each package
before the next release.
* fix(ci): restore pinned SHA for dtolnay/rust-toolchain
Keep the pinned commit hash consistent with ci.yml and codspeed.yml
for supply chain security. The rust-toolchain change was out of scope
for the trusted publishing migration.
* fix(gzip): support concatenated gzip streams in decompression (#131)
Replace GzDecoder with MultiGzDecoder from flate2 to correctly handle
concatenated gzip members in both one-shot and streaming decompression.
* fix(streaming): add finish-guard state tracking for brotli and zstd compress contexts (#132)
* feat(core): add missing async variants for one-shot APIs (#133)
* feat(streaming): add auto-detect decompression streams (#145)
* docs: add troubleshooting section to CONTRIBUTING.md (#146)
* docs: add CODE_OF_CONDUCT.md (#149)
* ci: apply least-privilege permissions to workflows (#147)
* chore: improve package.json metadata and exports map (#148)
* ci: clean up CI workflow configuration (#150)
* feat(core): expose CRC32 utility function (#134)
* feat(core): expose CRC32 utility function
* test(crc32): use skipIf for conditional Node.js zlib interop test
* ci(testing): add coverage thresholds to vitest config (#151)
* ci: add aggregate gate job to CI workflow (#152)
* feat(streaming): add configurable maxOutputSize to decompression contexts (#153)
Add optional maxOutputSize parameter to all streaming decompression
context constructors (GzipDecompressContext, DeflateDecompressContext,
ZstdDecompressContext, BrotliDecompressContext, ZstdDecompressDictContext).
When not specified, the default 256 MB limit is used.
The parameter is also passed through all streaming factory functions
(createXxxDecompressStream, createXxxDecompressTransform, and
createDecompressStream/createDecompressTransform).
Closes #129
* ci: add environment protection for npm publish (#154)
* ci: add cargo-shear for unused dependency detection (#155)
* ci: add WASM binary size tracking on pull requests (#156)
* ci: add scripts/ to CI workflow path triggers (#159)
* fix(wasm): add JS-side streaming adapters for browser WASM environment (#158)
Replace native streaming contexts with JS-side adapters in the browser
entry point to work around WebAssembly.Memory growth invalidating
ArrayBuffer views. Adapters accumulate chunks and delegate to one-shot
APIs, maintaining the same transform()/flush()/finish() API surface.
- browser-streaming.js: adapter classes for all 10 streaming contexts
- browser-entry.js: re-exports WASM APIs with adapter overrides
- package.json: browser condition points to browser-entry.js
- e2e: streaming tests enabled for browser WASM
* feat(gzip): add header metadata support for compression and reading (#160)
Closes #130
* chore(renovate): run daily and remove PR concurrent limit (#164)
* chore(renovate): run daily and remove PR concurrent limit
Closes #163
* chore(renovate): disable lock file maintenance
* chore(deps): update codspeedhq/action digest to 1c8ae48 (#161)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dtolnay/rust-toolchain digest to 631a55b (#162)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to 3869755 (#165)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to b988c18 (#166)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(bench): exclude third-party benchmarks from CodSpeed tracking (#169)
* fix(bench): exclude third-party benchmarks from CodSpeed tracking
Split benchmark files into zflate-only (*.bench.ts) and comparison
(*.compare.bench.ts) variants. CodSpeed now only tracks zflate's own
benchmarks, eliminating false regressions from pako/fflate/node:zlib
that are caused by GitHub Actions CPU heterogeneity affecting Callgrind
instruction counting.
Closes #167
* fix(bench): use deterministic random data in CodSpeed benchmarks
Replace non-deterministic randomBytes() with a seeded LCG to ensure
reproducible benchmark inputs across runs, reducing CodSpeed instruction
count variance.
* feat(bench): migrate from CodSpeed to Bencher.dev for regression detection (#170)
Replace CodSpeed simulation mode (Callgrind instruction counting) with
Bencher.dev (statistical wall-clock analysis). CodSpeed's instruction
counting was fundamentally incompatible with this napi-rs project due to
GitHub Actions CPU heterogeneity and glibc's CPU-adaptive malloc.
Changes:
- Add Bencher workflow with t-test threshold for regression detection
- Add vitest-to-bmf.js conversion script for vitest bench output
- Remove CodSpeed workflow and @codspeed/vitest-plugin dependency
- Remove CodSpeed plugin from vitest config
Closes #168
* feat(bench): switch to CodSpeed Walltime mode for accurate benchmarks (#172)
Replace Bencher.dev with CodSpeed Walltime mode (Macro Runners) which
runs benchmarks on dedicated bare-metal hardware, eliminating GitHub
Actions CPU variance. CodSpeed also provides native vitest integration
via @codspeed/vitest-plugin, removing the need for conversion scripts.
Key change: `mode: walltime` instead of the previous `mode: simulation`
which used Callgrind instruction counting.
Closes #171
* refactor(bench): split benchmark files per algorithm for granular CodSpeed reporting (#174)
Split index.bench.ts (mixed zstd+brotli) into separate zstd.bench.ts
and brotli.bench.ts so CodSpeed PR reports only show benchmarks relevant
to the changed algorithm. Extract shared test fixtures into
bench-fixtures.ts and add realistic data benchmarks (JSON, text) to
gzip and deflate for consistent coverage across all four algorithms.
Closes #173
* perf(core): eliminate double-copy buffer pattern in decompression (#178)
* perf(core): eliminate double-copy buffer pattern in decompression
Replace the manual 4 KiB read loop (stack buffer → Vec copy) with
read_to_end + Take, which writes directly into Vec spare capacity.
This eliminates one full copy of every decompressed byte and uses
adaptive buffer sizing instead of fixed 4 KiB chunks.
Affected: gzip, deflate, brotli (sync + async, default + with-capacity)
Closes #175
* fix(core): use saturating_mul for init capacity and update size limit tests
Address CodeRabbit review feedback:
- Replace len() * 4 with len().saturating_mul(4) across all call sites
to prevent overflow on 32-bit targets (wasm32-wasip1-threads)
- Replace old chunk-loop size limit tests with Take+read_to_end tests
that exercise the actual decompression pattern (avoiding napi::Error
symbol resolution issues in standalone test binaries)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks (#179)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks
- Add bench:ci script that excludes comparison benchmarks
- Simplify CodSpeed workflow to use bench:ci instead of explicit file list
- Use glob pattern for bench file path triggers
- Add crates/bench workspace member with Criterion benchmarks
(gzip, deflate, brotli, zstd) using codspeed-criterion-compat
- Add rust-benchmarks job to CodSpeed workflow (simulation mode)
Closes #177
* fix(ci): add Cargo.toml, Cargo.lock, package.json to CodSpeed path triggers
Ensures CodSpeed benchmarks run when workspace config or dependency
lockfiles change, which could affect benchmark behavior.
* feat(core): add LZ4 frame compression support (#180)
* feat(core): add LZ4 frame compression support
Add LZ4 as a fourth compression algorithm using lz4_flex (pure Rust):
- Bulk API: lz4Compress, lz4Decompress, lz4DecompressWithCapacity
- Async API: lz4CompressAsync, lz4DecompressAsync, lz4DecompressWithCapacityAsync
- Streaming: Lz4CompressContext, Lz4DecompressContext
- Web Streams: createLz4CompressStream, createLz4DecompressStream
- Format detection: detectFormat returns "lz4" for LZ4 frame magic bytes
- Auto-decompress: decompress/decompressAsync and createDecompressStream
now handle LZ4 data automatically
Closes #176
* fix(core): add Read import to LZ4 test modules
Tests use read_to_end which requires the Read trait in scope.
* refactor(core): validate capacity as non-negative integer (#183)
* refactor(core): validate capacity as non-negative integer across all algorithms
Extract shared validate_capacity() that rejects NaN, Infinity, negative,
fractional values, and values exceeding usize::MAX. Applied to all 10
DecompressWithCapacity functions (sync + async × 5 algorithms).
Closes #181
* test: add regression tests for fractional and oversized capacity
* perf(lz4): implement incremental streaming for LZ4 compression (#184)
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with cursor-based incremental output in
Lz4CompressContext. The FrameEncoder now persists across transform()
calls, emitting compressed blocks as they become available instead of
buffering all input until finish().
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with FrameEncoder-based incremental output
in Lz4CompressContext. Compressed blocks are emitted as internal buffers
fill, and old bytes are drained to bound memory usage.
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* chore(deps): update taiki-e/install-action digest to 0620367 (#185)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust crate lz4_flex to 0.13 (#186)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore: rename package from zflate to comprs (#190)
* chore: rename package from zflate to comprs
Rename npm package, Rust crates, napi binary name, platform packages,
and all references from "zflate" to "comprs". The name "comprs" conveys
"compress" with a Rust flavor, is available on npm, and avoids the
typosquat protection that blocks "zflate".
Changes across 70 files:
- npm: comprs, comprs-darwin-arm64, comprs-wasm32-wasi, etc.
- Rust: comprs (core), comprs-bench
- napi binaryName: comprs (→ comprs.*.node, comprs.*.wasm)
- README, workflows, changesets, tests, benchmarks
GitHub repository URL (derodero24/zflate) is preserved.
Closes #189
* chore: update GitHub repo URLs to derodero24/comprs
Update all repository references to match the planned GitHub repo
rename from derodero24/zflate to derodero24/comprs.
* chore(core): rename ZflateError to ComprsError and update bench vars (#192)
Rename internal Rust error type `ZflateError` → `ComprsError` across
all source files and update benchmark variable names `*_ZFLATE` →
`*_COMPRS` for consistency with the package rename in #190.
Closes #191
* fix: add missing LZ4 docs, browser exports, Node.js transforms, and publish files (#194)
- Add LZ4 one-shot and streaming context exports to browser-entry.js
- Add Lz4CompressContext/Lz4DecompressContext adapters to browser-streaming.js
- Add createLz4CompressTransform/createLz4DecompressTransform to node.js
- Add LZ4 type declarations to node.d.ts
- Add 'lz4' case to createDecompressContext in node.js
- Add browser-entry.js and browser-streaming.js to package.json files array
- Document LZ4 API in README.md (one-shot, async, streaming, comparison)
- Fix streaming import path in README.md (comprs → comprs/streams)
- Update auto-detect descriptions to include LZ4
- Add "lz4" to package.json keywords and description
- Add LZ4 changeset for upcoming release
Closes #193
* fix: add missing LZ4 exports to ESM entry point and complete API docs (#196)
- Add LZ4 one-shot APIs, Context classes, and streaming functions to
index.mjs (ESM users could not access LZ4 at all)
- Add createLz4CompressStream/createLz4DecompressStream to index.mjs
- Add gzipCompressWithHeader and gzipReadHeader to browser-entry.js
- Document gzipCompressWithHeader, gzipReadHeader in README
- Document crc32 and version() utilities in README
- Document createDecompressStream and createDecompressTransform in README
Closes #195
* fix(docs): add missing lz4 to decompressAsync doc and clarify async section (#198)
The decompressAsync Rust doc comment listed only zstd, gzip, brotli as
supported formats, but the implementation also handles lz4. The README
async section claimed "all one-shot functions" have async variants, which
is inaccurate for utility functions like crc32, version, gzipReadHeader.
Closes #197
* chore(release): version packages (#35)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* perf(ci): remove JS walltime benchmarks from CodSpeed regression detection (#201)
JS walltime benchmarks on standard GitHub Actions runners produce 20-50%
variance due to CPU heterogeneity (AMD EPYC vs Intel Xeon assigned randomly)
and glibc's CPU-adaptive malloc. This caused CodSpeed Performance Analysis to
fail on nearly every PR since #174, including documentation-only changes.
Keep only Rust simulation benchmarks in CodSpeed, which use instruction
counting via Callgrind and produce <1% variance. Also narrow the workflow
path triggers to Rust-only files since JS changes no longer affect the
CodSpeed pipeline.
JS benchmarks remain available locally via `pnpm run bench`.
Closes #200
* ci(release): use granular access token for initial npm publish
The first publish requires an NPM_TOKEN since OIDC trusted publishing
only works for packages that already exist on the registry. After the
initial release, this will be switched back to OIDC.
* chore: add .mcp.json to .gitignore (#205)
Closes #203
* fix(core): use decompress_with_limit in async auto-detect decompression (#216)
Replace manual chunk-read loops with double-copy in DecompressTask for
gzip and brotli branches. Now uses crate::decompress_with_limit(),
matching the pattern already used by the LZ4 branch and per-algorithm
async tasks.
Closes #207
* ci: add browser files to CI path triggers (#217)
Closes #212
* fix(zstd): validate maxDictSize parameter in dictionary training (#219)
* test: update ESM smoke test to cover all exports (#215)
* Add LZ4 edge-case and streaming test coverage (#218)
* test(lz4): add edge-case and streaming test coverage
* test(lz4): fix empty buffer test to match actual lz4 behavior
LZ4 returns empty buffer for empty input (like zstd and deflate),
rather than throwing an error (like gzip and brotli).
* fix(zstd): enforce maxOutputSize in browser dict decompression (#221)
* fix(zstd): enforce maxOutputSize in browser dict decompression
Add zstd_decompress_with_dict_with_capacity Rust function and use it
in browser-streaming.js ZstdDecompressDictContext.flush() when
maxOutputSize is provided.
Closes #208
* chore: regenerate napi-rs binding files
* refactor(core): consolidate MAX_DECOMPRESSED_SIZE constant (#222)
Remove duplicated MAX_DECOMPRESSED_SIZE definitions from zstd, gzip,
brotli_impl, lz4, and detect modules. Change the lib.rs definition to
pub(crate) and reference it via crate::MAX_DECOMPRESSED_SIZE in all
modules.
* chore(release): version packages (#220)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* docs: add benchmark comparison results to README (#224)
Closes #213
* feat(brotli): add dictionary compression and decompression support (#225)
* docs: add brotli dictionary to README and update Node.js transforms (#228)
* docs: readme visual and structural overhaul (#239)
- Center-align hero section with title, tagline, and badges
- Add Node.js version and TypeScript badges
- Add Table of Contents for navigation
- Wrap benchmark tables in collapsible <details> sections
- Replace plain blockquote with GitHub Admonition (> [!TIP])
- Replace raw build target triples with human-readable table
Closes #229
* docs: reposition and enhance comparison table (#240)
- Move comparison table from bottom of README to right after "Why comprs?"
- Split "Streaming" into "Web Streams API" and "Node.js Transform" rows
- Replace ambiguous `✅*` with `⚠️ Experimental*` for node:zlib zstd
- Replace footnote `†` with inline "Chunked†" for pako streaming
- Update Table of Contents to reflect new section order
- Center-align feature columns for better scannability
Closes #236
* docs: improve quick start examples and add runtime-specific usage (#241)
- Condense one-shot examples to single primary zstd block + multi-algorithm block
- Replace impractical manual ReadableStream example with fetch-style streaming
- Add Node.js Transform stream example to Quick Start
- Add brief comment explaining async runs on libuv thread pool
- Consolidate compression levels into compact format
- Add Deno and Bun import examples
Closes #235
* docs: add algorithm selection guide (#242)
- Add "Choosing an Algorithm" section with use case → recommendation table
- Add "Choosing an API mode" subsection for sync/async/streaming/dictionary
- Update Table of Contents
Closes #234
* docs: document browser WASM requirements (SharedArrayBuffer, COOP/COEP) (#243)
- Add SharedArrayBuffer requirement warning with required HTTP headers
- Remove misleading "or import directly from a CDN" claim
- Link to MDN SharedArrayBuffer security requirements
Closes #231
* docs: add SSR framework integration guide (#244)
- Add "Framework Integration (SSR)" subsection under Browser Usage
- Include Next.js serverExternalPackages and Vite SSR external config
- Note automatic WASM fallback on client side
Closes #232
* docs: add fflate migration guide (#245)
- Add "From fflate" migration section with diff example
- Note comprs advantages over fflate (zstd, lz4, brotli, dictionaries, Web Streams)
Closes #237
* docs: add known limitations and performance notes (#246)
- Add "Notes" section with three admonitions covering:
- 256 MB default decompression limit with WithCapacity() workaround
- Small payload WASM overhead advisory
- Brotli decompression performance nuance vs node:zlib
Closes #238
* docs: add all undocumented async and capacity API variants to readme (#247)
- Add zstdDecompressWithDictWithCapacity to sync zstd dictionary table
- Add 9 missing async variants: WithCapacityAsync for all algorithms,
zstd dict async, zstdTrainDictionaryAsync, decompressAsync
- Reorder async table to group by algorithm with capacity/dict variants
Closes #233
* docs: add benchmark comparison SVG charts for cross-algorithm and brotli (#248)
- Add cross-algorithm compression chart (zstd/lz4/gzip/brotli at 10KB)
- Add brotli compression speedup chart (134.7x faster than node:zlib)
- Embed both charts alongside existing gzip chart in Benchmarks section
Closes #230
* docs: fold repetitive API tables into collapsible sections (#250)
- Wrap Dictionary API (zstd + brotli) in <details>
- Replace Async table header with pattern explanation + code example,
fold full function list into <details>
- Add Streaming import path + fold table into <details>
- Keep Node.js Transform code example visible, fold table into <details>
- One-shot core API tables remain visible as primary reference
Closes #249
* chore(deps): update taiki-e/install-action digest to 7627fb4 (#251)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.9 (#252)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update renovatebot/github-action action to v46.1.6 (#254)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency vite to v8.0.3 (#253)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update pnpm to v10.33.0 (#256)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.1 (#255)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency typescript to v6 (#257)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* feat: add interactive playground with GitHub Pages deployment (#259)
Add a browser-based playground for trying comprs compression live.
Includes four algorithms (zstd, gzip, brotli, lz4), level slider,
comparison table, file upload, and download. Deploys to GitHub Pages
via new CI workflow. Uses a JS mock for local dev without a WASM build.
Closes #258
* docs: improve playground visibility and add OGP/SEO meta tags (#261)
Add prominent text link below badges in README header and playground
as a bullet in "Why comprs?". Add Open Graph, Twitter Card, and
canonical URL meta tags to playground/index.html for social sharing.
Closes #260
* fix(playground): handle COI not established on first visit (#263)
On first visit, the COI service worker has not yet reloaded the page,
so crossOriginIsolated is false. Calling the WASM function immediately
throws because SharedArrayBuffer is unavailable. This showed a
misleading "unsupported browser" error even in modern browsers like
Vivaldi, Brave, or Arc.
- Check crossOriginIsolated before invoking WASM
- Show "Enabling security features... reloading" while the service
worker handles the reload
- After 3 s, show a manual Reload button if auto-reload did not happen
- Include the actual error message when WASM init itself fails
- Add .loading-reload-btn styles to style.css
Closes #262
* fix(playground): add Buffer polyfill for emnapi WASM compatibility (#265)
The napi-rs WASM runtime (emnapi) requires globalThis.Buffer when
creating memory views for function arguments/return values. Browsers
do not define Buffer natively, causing zstdCompress and other WASM
calls to throw immediately.
Add the 'buffer' npm package (v6 browser polyfill) to playground
dependencies and set globalThis.Buffer before any WASM calls.
Closes #264
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init (#267)
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init
Static imports are hoisted and evaluated before any module body code runs.
This caused globalThis.Buffer to be unset when comprs-wasm32-wasi evaluated
__emnapiInstantiateNapiModuleSync (which calls emnapi_create_memory_view).
Switch to a dynamic import so the Buffer polyfill assignment executes first.
Closes #266
* fix(playground): allow Vite dev server to serve WASM from parent directory
When comprs.wasi-browser.js is present in the repo root, the WASM file
is fetched at runtime via /@fs/ from outside the playground directory.
Add server.fs.allow so Vite does not block the request with 403.
Only applied when a local WASM build is detected (hasLocalWasm=true).
* fix(playground): fix comparison table label direction and badge layout (#271)
- Rename bar chart column 'Ratio (smaller = better)' to 'Savings'
and numeric column 'Ratio' to 'Saved' to accurately reflect that
the metric is compression savings % (larger = better)
- Change 'best ratio' badge to 'best' (shorter, clearer)
- Add 'fastest' badge to the fastest algorithm row; remove the
duplicate 'Fastest:' text from the footer note
- Change badge column from fixed 72px to auto to prevent overflow
Closes #270
* fix(playground): fix negative ratio text, CSS variable, download label, and file size alert (#273)
- Show "X% larger" instead of "-X% smaller" when compressed output exceeds input
- Fix reload button using undefined var(--accent); use var(--text)/var(--bg) instead
- Add file extension to download button (e.g. "Download .zst") via #download-ext span
- Replace alert() for oversized file with inline error message that auto-dismisses after 5s
Closes #272
* chore(deps): update codecov/codecov-action digest to 75cd116 (#268)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.2 (#269)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(playground): add FileReader error handler and textarea aria-label (#275)
- Add reader.onerror handler to show inline error when file reading fails
instead of silently doing nothing (e.g. on disk error or revoked access)
- Add aria-label to the input textarea for screen reader accessibility
Closes #274
* chore(deps): update dtolnay/rust-toolchain digest to 29eef33 (#276)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to c10b806 (#277)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/configure-pages action to v6 (#283)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/deploy-pages action to v5 (#284)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/upload-pages-artifact action to v4 (#285)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to d858f81 (#278)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust-dependencies to v3.8.4 (#281)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.2 (#280)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.10 (#279)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update codecov/codecov-action action to v6 (#286)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 (#287)
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2
Update @napi-rs/cli from v3.5.1 to v3.6.0 along with @emnapi/core
and @emnapi/runtime from v1.9.1 to v1.9.2 to resolve emnapi version
mismatch that caused WASM build failures.
* fix(deps): align emnapi transitive dependency to v1.9.2
Update the emnapi package (transitive via @napi-rs/cli) from v1.9.1
to v1.9.2, matching @emnapi/core and @emnapi/runtime versions.
* chore(deps): update renovatebot/github-action action to v46.1.7 (#292)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @types/node to v24.12.2 (#291)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(config): update biome.json schema to v2.4.10 (#294)
* chore(deps): update dependency @playwright/test to v1.59.1 (#293)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(core): remove unused wildcard import in crc test module (#295)
* fix(ci): update setup-deno and changesets/action to resolve Renovate lookup failures (#296)
Pin to specific version tags instead of major version floating tags,
which do not exist in these repositories. This allows Renovate to
resolve digests for tracking updates.
- denoland/setup-deno: v2 → v2.0.4
- changesets/action: v1 → v1.7.0
Closes #290
* chore(release): version packages to v0.4.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* refactor: rename npm package to @derodero24/comprs (#298)
Rename all npm packages from `comprs-*` to `@derodero24/comprs-*` to
bypass npm's typosquatting protection that blocked the `comprs` name.
- Root package: comprs → @derodero24/comprs
- 9 platform packages: comprs-{platform} → @derodero24/comprs-{platform}
- Regenerated napi-rs loader (index.js, browser.js) with new names
- Updated release workflow version checks
- Updated README install/import examples
- Updated playground npm link
Closes #297
* fix(playground): update vite alias for renamed WASM package
Update the vite resolve alias from 'comprs-wasm32-wasi' to
'@derodero24/comprs-wasm32-wasi' to match the package rename in #298.
* fix(ci): add publishConfig and permissions for scoped npm publish (#302)
- Add publishConfig.access: "public" to root and all platform
package.json files so napi prepublish can publish scoped packages
- Add contents: write permission to Publish job for GitHub Release
creation by napi prepublish
- Add changeset for v0.4.1 patch release
Closes #301
* chore(release): version packages to v0.4.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: update stale package references after rename to @derodero24/comprs (#307)
Update remaining references from old unscoped names to scoped names:
- browser-streaming.js: comprs-wasm32-wasi → @derodero24/comprs-wasm32-wasi
- playground/main.js: import('comprs') → import('@derodero24/comprs')
- playground/package.json: comprs → @derodero24/comprs dependency
- playground/vite.config.js: Vite alias keys updated
This fixes the GitHub Pages deployment failure caused by Vite being
unable to resolve the old unscoped import in browser-streaming.js.
Closes #306
* fix(playground): regenerate lockfile after package rename
The playground's pnpm-lock.yaml was not updated when the dependency
was renamed from comprs to @derodero24/comprs, causing CI to fail
with frozen-lockfile mismatch.
* docs: fix README package references, streaming note, and WASM size (#311)
* fix(docs): update remaining old package name references in README
- Deno import: npm:comprs → npm:@derodero24/comprs
- Subpath imports: comprs/streams → @derodero24/comprs/streams,
comprs/node → @derodero24/comprs/node
- SSR config: Next.js and Vite external package names
Closes #308
* docs: clarify browser streaming memory behavior
Add note explaining that browser WASM streaming buffers all data in
memory (unlike Node.js which streams incrementally). Also qualify the
"bounded memory" claim in the feature list to specify Node.js.
Closes #309
* docs: add concrete WASM binary size to README
Show approximate optimized WASM size (~2.0 MB raw, ~800 KB gzipped)
directly in README so users can evaluate without navigating to CI.
Closes #310
* fix: update npm badges and sub-package descriptions (#317)
* fix(docs): update npm badges to use scoped package name
The npm version and downloads badges were pointing to the old unscoped
package name "comprs" instead of the scoped "@derodero24/comprs".
Closes #312
* fix: update platform sub-package descriptions to include lz4
All 9 platform sub-packages under npm/ had descriptions listing only
"zstd, gzip, and brotli" but were missing "lz4" which was added to
the root package. Updated to match: "zstd, gzip, brotli, and lz4".
Closes #315
* fix: add missing ESM export and test coverage (#318)
* fix: add missing zstdDecompressWithDictWithCapacity to ESM and browser exports
Add zstdDecompressWithDictWithCapacity to index.mjs destructured exports,
browser-entry.js one-shot re-exports, and ESM smoke test assertions.
Closes #313
* test: add coverage for WithDictWithCapacity functions and Node.js Transform streams
Add tests for zstdDecompressWithDictWithCapacity, brotliDecompressWithDictWithCapacity,
brotliDecompressWithDictWithCapacityAsync, LZ4 Node.js Transform round-trip,
zstd dict Node.js Transform round-trip, and brotli dict Node.js Transform round-trip.
Closes #314
* feat(wasm): replace WASI WASM with wasm-bindgen for browser compatibility (#320)
* refactor(core): extract pure Rust logic into core-lib crate
Split crates/core into two crates:
- crates/core-lib: pure Rust compression logic (no FFI dependencies)
- crates/core: napi-rs thin wrappers delegating to core-lib
This prepares for adding a wasm-bindgen crate that shares the same
core logic, enabling browser usage without SharedArrayBuffer.
Part of #319
* feat(wasm): add wasm-bindgen browser binding crate
Create crates/wasm with wasm-bindgen wrappers over comprs-core,
targeting wasm32-unknown-unknown (no SharedArrayBuffer required).
Includes one-shot and streaming APIs for all algorithms:
zstd, gzip, deflate, brotli, lz4, auto-detect, and crc32.
Part of #319
* feat(wasm): integrate wasm-bindgen build into package distribution
- Add build:wasm-bindgen script using wasm-pack
- Update browser.js to use wasm-bindgen output
- Update browser-streaming.js imports
- Add wasm-bindgen files to npm package
- Update CI with wasm-bindgen build job
- Update release workflow for wasm-bindgen artifacts
- Update playground configuration
- Remove SharedArrayBuffer requirement from README
Part of #319
* fix(ci): resolve CI failures for wasm-bindgen migration
- Remove browser.js from napi generated file diff check (napi build
regenerates it, but we now maintain it manually for wasm-bindgen)
- Remove unused direct compression deps from napi crate (all logic
now lives in core-lib)
- Update e2e browser test to use wasm-bindgen output instead of
WASI browser entry
- Remove COOP/COEP headers from e2e vite config (no longer needed)
Part of #319
* fix(e2e): use manual WASM instantiation for wasm-bindgen in browser test
Vite 8 does not support the TC39 WebAssembly ESM integration proposal.
Import _bg.js directly and instantiate WASM binary manually, following
the same pattern used in rapid-fuzzy.
Also add lz4, crc32, and detectFormat tests to the browser e2e suite.
Part of #319
* fix(e2e): update browser test spec to match wasm-bindgen test suite
Replace streaming tests (removed with WASI migration) with lz4,
crc32, and detectFormat assertions.
Part of #319
* chore: add changeset for v1.0.0 major release (#322)
Closes #321
* chore(release): version packages to v1.0.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: address post-release bugs in v1.0.0 (#327)
- Add missing zstdDecompressWithDictWithCapacityAsync to index.mjs
- Sync crates/core-lib and crates/wasm version to 1.0.0
- Fix sync-cargo-version.js to cover all three Cargo.toml files
- Add missing async functions to README API list
- Add missing ESM smoke test assertion
Closes #326
* chore: add changeset for v1.0.1 patch release (#328)
Covers the bug fixes from #327:
- Missing zstdDecompressWithDictWithCapacityAsync ESM export
- Version mismatch in core-lib and wasm Cargo.toml
- Missing async functions in README API list
* chore(release): version packages to v1.0.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(playground): fix GitHub Pages deployment with wasm-bindgen (#333)
Root cause: napi build/version overwrites browser.js to point at WASI
WASM, and Vite 8 rejects ESM WASM imports from wasm-pack bundler output.
Fixes:
- Add scripts/patch-browser-js.js to restore browser.js after napi
overwrites it (runs in changeset:version and build:wasm-bindgen)
- Add playground/wasm-loader.js for manual WASM instantiation
(Vite 8 ESM WASM workaround, same pattern as rapid-fuzzy)
- Update playground vite.config.js to use wasm-loader
- Remove obsolete Buffer polyfill from playground/main.js
Closes #332
* chore: improve npm discoverability and update CONTRIBUTING.md (#334)
- Add keywords: compress, wasm-bindgen, native, web-streams
- Update CONTRIBUTING.md project structure for three-crate architecture
- Fix dead "GitHub Discussion" reference (Discussions is disabled)
* perf: v1.1.0 performance improvements (#348)
* perf(gzip): use ISIZE field for decompression buffer pre-allocation
Read the gzip footer's ISIZE field (last 4 bytes, little-endian uint32)
to pre-allocate the exact output buffer size instead of using a fixed
4x heuristic. This eliminates Vec re-allocations for most decompression
workloads.
Falls back to 4x heuristic when ISIZE is 0 (empty data or >4GB wrapping)
or when input is too small to be valid gzip.
Closes #335
* perf(zstd): enable multi-threaded compression (zstdmt)
Add zstdmt feature flag to comprs-core, enabled by default in the
napi crate (Node.js) but not in the wasm crate (single-threaded).
This allows zstd to use multiple threads for compression of large data,
providing 2-4x speedup on multi-core systems.
Closes #336
* perf(streaming): reuse internal buffers instead of allocating per transform
Add a reusable output_buf field to all zstd streaming context structs.
Vec::resize reuses existing capacity after the first call, eliminating
the 128KB allocation + zeroing overhead on subsequent transform calls.
The gzip, brotli, and lz4 streaming contexts delegate buffering to
their underlying encoder/decoder inner Vec<u8> (drained via mem::take),
so they do not have the same per-call allocation pattern and are
unaffected.
Closes #337
* perf(zstd): add comparison benchmark against node:zlib
Add zstd.compare.bench.ts comparing comprs zstd performance against
Node.js 22+ built-in zlib.zstdCompressSync/zstdDecompressSync.
Closes #338
* perf(streaming): add Transform stream throughput benchmarks
Add streaming.compare.bench.ts measuring Node.js Transform stream
throughput for comprs vs node:zlib, with various chunk sizes.
Closes #340
* chore: add changeset for v1.1.0 performance improvements (#349)
Covers #335 (gzip ISIZE), #336 (zstdmt), #337 (streaming buffer reuse).
* chore(release): version packages to v1.1.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* chore(deps): update taiki-e/install-action digest to 7a562df (#353)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update embarkstudios/cargo-deny-action digest to 175dc7f (#354)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to 97a5807 (#355)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update renovatebot/github-action action to v46.1.8 (#361)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency lefthook to v2.1.5 (#359)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.11 (#356)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency vite to v8.0.8 (#360)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/cli to v3.6.1 (#357)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.3 (#358)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.4 (#362)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): set minimumReleaseAge to 3 days in Renovate (#363)
Prevent adopting newly released packages that may be yanked or have
undiscovered issues. This applies globally to npm, Cargo, and GitHub
Actions dependencies.
* feat(middleware): add Express HTTP compression middleware (#364)
* feat(middleware): add Express HTTP compression middleware
Add @derodero24/comprs-middleware — the first Express middleware with
zstd support. Includes Accept-Encoding negotiation, configurable
algorithm priority, threshold-based skip, and Content-Type filtering.
- Set up pnpm workspace for multi-package monorepo
- Implement negotiation, compression, and middleware modules
- Add 31 unit and integration tests
- Update CI to lint/test the middleware package
- Update release workflow for multi-package publishing
Closes #343
* fix(middleware): address CodeRabbit review feedback
- Make isCompressibleType return false for missing Content-Type
- Add stream error handler to prevent unhandled exceptions
- Remove wildcard fallback that could override explicit rejections
- Separate middleware publish job for independent releases
- Limit middleware CI tests to ubuntu/node24 only
- Narrow biome noBarrelFile override to middleware package
- Refactor test helper to support HEAD requests
- Add HEAD request skip test
- Remove "first" claim from README descriptions
* fix(middleware): fix description, docs, and add typescript devDep
- Remove "First" claim from package.json description
- Update filter JSDoc to reflect Content-Type skip behavior
- Add typescript to devDependencies for standalone builds
- Fix filter default description in README table
* fix(middleware): update lockfile for typescript devDependency
* fix(middleware): normalize write/end args and preserve callbacks
Properly handle all overload forms of res.write() and res.end():
- Normalize chunk, encoding, and callback from variadic args
- Pass callbacks through to compressStream.write/end
- Handle res.end(callback) without treating function as chunk body
- Use null-check (!=) instead of truthy for chunk to handle empty strings
* chore(release): version packages (#365)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(ci): check all workspace packages for unpublished versions (#367)
* fix(ci): check all workspace packages for unpublished versions
The release PR creation step only checked the core package version,
preventing middleware-only releases from triggering a develop→main PR.
Now checks both @derodero24/comprs and @derodero24/comprs-middleware
versions against npm, creating a release PR if any package is unpublished.
Closes #366
* fix(ci): read package names from manifest and handle npm errors
- Read package names from package.json instead of hardcoding
- Distinguish E404 (unpublished) from network/auth errors
- Fail the job on unexpected npm view errors instead of silently skipping
* feat(middleware): add Fastify and Hono support via subpath exports (#370)
* feat(middleware): add Fastify and Hono support via subpath exports
Restructure @derodero24/comprs-middleware from Express-only to
multi-framework with subpath exports:
- ./express — Express/Connect middleware (refactored from root export)
- ./fastify — Fastify plugin (onSend hook with stream compression)
- ./hono — Hono middleware (sync compression via Web Standards API)
- . — shared utilities (negotiate, types)
Extract framework-agnostic logic into shared.ts. Add 21 new tests
for Fastify (10) and Hono (11), total 53 tests across 4 test files.
Closes #369
* fix(middleware): address CodeRabbit review feedback
- Handle Vary: * without appending Accept-Encoding (RFC 7231)
- Extract shouldSkip helper in Hono adapter to reduce complexity
- Add Content-Length early threshold check in Hono to avoid body read
- Log compression pipeline errors in Fastify (skip premature close)
- Add stream compression behavior comment in Fastify
- Add identity-only Accept-Encoding test for Fastify
- Fix README import examples to use separate blocks per framework
* fix(middleware): case-insensitive Cache-Control check, Hono error handling
- Normalize Cache-Control to lowercase before checking no-transform (all adapters)
- Set Vary header before HEAD early return in Hono
- Wrap Hono body read + compression in try/catch for graceful fallback
* fix(ci): use manifest name and E404 check in publish-middleware job (#373)
Read package name from package.json instead of hardcoding.
Distinguish E404 (unpublished) from network/auth errors to prevent
silent failures during middleware publishing.
Closes #372
* chore(release): version packages (#371)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(ci): add post-release merge-back from main to develop (#375)
After each release, merge main back into develop to prevent history
divergence. Without this, squash-merged release commits on main
cause growing conflicts on the next develop→main release PR.
Closes #374
* chore(deps): update commitlint monorepo to v20.5.3 (#382)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.13 (#381)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/upload-artifact digest to 043fb46 (#376)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to db5fb34 (#377)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update embarkstudios/cargo-deny-action digest to 91bf2b6 (#378)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/setup-node digest to 48b55a0 (#380)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to e46ed2c (#379)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/cli to v3.6.2 (#383)
* chore(deps): update dependency @napi-rs/cli to v3.6.2
* fix(deps): bump @emnapi/core and @emnapi/runtime to ^1.10.0
@napi-rs/cli@3.6.2 pulls in emnapi@1.10.0 transitively but @emnapi/core
and @emnapi/runtime were pinned to ^1.9.2, causing a runtime version
mismatch error during 'napi build --target wasm32-wasip1-threads':
Internal Error: emnapi version mismatch:
emnapi@1.10.0, @emnapi/core@1.9.2, @emnapi/runtime@1.9.2
Bump both peers to ^1.10.0 so all three resolve to the same version.
---------
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore!: drop Node.js 20 support and bump napi ABI to napi9 (#386)
Node.js 20 reached end-of-life on 2026-04-30. Raise the minimum supported
Node.js to 22 (Active LTS) and bump the napi-rs ABI feature from napi6 to
napi9 (Node 18.17+ / 20.3+), which is safe under the new floor.
- Remove Node 20 from CI test matrix
-…
derodero24
added a commit
that referenced
this pull request
Jul 10, 2026
* docs: add SECURITY.md (#23)
* docs: add SECURITY.md
* docs: address review feedback on SECURITY.md
- Replace vague "extremely large size" with "massive size" for clarity
- Unify SLA day units by removing "business" qualifier from initial
assessment timeline for consistency
* docs: add CONTRIBUTING.md (#24)
* test(zstd): expand test coverage and benchmarks (#25)
* test(zstd): expand test coverage and benchmarks
Add comprehensive JS tests covering compression level boundaries (0,
negative, 22), decompressWithCapacity edge cases (insufficient, oversized,
invalid inputs), and round-trip verification with random data.
Expand benchmarks with random (incompressible) and realistic (JSON, text)
data variants alongside existing patterned data.
Update Rust doc comments to document negative compression levels and
level 0 behavior. Add Rust tests for boundary levels. Update changeset
to include all public API functions.
Closes #19
* chore: update napi-rs generated type definitions
Regenerate index.d.ts to reflect updated doc comments for
zstd_compress documenting negative levels and level 0 behavior.
* feat(zstd): add streaming compression/decompression API (#26)
Implement streaming zstd compression and decompression using the Web
Streams API (TransformStream). The Rust layer exposes ZstdCompressContext
and ZstdDecompressContext classes with transform/flush/finish methods
using zstd::stream::raw for stateful chunked processing.
The JS layer wraps these in TransformStream via createZstdCompressStream()
and createZstdDecompressStream(), available from both ESM and CJS entry
points. Streaming output is fully interoperable with one-shot APIs.
Add comprehensive tests covering chunked round-trips, various chunk
sizes, large data (1MB), random data, and one-shot/streaming interop.
Closes #5
* docs: add README and verify npm package readiness (#28)
* docs: add README and verify npm package readiness
Add comprehensive README with project description, installation guide,
quick-start examples (one-shot and streaming), full API reference,
supported algorithms table, platform compatibility matrix, and
benchmark results.
Verify npm package with publint (no errors). Package includes all
required files: index.js, index.mjs, index.d.ts, index.d.mts,
streams.js, streams.d.ts, browser.js, LICENSE, and README.md.
Closes #8
* docs(readme): address review feedback
- Add bun installation instructions
- Replace streaming example with self-contained runnable snippet
* fix(ci): resolve release workflow failures (#29)
* fix(ci): resolve release workflow failures
- Generate npm platform package directories for all 9 targets
- Add optionalDependencies to root package.json for platform binaries
- Fix changeset version command argument parsing in release workflow
by using a dedicated npm script instead of inline shell command
Closes #27
* fix(ci): remove optionalDependencies from committed package.json
The optionalDependencies for platform-specific packages are added
at publish time by the prepublishOnly hook (napi prepublish -t npm),
not in the committed source. Having them in package.json breaks
pnpm install --frozen-lockfile since the packages are not yet on npm.
* chore(release): version packages (#30)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(ci): regenerate napi bindings for v0.2.0 and fix version workflow (#34)
- Regenerate index.js with v0.2.0 version strings
- Update npm platform package versions to 0.2.0
- Add Rust toolchain to version job for napi build during version bump
- Add napi version, build, and format to changeset:version script
- Fix version test to read from package.json dynamically
* feat(gzip): add gzip and deflate compression support (#33)
Add gzip and raw deflate compression/decompression with one-shot and
streaming APIs. Includes interoperability tests with Node.js built-in
zlib module.
Closes #6
* feat(brotli): add brotli compression support (#32)
* feat(brotli): add brotli compression support
Add brotli compression/decompression with one-shot and streaming APIs.
Quality levels 0-11 (default: 6). Includes interoperability tests with
Node.js built-in zlib brotli module.
Closes #7
* test(brotli): add brotli compression benchmarks
Add brotli compress/decompress benchmarks matching the zstd benchmark
structure for patterned, random, and realistic data at various sizes.
* test(gzip): increase timeout for 1MB round-trip tests
Extend timeout from 5s to 30s for gzip and deflate 1MB data tests
to prevent CI failures on slower runners.
* test: increase timeout for all 1MB tests
Set 30s timeout on all 1MB data round-trip and streaming tests to
prevent failures on slower CI runners (Windows).
* docs(readme): update API reference for gzip and brotli (#36)
Add gzip, deflate, and brotli to the API reference tables. Update
supported algorithms table to reflect all three algorithms are now
available with both one-shot and streaming APIs.
* chore(deps): update taiki-e/install-action digest to c12d62a (#20)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(exports): add ./streams subpath export to package.json (#58)
Add a `./streams` subpath export so users can import streaming APIs
directly via `import { ... } from 'zflate/streams'`.
Closes #48
* fix(gzip): add compression level validation for gzip and deflate (#60)
Validate that the compression level parameter is within the valid range
(0-9) for gzip_compress, deflate_compress, GzipCompressContext, and
DeflateCompressContext. Return InvalidArg error if out of range, matching
the existing validation pattern in brotli_compress.
Closes #38
* fix(gzip): add decompression size limits to gzip and deflate (#64)
* fix(gzip): add decompression size limits to gzip and deflate
Add MAX_DECOMPRESSED_SIZE (256 MB) to gzip_decompress and
deflate_decompress to prevent zip bomb attacks. Both functions now use
chunked reading with size checking instead of unbounded read_to_end().
Add gzipDecompressWithCapacity and deflateDecompressWithCapacity for
explicit control over the maximum decompressed size, matching the
existing zstd and brotli patterns.
Closes #37
* chore(napi): regenerate bindings for new WithCapacity functions
* feat(api): accept Uint8Array input in all compression functions (#69)
Change all input parameters from Buffer to Either<Buffer, Uint8Array>
across all one-shot and streaming functions. This generates
Buffer | Uint8Array TypeScript types, enabling browser/Deno/Bun users
to pass Uint8Array directly without type errors.
Return types remain Buffer. No runtime behavior changes for existing
Buffer usage.
Closes #42
* docs: enhance README with examples, comparison table, and migration guides (#63)
* test: add cargo-fuzz targets for decompression fuzzing (#71)
* test: add edge case and error handling tests (#65)
* test: expand Node.js zlib interop tests to brotli and zstd (#67)
* ci: add hand-written JS files to workflow path filters (#56)
* ci(codeql): suppress rust/access-invalid-pointer false positives (#57)
* fix(dx): add ./streams subpath export to package.json (#59)
* ci: add cargo-audit to dependency audit pipeline (#61)
* feat(core): add unified auto-detect decompression API (#72)
Add decompress() function that auto-detects compression format from
magic bytes (zstd: 0xFD2FB528, gzip: 0x1F8B) and falls back to brotli
heuristic detection. Add detectFormat() for format identification
without decompression.
Supported formats: zstd, gzip, brotli. Raw deflate cannot be
auto-detected due to lack of magic bytes.
Closes #16
* ci(dx): add @arethetypeswrong/cli type validation (#62)
* perf(bench): add competitive benchmarks across all algorithms (#70)
* perf(wasm): add WASM binary size reporting to CI (#66)
* fix(exports): add missing ESM exports for new functions (#74)
Add decompress, detectFormat, gzipDecompressWithCapacity, and
deflateDecompressWithCapacity to index.mjs ESM export list. These
functions were available via CJS but missing from ESM imports.
Update ESM smoke test to cover all WithCapacity functions, decompress,
and detectFormat.
Closes #73
* feat(node): add Node.js stream.Transform compatibility layer (#76)
Add `zflate/node` subpath export with stream.Transform factories for all
compression algorithms (zstd, gzip, deflate, brotli), enabling use with
Node.js stream.pipeline() and pipe-based workflows.
Closes #43
* fix(brotli): use consistent error message for capacity exceeded (#78)
Change brotli_decompress_with_capacity error message from
"Destination buffer is too small" to "brotli decompress exceeded
maximum size of {} bytes" to match the pattern used by all other
decompress functions.
Closes #77
* fix(docs): remove merge conflict marker and stale src/ references (#80)
* feat(core): add async compression/decompression API (#82)
Add async versions of all one-shot compression/decompression functions
using napi-rs AsyncTask with the Task trait. This runs compression on
the libuv thread pool without blocking the Node.js event loop.
New functions: zstdCompressAsync, zstdDecompressAsync,
gzipCompressAsync, gzipDecompressAsync, deflateCompressAsync,
deflateDecompressAsync, brotliCompressAsync, brotliDecompressAsync.
Closes #40
* feat(zstd): add dictionary compression API (#83)
Add zstd dictionary training, compression, and decompression functions:
- zstdTrainDictionary: train a dictionary from sample data
- zstdCompressWithDict: compress with a pre-trained dictionary
- zstdDecompressWithDict: decompress data compressed with a dictionary
Enable the zdict_builder feature for the zstd crate to support
dictionary training. Include Rust unit tests and Vitest integration
tests verifying round-trip correctness and improved compression ratios
for small, similar data.
Closes #41
* docs: remove Cloudflare Workers from supported platforms (#91)
* ci: include Node.js WASM loader files in CI artifact (#90)
* test(wasm): add Node.js WASM compatibility test with native parity verification (#92)
* refactor(core): introduce thiserror for structured error types (#93)
Replace manual `Error::new(Status::..., ...)` calls with structured
`ZflateError` enum variants across all compression modules. Error
messages remain identical to preserve backward compatibility.
Closes #51
* test(deno): add Deno WASM smoke test (#96)
* test(bun): add Bun WASM smoke test (#95)
* test(browser): add Playwright browser WASM test (#94)
* ci(security): tighten cargo-deny bans and sources to deny mode (#108)
* ci: pin dtolnay/rust-toolchain to SHA digest (#107)
* fix(core): add decompression size limits to streaming contexts (#109)
Add cumulative output size tracking to all streaming decompression
contexts (ZstdDecompressContext, GzipDecompressContext,
DeflateDecompressContext, BrotliDecompressContext).
Each context now enforces MAX_DECOMPRESSED_SIZE (256 MB) across
transform() calls, preventing decompression bomb attacks that could
cause unbounded memory growth.
Closes #97
* feat(core): narrow detectFormat return type to string literal union (#111)
Use napi-rs string_enum to return CompressionFormat instead of plain
string, generating a const enum with 'zstd' | 'gzip' | 'brotli' |
'unknown' values in TypeScript.
Closes #103
* perf(core): optimize buffer allocation in compression and decompression (#110)
* perf(core): optimize buffer allocation in compression and decompression
Increase decompression read buffer from 4KB to 32KB in gzip, brotli,
and brotli_stream modules to reduce syscall overhead during chunked
reads. Pre-allocate compression output Vecs with input.len() capacity
for gzip and deflate encoders (both sync and async) to avoid repeated
reallocations during compression.
Closes #100
* revert: keep BUFFER_SIZE at 4096 to avoid decompress regression
The 32KB buffer caused 21 benchmark regressions in CodSpeed due to
increased stack zeroing cost. Revert to 4KB. The Vec::with_capacity
optimization for compression output is retained.
* perf(core): replace Vec clone and split_off with std::mem::take in streaming contexts (#112)
Use std::mem::take() instead of .get_ref().clone() + .get_mut().clear()
in brotli streaming contexts, and instead of .split_off(0) in gzip/deflate
streaming contexts. This avoids unnecessary memory allocation and copying
by moving ownership of the buffer contents in-place.
Closes #99
* fix(zstd): add compression level validation (#113)
Validate that the compression level is within the valid zstd range
(-131072 to 22) before passing it to the zstd library. This provides
clear error messages instead of opaque library errors.
Validation is added to:
- zstd_compress (sync)
- zstd_compress_async
- ZstdCompressContext::new (streaming)
- zstd_compress_with_dict
Closes #81
* feat(zstd): add streaming dictionary compression/decompression (#114)
Add ZstdCompressDictContext and ZstdDecompressDictContext classes that
accept a pre-trained dictionary for streaming compression/decompression.
Include Web Streams (createZstdCompressDictStream/createZstdDecompressDictStream)
and Node.js Transform (createZstdCompressDictTransform/createZstdDecompressDictTransform)
wrappers, type definitions, ESM exports, and streaming dict round-trip tests.
Closes #104
* perf(core): switch flate2 backend from miniz_oxide to zlib-rs (#115)
Replace the `rust_backend` (miniz_oxide) feature with `zlib-rs` for
2-3x faster gzip/deflate compression. zlib-rs is a pure-Rust
implementation that works on all targets including WASM, is ISRG
audited, and is on track to become flate2's default backend.
Relax compression level size ordering test since zlib-rs may produce
slightly different output sizes per level than miniz_oxide — round-trip
correctness is what matters.
Closes #98
* perf(zstd): enable multi-threaded compression for native targets (#116)
Add `zstdmt` feature for non-WASM targets via conditional dependency.
This enables zstd's internal thread pool for compression, improving
performance by 20-80% for large inputs on multi-core systems.
WASM targets are excluded because wasi-threads is experimental and
the interaction with napi-rs worker-based threading is untested.
Closes #102
* docs: comprehensive README update with missing APIs and SVG benchmark (#118)
* fix(docs): correct benchmark environment label to Apple M2 / Node.js v22 (#119)
* perf(wasm): add --converge flag to wasm-opt optimization (#122)
Re-run optimization passes until no further gains are achieved.
Increase timeout to 5 minutes to accommodate additional passes.
Closes #121
* perf(core): use 4x input size for decompression output pre-allocation (#123)
Compressed data typically decompresses to several times its original
size. Pre-allocating with input.len() alone leads to frequent
reallocations during decompression. Using input.len() * 4 as the
initial capacity provides a better heuristic while the existing
.min() caps continue to prevent excessive allocation.
Closes #120
* ci: migrate release workflow to npm trusted publishing (OIDC) (#68)
* ci: migrate release workflow to npm trusted publishing (OIDC)
Remove NPM_TOKEN/NODE_AUTH_TOKEN secret dependency and rely on
OIDC token exchange via id-token:write permission (already set).
Trusted publishing must be configured on npmjs.com for each package
before the next release.
* fix(ci): restore pinned SHA for dtolnay/rust-toolchain
Keep the pinned commit hash consistent with ci.yml and codspeed.yml
for supply chain security. The rust-toolchain change was out of scope
for the trusted publishing migration.
* fix(gzip): support concatenated gzip streams in decompression (#131)
Replace GzDecoder with MultiGzDecoder from flate2 to correctly handle
concatenated gzip members in both one-shot and streaming decompression.
* fix(streaming): add finish-guard state tracking for brotli and zstd compress contexts (#132)
* feat(core): add missing async variants for one-shot APIs (#133)
* feat(streaming): add auto-detect decompression streams (#145)
* docs: add troubleshooting section to CONTRIBUTING.md (#146)
* docs: add CODE_OF_CONDUCT.md (#149)
* ci: apply least-privilege permissions to workflows (#147)
* chore: improve package.json metadata and exports map (#148)
* ci: clean up CI workflow configuration (#150)
* feat(core): expose CRC32 utility function (#134)
* feat(core): expose CRC32 utility function
* test(crc32): use skipIf for conditional Node.js zlib interop test
* ci(testing): add coverage thresholds to vitest config (#151)
* ci: add aggregate gate job to CI workflow (#152)
* feat(streaming): add configurable maxOutputSize to decompression contexts (#153)
Add optional maxOutputSize parameter to all streaming decompression
context constructors (GzipDecompressContext, DeflateDecompressContext,
ZstdDecompressContext, BrotliDecompressContext, ZstdDecompressDictContext).
When not specified, the default 256 MB limit is used.
The parameter is also passed through all streaming factory functions
(createXxxDecompressStream, createXxxDecompressTransform, and
createDecompressStream/createDecompressTransform).
Closes #129
* ci: add environment protection for npm publish (#154)
* ci: add cargo-shear for unused dependency detection (#155)
* ci: add WASM binary size tracking on pull requests (#156)
* ci: add scripts/ to CI workflow path triggers (#159)
* fix(wasm): add JS-side streaming adapters for browser WASM environment (#158)
Replace native streaming contexts with JS-side adapters in the browser
entry point to work around WebAssembly.Memory growth invalidating
ArrayBuffer views. Adapters accumulate chunks and delegate to one-shot
APIs, maintaining the same transform()/flush()/finish() API surface.
- browser-streaming.js: adapter classes for all 10 streaming contexts
- browser-entry.js: re-exports WASM APIs with adapter overrides
- package.json: browser condition points to browser-entry.js
- e2e: streaming tests enabled for browser WASM
* feat(gzip): add header metadata support for compression and reading (#160)
Closes #130
* chore(renovate): run daily and remove PR concurrent limit (#164)
* chore(renovate): run daily and remove PR concurrent limit
Closes #163
* chore(renovate): disable lock file maintenance
* chore(deps): update codspeedhq/action digest to 1c8ae48 (#161)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dtolnay/rust-toolchain digest to 631a55b (#162)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to 3869755 (#165)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to b988c18 (#166)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(bench): exclude third-party benchmarks from CodSpeed tracking (#169)
* fix(bench): exclude third-party benchmarks from CodSpeed tracking
Split benchmark files into zflate-only (*.bench.ts) and comparison
(*.compare.bench.ts) variants. CodSpeed now only tracks zflate's own
benchmarks, eliminating false regressions from pako/fflate/node:zlib
that are caused by GitHub Actions CPU heterogeneity affecting Callgrind
instruction counting.
Closes #167
* fix(bench): use deterministic random data in CodSpeed benchmarks
Replace non-deterministic randomBytes() with a seeded LCG to ensure
reproducible benchmark inputs across runs, reducing CodSpeed instruction
count variance.
* feat(bench): migrate from CodSpeed to Bencher.dev for regression detection (#170)
Replace CodSpeed simulation mode (Callgrind instruction counting) with
Bencher.dev (statistical wall-clock analysis). CodSpeed's instruction
counting was fundamentally incompatible with this napi-rs project due to
GitHub Actions CPU heterogeneity and glibc's CPU-adaptive malloc.
Changes:
- Add Bencher workflow with t-test threshold for regression detection
- Add vitest-to-bmf.js conversion script for vitest bench output
- Remove CodSpeed workflow and @codspeed/vitest-plugin dependency
- Remove CodSpeed plugin from vitest config
Closes #168
* feat(bench): switch to CodSpeed Walltime mode for accurate benchmarks (#172)
Replace Bencher.dev with CodSpeed Walltime mode (Macro Runners) which
runs benchmarks on dedicated bare-metal hardware, eliminating GitHub
Actions CPU variance. CodSpeed also provides native vitest integration
via @codspeed/vitest-plugin, removing the need for conversion scripts.
Key change: `mode: walltime` instead of the previous `mode: simulation`
which used Callgrind instruction counting.
Closes #171
* refactor(bench): split benchmark files per algorithm for granular CodSpeed reporting (#174)
Split index.bench.ts (mixed zstd+brotli) into separate zstd.bench.ts
and brotli.bench.ts so CodSpeed PR reports only show benchmarks relevant
to the changed algorithm. Extract shared test fixtures into
bench-fixtures.ts and add realistic data benchmarks (JSON, text) to
gzip and deflate for consistent coverage across all four algorithms.
Closes #173
* perf(core): eliminate double-copy buffer pattern in decompression (#178)
* perf(core): eliminate double-copy buffer pattern in decompression
Replace the manual 4 KiB read loop (stack buffer → Vec copy) with
read_to_end + Take, which writes directly into Vec spare capacity.
This eliminates one full copy of every decompressed byte and uses
adaptive buffer sizing instead of fixed 4 KiB chunks.
Affected: gzip, deflate, brotli (sync + async, default + with-capacity)
Closes #175
* fix(core): use saturating_mul for init capacity and update size limit tests
Address CodeRabbit review feedback:
- Replace len() * 4 with len().saturating_mul(4) across all call sites
to prevent overflow on 32-bit targets (wasm32-wasip1-threads)
- Replace old chunk-loop size limit tests with Take+read_to_end tests
that exercise the actual decompression pattern (avoiding napi::Error
symbol resolution issues in standalone test binaries)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks (#179)
* chore(bench): add bench:ci script and Rust CodSpeed benchmarks
- Add bench:ci script that excludes comparison benchmarks
- Simplify CodSpeed workflow to use bench:ci instead of explicit file list
- Use glob pattern for bench file path triggers
- Add crates/bench workspace member with Criterion benchmarks
(gzip, deflate, brotli, zstd) using codspeed-criterion-compat
- Add rust-benchmarks job to CodSpeed workflow (simulation mode)
Closes #177
* fix(ci): add Cargo.toml, Cargo.lock, package.json to CodSpeed path triggers
Ensures CodSpeed benchmarks run when workspace config or dependency
lockfiles change, which could affect benchmark behavior.
* feat(core): add LZ4 frame compression support (#180)
* feat(core): add LZ4 frame compression support
Add LZ4 as a fourth compression algorithm using lz4_flex (pure Rust):
- Bulk API: lz4Compress, lz4Decompress, lz4DecompressWithCapacity
- Async API: lz4CompressAsync, lz4DecompressAsync, lz4DecompressWithCapacityAsync
- Streaming: Lz4CompressContext, Lz4DecompressContext
- Web Streams: createLz4CompressStream, createLz4DecompressStream
- Format detection: detectFormat returns "lz4" for LZ4 frame magic bytes
- Auto-decompress: decompress/decompressAsync and createDecompressStream
now handle LZ4 data automatically
Closes #176
* fix(core): add Read import to LZ4 test modules
Tests use read_to_end which requires the Read trait in scope.
* refactor(core): validate capacity as non-negative integer (#183)
* refactor(core): validate capacity as non-negative integer across all algorithms
Extract shared validate_capacity() that rejects NaN, Infinity, negative,
fractional values, and values exceeding usize::MAX. Applied to all 10
DecompressWithCapacity functions (sync + async × 5 algorithms).
Closes #181
* test: add regression tests for fractional and oversized capacity
* perf(lz4): implement incremental streaming for LZ4 compression (#184)
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with cursor-based incremental output in
Lz4CompressContext. The FrameEncoder now persists across transform()
calls, emitting compressed blocks as they become available instead of
buffering all input until finish().
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* perf(lz4): implement incremental streaming for LZ4 compression
Replace full-input buffering with FrameEncoder-based incremental output
in Lz4CompressContext. Compressed blocks are emitted as internal buffers
fill, and old bytes are drained to bound memory usage.
Decompression remains buffered due to lz4_flex FrameDecoder requiring
full input (documented in #182).
Closes #182
* chore(deps): update taiki-e/install-action digest to 0620367 (#185)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust crate lz4_flex to 0.13 (#186)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore: rename package from zflate to comprs (#190)
* chore: rename package from zflate to comprs
Rename npm package, Rust crates, napi binary name, platform packages,
and all references from "zflate" to "comprs". The name "comprs" conveys
"compress" with a Rust flavor, is available on npm, and avoids the
typosquat protection that blocks "zflate".
Changes across 70 files:
- npm: comprs, comprs-darwin-arm64, comprs-wasm32-wasi, etc.
- Rust: comprs (core), comprs-bench
- napi binaryName: comprs (→ comprs.*.node, comprs.*.wasm)
- README, workflows, changesets, tests, benchmarks
GitHub repository URL (derodero24/zflate) is preserved.
Closes #189
* chore: update GitHub repo URLs to derodero24/comprs
Update all repository references to match the planned GitHub repo
rename from derodero24/zflate to derodero24/comprs.
* chore(core): rename ZflateError to ComprsError and update bench vars (#192)
Rename internal Rust error type `ZflateError` → `ComprsError` across
all source files and update benchmark variable names `*_ZFLATE` →
`*_COMPRS` for consistency with the package rename in #190.
Closes #191
* fix: add missing LZ4 docs, browser exports, Node.js transforms, and publish files (#194)
- Add LZ4 one-shot and streaming context exports to browser-entry.js
- Add Lz4CompressContext/Lz4DecompressContext adapters to browser-streaming.js
- Add createLz4CompressTransform/createLz4DecompressTransform to node.js
- Add LZ4 type declarations to node.d.ts
- Add 'lz4' case to createDecompressContext in node.js
- Add browser-entry.js and browser-streaming.js to package.json files array
- Document LZ4 API in README.md (one-shot, async, streaming, comparison)
- Fix streaming import path in README.md (comprs → comprs/streams)
- Update auto-detect descriptions to include LZ4
- Add "lz4" to package.json keywords and description
- Add LZ4 changeset for upcoming release
Closes #193
* fix: add missing LZ4 exports to ESM entry point and complete API docs (#196)
- Add LZ4 one-shot APIs, Context classes, and streaming functions to
index.mjs (ESM users could not access LZ4 at all)
- Add createLz4CompressStream/createLz4DecompressStream to index.mjs
- Add gzipCompressWithHeader and gzipReadHeader to browser-entry.js
- Document gzipCompressWithHeader, gzipReadHeader in README
- Document crc32 and version() utilities in README
- Document createDecompressStream and createDecompressTransform in README
Closes #195
* fix(docs): add missing lz4 to decompressAsync doc and clarify async section (#198)
The decompressAsync Rust doc comment listed only zstd, gzip, brotli as
supported formats, but the implementation also handles lz4. The README
async section claimed "all one-shot functions" have async variants, which
is inaccurate for utility functions like crc32, version, gzipReadHeader.
Closes #197
* chore(release): version packages (#35)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* perf(ci): remove JS walltime benchmarks from CodSpeed regression detection (#201)
JS walltime benchmarks on standard GitHub Actions runners produce 20-50%
variance due to CPU heterogeneity (AMD EPYC vs Intel Xeon assigned randomly)
and glibc's CPU-adaptive malloc. This caused CodSpeed Performance Analysis to
fail on nearly every PR since #174, including documentation-only changes.
Keep only Rust simulation benchmarks in CodSpeed, which use instruction
counting via Callgrind and produce <1% variance. Also narrow the workflow
path triggers to Rust-only files since JS changes no longer affect the
CodSpeed pipeline.
JS benchmarks remain available locally via `pnpm run bench`.
Closes #200
* ci(release): use granular access token for initial npm publish
The first publish requires an NPM_TOKEN since OIDC trusted publishing
only works for packages that already exist on the registry. After the
initial release, this will be switched back to OIDC.
* chore: add .mcp.json to .gitignore (#205)
Closes #203
* fix(core): use decompress_with_limit in async auto-detect decompression (#216)
Replace manual chunk-read loops with double-copy in DecompressTask for
gzip and brotli branches. Now uses crate::decompress_with_limit(),
matching the pattern already used by the LZ4 branch and per-algorithm
async tasks.
Closes #207
* ci: add browser files to CI path triggers (#217)
Closes #212
* fix(zstd): validate maxDictSize parameter in dictionary training (#219)
* test: update ESM smoke test to cover all exports (#215)
* Add LZ4 edge-case and streaming test coverage (#218)
* test(lz4): add edge-case and streaming test coverage
* test(lz4): fix empty buffer test to match actual lz4 behavior
LZ4 returns empty buffer for empty input (like zstd and deflate),
rather than throwing an error (like gzip and brotli).
* fix(zstd): enforce maxOutputSize in browser dict decompression (#221)
* fix(zstd): enforce maxOutputSize in browser dict decompression
Add zstd_decompress_with_dict_with_capacity Rust function and use it
in browser-streaming.js ZstdDecompressDictContext.flush() when
maxOutputSize is provided.
Closes #208
* chore: regenerate napi-rs binding files
* refactor(core): consolidate MAX_DECOMPRESSED_SIZE constant (#222)
Remove duplicated MAX_DECOMPRESSED_SIZE definitions from zstd, gzip,
brotli_impl, lz4, and detect modules. Change the lib.rs definition to
pub(crate) and reference it via crate::MAX_DECOMPRESSED_SIZE in all
modules.
* chore(release): version packages (#220)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* docs: add benchmark comparison results to README (#224)
Closes #213
* feat(brotli): add dictionary compression and decompression support (#225)
* docs: add brotli dictionary to README and update Node.js transforms (#228)
* docs: readme visual and structural overhaul (#239)
- Center-align hero section with title, tagline, and badges
- Add Node.js version and TypeScript badges
- Add Table of Contents for navigation
- Wrap benchmark tables in collapsible <details> sections
- Replace plain blockquote with GitHub Admonition (> [!TIP])
- Replace raw build target triples with human-readable table
Closes #229
* docs: reposition and enhance comparison table (#240)
- Move comparison table from bottom of README to right after "Why comprs?"
- Split "Streaming" into "Web Streams API" and "Node.js Transform" rows
- Replace ambiguous `✅*` with `⚠️ Experimental*` for node:zlib zstd
- Replace footnote `†` with inline "Chunked†" for pako streaming
- Update Table of Contents to reflect new section order
- Center-align feature columns for better scannability
Closes #236
* docs: improve quick start examples and add runtime-specific usage (#241)
- Condense one-shot examples to single primary zstd block + multi-algorithm block
- Replace impractical manual ReadableStream example with fetch-style streaming
- Add Node.js Transform stream example to Quick Start
- Add brief comment explaining async runs on libuv thread pool
- Consolidate compression levels into compact format
- Add Deno and Bun import examples
Closes #235
* docs: add algorithm selection guide (#242)
- Add "Choosing an Algorithm" section with use case → recommendation table
- Add "Choosing an API mode" subsection for sync/async/streaming/dictionary
- Update Table of Contents
Closes #234
* docs: document browser WASM requirements (SharedArrayBuffer, COOP/COEP) (#243)
- Add SharedArrayBuffer requirement warning with required HTTP headers
- Remove misleading "or import directly from a CDN" claim
- Link to MDN SharedArrayBuffer security requirements
Closes #231
* docs: add SSR framework integration guide (#244)
- Add "Framework Integration (SSR)" subsection under Browser Usage
- Include Next.js serverExternalPackages and Vite SSR external config
- Note automatic WASM fallback on client side
Closes #232
* docs: add fflate migration guide (#245)
- Add "From fflate" migration section with diff example
- Note comprs advantages over fflate (zstd, lz4, brotli, dictionaries, Web Streams)
Closes #237
* docs: add known limitations and performance notes (#246)
- Add "Notes" section with three admonitions covering:
- 256 MB default decompression limit with WithCapacity() workaround
- Small payload WASM overhead advisory
- Brotli decompression performance nuance vs node:zlib
Closes #238
* docs: add all undocumented async and capacity API variants to readme (#247)
- Add zstdDecompressWithDictWithCapacity to sync zstd dictionary table
- Add 9 missing async variants: WithCapacityAsync for all algorithms,
zstd dict async, zstdTrainDictionaryAsync, decompressAsync
- Reorder async table to group by algorithm with capacity/dict variants
Closes #233
* docs: add benchmark comparison SVG charts for cross-algorithm and brotli (#248)
- Add cross-algorithm compression chart (zstd/lz4/gzip/brotli at 10KB)
- Add brotli compression speedup chart (134.7x faster than node:zlib)
- Embed both charts alongside existing gzip chart in Benchmarks section
Closes #230
* docs: fold repetitive API tables into collapsible sections (#250)
- Wrap Dictionary API (zstd + brotli) in <details>
- Replace Async table header with pattern explanation + code example,
fold full function list into <details>
- Add Streaming import path + fold table into <details>
- Keep Node.js Transform code example visible, fold table into <details>
- One-shot core API tables remain visible as primary reference
Closes #249
* chore(deps): update taiki-e/install-action digest to 7627fb4 (#251)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.9 (#252)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update renovatebot/github-action action to v46.1.6 (#254)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency vite to v8.0.3 (#253)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update pnpm to v10.33.0 (#256)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.1 (#255)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency typescript to v6 (#257)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* feat: add interactive playground with GitHub Pages deployment (#259)
Add a browser-based playground for trying comprs compression live.
Includes four algorithms (zstd, gzip, brotli, lz4), level slider,
comparison table, file upload, and download. Deploys to GitHub Pages
via new CI workflow. Uses a JS mock for local dev without a WASM build.
Closes #258
* docs: improve playground visibility and add OGP/SEO meta tags (#261)
Add prominent text link below badges in README header and playground
as a bullet in "Why comprs?". Add Open Graph, Twitter Card, and
canonical URL meta tags to playground/index.html for social sharing.
Closes #260
* fix(playground): handle COI not established on first visit (#263)
On first visit, the COI service worker has not yet reloaded the page,
so crossOriginIsolated is false. Calling the WASM function immediately
throws because SharedArrayBuffer is unavailable. This showed a
misleading "unsupported browser" error even in modern browsers like
Vivaldi, Brave, or Arc.
- Check crossOriginIsolated before invoking WASM
- Show "Enabling security features... reloading" while the service
worker handles the reload
- After 3 s, show a manual Reload button if auto-reload did not happen
- Include the actual error message when WASM init itself fails
- Add .loading-reload-btn styles to style.css
Closes #262
* fix(playground): add Buffer polyfill for emnapi WASM compatibility (#265)
The napi-rs WASM runtime (emnapi) requires globalThis.Buffer when
creating memory views for function arguments/return values. Browsers
do not define Buffer natively, causing zstdCompress and other WASM
calls to throw immediately.
Add the 'buffer' npm package (v6 browser polyfill) to playground
dependencies and set globalThis.Buffer before any WASM calls.
Closes #264
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init (#267)
* fix(playground): use dynamic import for comprs to ensure Buffer is set before WASM init
Static imports are hoisted and evaluated before any module body code runs.
This caused globalThis.Buffer to be unset when comprs-wasm32-wasi evaluated
__emnapiInstantiateNapiModuleSync (which calls emnapi_create_memory_view).
Switch to a dynamic import so the Buffer polyfill assignment executes first.
Closes #266
* fix(playground): allow Vite dev server to serve WASM from parent directory
When comprs.wasi-browser.js is present in the repo root, the WASM file
is fetched at runtime via /@fs/ from outside the playground directory.
Add server.fs.allow so Vite does not block the request with 403.
Only applied when a local WASM build is detected (hasLocalWasm=true).
* fix(playground): fix comparison table label direction and badge layout (#271)
- Rename bar chart column 'Ratio (smaller = better)' to 'Savings'
and numeric column 'Ratio' to 'Saved' to accurately reflect that
the metric is compression savings % (larger = better)
- Change 'best ratio' badge to 'best' (shorter, clearer)
- Add 'fastest' badge to the fastest algorithm row; remove the
duplicate 'Fastest:' text from the footer note
- Change badge column from fixed 72px to auto to prevent overflow
Closes #270
* fix(playground): fix negative ratio text, CSS variable, download label, and file size alert (#273)
- Show "X% larger" instead of "-X% smaller" when compressed output exceeds input
- Fix reload button using undefined var(--accent); use var(--text)/var(--bg) instead
- Add file extension to download button (e.g. "Download .zst") via #download-ext span
- Replace alert() for oversized file with inline error message that auto-dismisses after 5s
Closes #272
* chore(deps): update codecov/codecov-action digest to 75cd116 (#268)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.2 (#269)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(playground): add FileReader error handler and textarea aria-label (#275)
- Add reader.onerror handler to show inline error when file reading fails
instead of silently doing nothing (e.g. on disk error or revoked access)
- Add aria-label to the input textarea for screen reader accessibility
Closes #274
* chore(deps): update dtolnay/rust-toolchain digest to 29eef33 (#276)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to c10b806 (#277)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/configure-pages action to v6 (#283)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/deploy-pages action to v5 (#284)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/upload-pages-artifact action to v4 (#285)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to d858f81 (#278)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update rust-dependencies to v3.8.4 (#281)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.2 (#280)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.10 (#279)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update codecov/codecov-action action to v6 (#286)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2 (#287)
* chore(deps): update @napi-rs/cli to v3.6.0 and @emnapi/* to v1.9.2
Update @napi-rs/cli from v3.5.1 to v3.6.0 along with @emnapi/core
and @emnapi/runtime from v1.9.1 to v1.9.2 to resolve emnapi version
mismatch that caused WASM build failures.
* fix(deps): align emnapi transitive dependency to v1.9.2
Update the emnapi package (transitive via @napi-rs/cli) from v1.9.1
to v1.9.2, matching @emnapi/core and @emnapi/runtime versions.
* chore(deps): update renovatebot/github-action action to v46.1.7 (#292)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @types/node to v24.12.2 (#291)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(config): update biome.json schema to v2.4.10 (#294)
* chore(deps): update dependency @playwright/test to v1.59.1 (#293)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(core): remove unused wildcard import in crc test module (#295)
* fix(ci): update setup-deno and changesets/action to resolve Renovate lookup failures (#296)
Pin to specific version tags instead of major version floating tags,
which do not exist in these repositories. This allows Renovate to
resolve digests for tracking updates.
- denoland/setup-deno: v2 → v2.0.4
- changesets/action: v1 → v1.7.0
Closes #290
* chore(release): version packages to v0.4.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* refactor: rename npm package to @derodero24/comprs (#298)
Rename all npm packages from `comprs-*` to `@derodero24/comprs-*` to
bypass npm's typosquatting protection that blocked the `comprs` name.
- Root package: comprs → @derodero24/comprs
- 9 platform packages: comprs-{platform} → @derodero24/comprs-{platform}
- Regenerated napi-rs loader (index.js, browser.js) with new names
- Updated release workflow version checks
- Updated README install/import examples
- Updated playground npm link
Closes #297
* fix(playground): update vite alias for renamed WASM package
Update the vite resolve alias from 'comprs-wasm32-wasi' to
'@derodero24/comprs-wasm32-wasi' to match the package rename in #298.
* fix(ci): add publishConfig and permissions for scoped npm publish (#302)
- Add publishConfig.access: "public" to root and all platform
package.json files so napi prepublish can publish scoped packages
- Add contents: write permission to Publish job for GitHub Release
creation by napi prepublish
- Add changeset for v0.4.1 patch release
Closes #301
* chore(release): version packages to v0.4.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: update stale package references after rename to @derodero24/comprs (#307)
Update remaining references from old unscoped names to scoped names:
- browser-streaming.js: comprs-wasm32-wasi → @derodero24/comprs-wasm32-wasi
- playground/main.js: import('comprs') → import('@derodero24/comprs')
- playground/package.json: comprs → @derodero24/comprs dependency
- playground/vite.config.js: Vite alias keys updated
This fixes the GitHub Pages deployment failure caused by Vite being
unable to resolve the old unscoped import in browser-streaming.js.
Closes #306
* fix(playground): regenerate lockfile after package rename
The playground's pnpm-lock.yaml was not updated when the dependency
was renamed from comprs to @derodero24/comprs, causing CI to fail
with frozen-lockfile mismatch.
* docs: fix README package references, streaming note, and WASM size (#311)
* fix(docs): update remaining old package name references in README
- Deno import: npm:comprs → npm:@derodero24/comprs
- Subpath imports: comprs/streams → @derodero24/comprs/streams,
comprs/node → @derodero24/comprs/node
- SSR config: Next.js and Vite external package names
Closes #308
* docs: clarify browser streaming memory behavior
Add note explaining that browser WASM streaming buffers all data in
memory (unlike Node.js which streams incrementally). Also qualify the
"bounded memory" claim in the feature list to specify Node.js.
Closes #309
* docs: add concrete WASM binary size to README
Show approximate optimized WASM size (~2.0 MB raw, ~800 KB gzipped)
directly in README so users can evaluate without navigating to CI.
Closes #310
* fix: update npm badges and sub-package descriptions (#317)
* fix(docs): update npm badges to use scoped package name
The npm version and downloads badges were pointing to the old unscoped
package name "comprs" instead of the scoped "@derodero24/comprs".
Closes #312
* fix: update platform sub-package descriptions to include lz4
All 9 platform sub-packages under npm/ had descriptions listing only
"zstd, gzip, and brotli" but were missing "lz4" which was added to
the root package. Updated to match: "zstd, gzip, brotli, and lz4".
Closes #315
* fix: add missing ESM export and test coverage (#318)
* fix: add missing zstdDecompressWithDictWithCapacity to ESM and browser exports
Add zstdDecompressWithDictWithCapacity to index.mjs destructured exports,
browser-entry.js one-shot re-exports, and ESM smoke test assertions.
Closes #313
* test: add coverage for WithDictWithCapacity functions and Node.js Transform streams
Add tests for zstdDecompressWithDictWithCapacity, brotliDecompressWithDictWithCapacity,
brotliDecompressWithDictWithCapacityAsync, LZ4 Node.js Transform round-trip,
zstd dict Node.js Transform round-trip, and brotli dict Node.js Transform round-trip.
Closes #314
* feat(wasm): replace WASI WASM with wasm-bindgen for browser compatibility (#320)
* refactor(core): extract pure Rust logic into core-lib crate
Split crates/core into two crates:
- crates/core-lib: pure Rust compression logic (no FFI dependencies)
- crates/core: napi-rs thin wrappers delegating to core-lib
This prepares for adding a wasm-bindgen crate that shares the same
core logic, enabling browser usage without SharedArrayBuffer.
Part of #319
* feat(wasm): add wasm-bindgen browser binding crate
Create crates/wasm with wasm-bindgen wrappers over comprs-core,
targeting wasm32-unknown-unknown (no SharedArrayBuffer required).
Includes one-shot and streaming APIs for all algorithms:
zstd, gzip, deflate, brotli, lz4, auto-detect, and crc32.
Part of #319
* feat(wasm): integrate wasm-bindgen build into package distribution
- Add build:wasm-bindgen script using wasm-pack
- Update browser.js to use wasm-bindgen output
- Update browser-streaming.js imports
- Add wasm-bindgen files to npm package
- Update CI with wasm-bindgen build job
- Update release workflow for wasm-bindgen artifacts
- Update playground configuration
- Remove SharedArrayBuffer requirement from README
Part of #319
* fix(ci): resolve CI failures for wasm-bindgen migration
- Remove browser.js from napi generated file diff check (napi build
regenerates it, but we now maintain it manually for wasm-bindgen)
- Remove unused direct compression deps from napi crate (all logic
now lives in core-lib)
- Update e2e browser test to use wasm-bindgen output instead of
WASI browser entry
- Remove COOP/COEP headers from e2e vite config (no longer needed)
Part of #319
* fix(e2e): use manual WASM instantiation for wasm-bindgen in browser test
Vite 8 does not support the TC39 WebAssembly ESM integration proposal.
Import _bg.js directly and instantiate WASM binary manually, following
the same pattern used in rapid-fuzzy.
Also add lz4, crc32, and detectFormat tests to the browser e2e suite.
Part of #319
* fix(e2e): update browser test spec to match wasm-bindgen test suite
Replace streaming tests (removed with WASI migration) with lz4,
crc32, and detectFormat assertions.
Part of #319
* chore: add changeset for v1.0.0 major release (#322)
Closes #321
* chore(release): version packages to v1.0.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix: address post-release bugs in v1.0.0 (#327)
- Add missing zstdDecompressWithDictWithCapacityAsync to index.mjs
- Sync crates/core-lib and crates/wasm version to 1.0.0
- Fix sync-cargo-version.js to cover all three Cargo.toml files
- Add missing async functions to README API list
- Add missing ESM smoke test assertion
Closes #326
* chore: add changeset for v1.0.1 patch release (#328)
Covers the bug fixes from #327:
- Missing zstdDecompressWithDictWithCapacityAsync ESM export
- Version mismatch in core-lib and wasm Cargo.toml
- Missing async functions in README API list
* chore(release): version packages to v1.0.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(playground): fix GitHub Pages deployment with wasm-bindgen (#333)
Root cause: napi build/version overwrites browser.js to point at WASI
WASM, and Vite 8 rejects ESM WASM imports from wasm-pack bundler output.
Fixes:
- Add scripts/patch-browser-js.js to restore browser.js after napi
overwrites it (runs in changeset:version and build:wasm-bindgen)
- Add playground/wasm-loader.js for manual WASM instantiation
(Vite 8 ESM WASM workaround, same pattern as rapid-fuzzy)
- Update playground vite.config.js to use wasm-loader
- Remove obsolete Buffer polyfill from playground/main.js
Closes #332
* chore: improve npm discoverability and update CONTRIBUTING.md (#334)
- Add keywords: compress, wasm-bindgen, native, web-streams
- Update CONTRIBUTING.md project structure for three-crate architecture
- Fix dead "GitHub Discussion" reference (Discussions is disabled)
* perf: v1.1.0 performance improvements (#348)
* perf(gzip): use ISIZE field for decompression buffer pre-allocation
Read the gzip footer's ISIZE field (last 4 bytes, little-endian uint32)
to pre-allocate the exact output buffer size instead of using a fixed
4x heuristic. This eliminates Vec re-allocations for most decompression
workloads.
Falls back to 4x heuristic when ISIZE is 0 (empty data or >4GB wrapping)
or when input is too small to be valid gzip.
Closes #335
* perf(zstd): enable multi-threaded compression (zstdmt)
Add zstdmt feature flag to comprs-core, enabled by default in the
napi crate (Node.js) but not in the wasm crate (single-threaded).
This allows zstd to use multiple threads for compression of large data,
providing 2-4x speedup on multi-core systems.
Closes #336
* perf(streaming): reuse internal buffers instead of allocating per transform
Add a reusable output_buf field to all zstd streaming context structs.
Vec::resize reuses existing capacity after the first call, eliminating
the 128KB allocation + zeroing overhead on subsequent transform calls.
The gzip, brotli, and lz4 streaming contexts delegate buffering to
their underlying encoder/decoder inner Vec<u8> (drained via mem::take),
so they do not have the same per-call allocation pattern and are
unaffected.
Closes #337
* perf(zstd): add comparison benchmark against node:zlib
Add zstd.compare.bench.ts comparing comprs zstd performance against
Node.js 22+ built-in zlib.zstdCompressSync/zstdDecompressSync.
Closes #338
* perf(streaming): add Transform stream throughput benchmarks
Add streaming.compare.bench.ts measuring Node.js Transform stream
throughput for comprs vs node:zlib, with various chunk sizes.
Closes #340
* chore: add changeset for v1.1.0 performance improvements (#349)
Covers #335 (gzip ISIZE), #336 (zstdmt), #337 (streaming buffer reuse).
* chore(release): version packages to v1.1.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* chore(deps): update taiki-e/install-action digest to 7a562df (#353)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update embarkstudios/cargo-deny-action digest to 175dc7f (#354)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to 97a5807 (#355)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update renovatebot/github-action action to v46.1.8 (#361)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency lefthook to v2.1.5 (#359)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.11 (#356)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency vite to v8.0.8 (#360)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/cli to v3.6.1 (#357)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/wasm-runtime to v1.1.3 (#358)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update vitest monorepo to v4.1.4 (#362)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): set minimumReleaseAge to 3 days in Renovate (#363)
Prevent adopting newly released packages that may be yanked or have
undiscovered issues. This applies globally to npm, Cargo, and GitHub
Actions dependencies.
* feat(middleware): add Express HTTP compression middleware (#364)
* feat(middleware): add Express HTTP compression middleware
Add @derodero24/comprs-middleware — the first Express middleware with
zstd support. Includes Accept-Encoding negotiation, configurable
algorithm priority, threshold-based skip, and Content-Type filtering.
- Set up pnpm workspace for multi-package monorepo
- Implement negotiation, compression, and middleware modules
- Add 31 unit and integration tests
- Update CI to lint/test the middleware package
- Update release workflow for multi-package publishing
Closes #343
* fix(middleware): address CodeRabbit review feedback
- Make isCompressibleType return false for missing Content-Type
- Add stream error handler to prevent unhandled exceptions
- Remove wildcard fallback that could override explicit rejections
- Separate middleware publish job for independent releases
- Limit middleware CI tests to ubuntu/node24 only
- Narrow biome noBarrelFile override to middleware package
- Refactor test helper to support HEAD requests
- Add HEAD request skip test
- Remove "first" claim from README descriptions
* fix(middleware): fix description, docs, and add typescript devDep
- Remove "First" claim from package.json description
- Update filter JSDoc to reflect Content-Type skip behavior
- Add typescript to devDependencies for standalone builds
- Fix filter default description in README table
* fix(middleware): update lockfile for typescript devDependency
* fix(middleware): normalize write/end args and preserve callbacks
Properly handle all overload forms of res.write() and res.end():
- Normalize chunk, encoding, and callback from variadic args
- Pass callbacks through to compressStream.write/end
- Handle res.end(callback) without treating function as chunk body
- Use null-check (!=) instead of truthy for chunk to handle empty strings
* chore(release): version packages (#365)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(ci): check all workspace packages for unpublished versions (#367)
* fix(ci): check all workspace packages for unpublished versions
The release PR creation step only checked the core package version,
preventing middleware-only releases from triggering a develop→main PR.
Now checks both @derodero24/comprs and @derodero24/comprs-middleware
versions against npm, creating a release PR if any package is unpublished.
Closes #366
* fix(ci): read package names from manifest and handle npm errors
- Read package names from package.json instead of hardcoding
- Distinguish E404 (unpublished) from network/auth errors
- Fail the job on unexpected npm view errors instead of silently skipping
* feat(middleware): add Fastify and Hono support via subpath exports (#370)
* feat(middleware): add Fastify and Hono support via subpath exports
Restructure @derodero24/comprs-middleware from Express-only to
multi-framework with subpath exports:
- ./express — Express/Connect middleware (refactored from root export)
- ./fastify — Fastify plugin (onSend hook with stream compression)
- ./hono — Hono middleware (sync compression via Web Standards API)
- . — shared utilities (negotiate, types)
Extract framework-agnostic logic into shared.ts. Add 21 new tests
for Fastify (10) and Hono (11), total 53 tests across 4 test files.
Closes #369
* fix(middleware): address CodeRabbit review feedback
- Handle Vary: * without appending Accept-Encoding (RFC 7231)
- Extract shouldSkip helper in Hono adapter to reduce complexity
- Add Content-Length early threshold check in Hono to avoid body read
- Log compression pipeline errors in Fastify (skip premature close)
- Add stream compression behavior comment in Fastify
- Add identity-only Accept-Encoding test for Fastify
- Fix README import examples to use separate blocks per framework
* fix(middleware): case-insensitive Cache-Control check, Hono error handling
- Normalize Cache-Control to lowercase before checking no-transform (all adapters)
- Set Vary header before HEAD early return in Hono
- Wrap Hono body read + compression in try/catch for graceful fallback
* fix(ci): use manifest name and E404 check in publish-middleware job (#373)
Read package name from package.json instead of hardcoding.
Distinguish E404 (unpublished) from network/auth errors to prevent
silent failures during middleware publishing.
Closes #372
* chore(release): version packages (#371)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(ci): add post-release merge-back from main to develop (#375)
After each release, merge main back into develop to prevent history
divergence. Without this, squash-merged release commits on main
cause growing conflicts on the next develop→main release PR.
Closes #374
* chore(deps): update commitlint monorepo to v20.5.3 (#382)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.13 (#381)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/upload-artifact digest to 043fb46 (#376)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update taiki-e/install-action digest to db5fb34 (#377)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update embarkstudios/cargo-deny-action digest to 91bf2b6 (#378)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/setup-node digest to 48b55a0 (#380)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update github/codeql-action digest to e46ed2c (#379)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @napi-rs/cli to v3.6.2 (#383)
* chore(deps): update dependency @napi-rs/cli to v3.6.2
* fix(deps): bump @emnapi/core and @emnapi/runtime to ^1.10.0
@napi-rs/cli@3.6.2 pulls in emnapi@1.10.0 transitively but @emnapi/core
and @emnapi/runtime were pinned to ^1.9.2, causing a runtime version
mismatch error during 'napi build --target wasm32-wasip1-threads':
Internal Error: emnapi version mismatch:
emnapi@1.10.0, @emnapi/core@1.9.2, @emnapi/runtime@1.9.2
Bump both peers to ^1.10.0 so all three resolve to the same version.
---------
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore!: drop Node.js 20 support and bump napi ABI to napi9 (#386)
Node.js 20 reached end-of-life on 2026-04-30. Raise the minimum supported
Node.js to 22 (Active LTS) and bump the napi-rs ABI feature from napi6 to
napi9 (Node 18.17+ / 20.3+), which is safe under the new floor.
- Remove Node 20 from CI test matrix
- Raise engines.node to >=22.0.0
- Bump napi feature from napi6 to napi9
- Update README badge to >=22
BREAKING CHANGE: minimum supported Node.js is now 22.
Closes #384
* chore(renovate): skip stability days for GitHub Actions digest updates (#387)
The global minimumReleaseAge of 3 days is meant to catch malicious npm/cargo
releases, but it has no analogous benefit for GitHub Actions digest pins —
those only retarget our pin to a newer commit on a release we already trust
by tag. Holding them 3 days delays automerge without security benefit (PR #379
had to be merged manually for this reason).
Override minimumReleaseAge to 0 days for the github-actions manager so the
existing automerge rule can fire immediately.
Closes #385
* chore(deps): update renovatebot/github-action action to v46.1.13 (#398)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update actions/upload-pages-artifact action to v5 (#403)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update commitlint monorepo to v20.5.3 (#389)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @biomejs/biome to v2.4.13 (#390)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency fastify to v5.8.5 (#392)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency hono to v4.12.16 (#393)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency lefthook to v2.1.6 (#394)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency vite to v8.0.10 (#396)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update pnpm to v10.33.2 (#397)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update dependency @changesets/cli to v2.31.0 (#400)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update pnpm/action-setup action to v6 (#404)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* fix(deps): regenerate pnpm-lock.yaml to remove duplicated keys (#405)
Several recent Renovate auto-merges left pnpm-lock.yaml with duplicated
mapping keys (e.g. ajv@8.20.0 listed twice), which causes pnpm and YAML
parsers to reject the lockfile with ERR_PNPM_BROKEN_LOCKFILE. CI on develop
and on every open PR fails at "pnpm install --frozen-lockfile" until this is
fixed.
Regenerating the lockfile from the existing package.json removes the
duplicates and naturally resolves the latest matching versions for several
devDependencies whose Renovate PRs were closed during rebase
(@napi-rs/wasm-runtime, typescript, vitest, @codspeed/vitest-plugin, etc.).
No package.json changes — all bumps satisfy existing semver ranges.
* chore(deps): update rust-dependencies (#402)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore: sync engines.node and docs to >=22 across remaining files (#407)
PR #386 raised the root engines.node to >=22.0.0 but missed several places
that still advertise Node 20 support. Sync them so the v2.0.0 release ships
consistent metadata everywhere.
- README.md Platform Support table: Node.js ≥ 20 → ≥ 22
- CONTRIBUTING.md prerequisites: Node.js ≥ 20 → ≥ 22
- packages/middleware/package.json engines.node: >=20.0.0 → >=22.0.0
- npm/<8 native platforms>/package.json engines.node: >=20.0.0 → >=22.0.0
(napi version preserves the existing field rather than rewriting it from
the root, so this one-time sync is required)
The wasm32-wasi platform package keeps engines.node: ">=14.0.0" since the
WASI binding does not depend on N-API.
Closes #406
* chore(release): version packages (#388)
* chore(release): version packages
* ci: trigger CI on release PR
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: derodero24 <shintaro@payn.io>
* chore(deps): update pnpm/action-setup digest to 0e279bb (#410)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): update codecov/codecov-action digest to e79a696 (#413)
Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>
* chore(deps): bulk-update JavaScript dev dependencies (#422)
Batch the outstanding JavaScript dev-dependency updates Renovate had
queued into a single PR. All are devDependencies — the library ships no
runtime JS dependencies.
- @biomejs/biome 2.5.0 (new a11y rules: associate the playground level
slider label with its co…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
./streamssubpath export topackage.jsonso users can import streaming APIs directly viaimport { createGzipCompressStream } from 'zflate/streams'importandrequireconditions point tostreams.js/streams.d.tssincestreams.jsuses CommonJS which works for both via Node.js CJS-ESM interopCloses #48
Checklist
pnpm run check(Biome lint) passespnpm run build(napi-rs build) passespnpm run typecheck(TypeScript) passespnpm test(Vitest — 121 tests) passespnpm run publint(package validation) passes