Skip to content

refactor(pack): use node-stdlib-browser polyfills - #3349

Draft
yuzheng14 wants to merge 1 commit into
nextfrom
refactor/pack-node-stdlib-browser
Draft

yuzheng14 wants to merge 1 commit into
nextfrom
refactor/pack-node-stdlib-browser

Conversation

@yuzheng14

@yuzheng14 yuzheng14 commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replace the copied, ncc-prebundled Node.js polyfills with a generated source dependency closure from node-stdlib-browser@1.3.1.
  • Generate and embed 99 aliases across 132 package instances so NAPI and WASM builds do not depend on the consumer's node_modules.
  • Preserve the existing 20 unsupported built-ins as empty modules and retain node: protocol aliases.
  • Resolve dependencies inside the embedded generated/node_modules boundary before applying the generic foreign-module rule.

Bundle-size comparison

This is a one-off production-build comparison and is intentionally not committed as a snapshot or benchmark fixture.

Methodology

  • Both sides use the same @utoo/pack JavaScript wrapper and build configuration; only the NAPI binding changes.
  • Configuration: nodePolyfill: true, minification enabled, target: "chrome 100", source maps disabled, persistent caching disabled.
  • Every import is retained through a globalThis assignment.
  • Each result sums all emitted JavaScript assets. Gzip is level 9; Brotli is quality 11; files are compressed individually before summing.
  • The all-modules case imports the 42 canonical module names supported before this migration: 22 real polyfills and 20 legacy empty modules.
  • Duplicate node: aliases are not imported because they resolve to the same implementations. Eight new canonical aliases only available after the migration are also excluded to keep the input identical.
  • The common-modules case imports buffer, events, path, process, stream, url, and util.
  • Values are shown as before → after (change).

Aggregate results

Scenario Raw Gzip Brotli
All 42 existing modules 632.70 → 684.19 KiB (+8.14%) 188.52 → 201.49 KiB (+6.88%) 122.22 → 136.51 KiB (+11.69%)
7 common modules 118.19 → 157.41 KiB (+33.18%) 35.56 → 48.06 KiB (+35.16%) 31.41 → 42.50 KiB (+35.31%)

The 7 common modules are buffer, events, path, process, stream, url, and util.

The embedded NAPI binding changes from 134.28 MiB to 136.51 MiB (+1.66%).

Per-module results

Module Raw Gzip Brotli
assert 66.53 → 75.54 KiB (+13.53%) 20.23 → 22.66 KiB (+12.03%) 18.09 → 20.28 KiB (+12.10%)
async_hooks (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
buffer 33.04 → 33.10 KiB (+0.18%) 11.07 → 10.99 KiB (-0.73%) 9.93 → 9.84 KiB (-0.83%)
child_process (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
cluster (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
constants 14.36 → 14.36 KiB (0.00%) 5.80 → 5.80 KiB (0.00%) 5.14 → 5.14 KiB (+0.02%)
crypto 447.04 → 471.44 KiB (+5.46%) 133.18 → 139.80 KiB (+4.97%) 91.95 → 92.81 KiB (+0.94%)
dgram (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
diagnostics_channel (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
dns (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
domain 18.08 → 17.53 KiB (-3.06%) 6.52 → 6.32 KiB (-3.03%) 5.86 → 5.67 KiB (-3.20%)
events 17.10 → 16.86 KiB (-1.44%) 6.25 → 6.13 KiB (-1.97%) 5.63 → 5.49 KiB (-2.40%)
fs (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
fs/promises (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
http 161.22 → 138.04 KiB (-14.37%) 47.82 → 42.66 KiB (-10.79%) 32.91 → 37.92 KiB (+15.23%)
http2 (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
https 161.96 → 138.45 KiB (-14.51%) 47.98 → 42.79 KiB (-10.83%) 33.00 → 38.09 KiB (+15.41%)
inspector (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
module (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
net (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
os 11.77 → 11.65 KiB (-0.96%) 4.53 → 4.46 KiB (-1.44%) 4.00 → 3.95 KiB (-1.32%)
path 15.33 → 15.09 KiB (-1.59%) 5.94 → 5.82 KiB (-1.99%) 5.33 → 5.21 KiB (-2.18%)
perf_hooks (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
process 12.86 → 14.22 KiB (+10.51%) 4.98 → 5.34 KiB (+7.32%) 4.42 → 4.77 KiB (+7.86%)
punycode 13.48 → 13.57 KiB (+0.70%) 5.46 → 5.54 KiB (+1.45%) 4.89 → 4.95 KiB (+1.26%)
querystring 12.71 → 13.13 KiB (+3.29%) 4.98 → 5.02 KiB (+0.84%) 4.45 → 4.49 KiB (+0.90%)
readline (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
repl (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
setimmediate 12.71 → 12.58 KiB (-1.01%) 5.02 → 4.95 KiB (-1.40%) 4.43 → 4.35 KiB (-1.72%)
stream 106.21 → 78.20 KiB (-26.37%) 31.45 → 23.34 KiB (-25.80%) 27.79 → 20.93 KiB (-24.70%)
string_decoder 37.53 → 37.31 KiB (-0.58%) 12.12 → 12.01 KiB (-0.89%) 10.88 → 10.75 KiB (-1.25%)
timers 13.96 → 13.53 KiB (-3.13%) 5.40 → 5.21 KiB (-3.51%) 4.78 → 4.61 KiB (-3.53%)
tls (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
trace_events (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
tty 11.34 → 11.22 KiB (-1.02%) 4.39 → 4.32 KiB (-1.54%) 3.89 → 3.83 KiB (-1.43%)
url 16.81 → 64.11 KiB (+281.33%) 6.65 → 21.18 KiB (+218.55%) 6.01 → 19.08 KiB (+217.16%)
util 38.21 → 42.85 KiB (+12.16%) 12.29 → 13.74 KiB (+11.83%) 11.03 → 12.36 KiB (+12.05%)
v8 (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
vm 13.59 → 13.30 KiB (-2.12%) 5.23 → 5.10 KiB (-2.32%) 4.62 → 4.51 KiB (-2.28%)
wasi (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
worker_threads (empty) 11.06 → 11.06 KiB (0.00%) 4.25 → 4.25 KiB (0.00%) 3.76 → 3.76 KiB (0.00%)
zlib 211.72 → 191.71 KiB (-9.45%) 61.52 → 56.14 KiB (-8.74%) 47.52 → 48.96 KiB (+3.01%)

Observations

  • By Gzip size, 14 modules become smaller, 7 become larger, and 21 are unchanged (the 20 legacy empty modules plus constants).
  • stream has the largest reduction: 31.45 → 23.34 KiB (-25.80% Gzip).
  • url has the largest increase: 6.65 → 21.18 KiB (+218.55% Gzip), because the previous compact native-url implementation is replaced by the fuller node-stdlib-browser URL compatibility implementation and its dependencies.
  • The complete 42-module bundle grows by 6.88% Gzip; the common-module bundle grows by 35.16% Gzip.

Test Plan

  • cargo fmt --check
  • cargo clippy --all-targets -- -D warnings --no-deps
  • cargo test -p pack-core node_polyfill
  • cargo test -p pack-tests --test snapshot node_polyfill
  • cargo test -p pack-tests (140 passed, 37 ignored)
  • npm run build-wasm --workspace @utoo/web -- --profile release-local
  • Regenerated the polyfill manifest and dependency closure with npm run gen:node-polyfills from crates/pack-core/js.

Closes #2542

@yuzheng14
yuzheng14 force-pushed the refactor/pack-node-stdlib-browser branch from 9b79ac3 to 0287c01 Compare September 14, 2026 08:21
@github-actions

Copy link
Copy Markdown

📊 Performance Benchmark Report (with-antd)

Utoopack Performance Report

Report ID: utoopack_performance_report_20260914_083505
Generated: 2026-09-14 08:35:05
Trace File: trace_antd.json (0.3GB, 0.82M spans)
Test Project: examples/with-antd


Executive Summary

Metric Value Assessment
Total Wall Time 5,284.2 ms Baseline
Total Thread Work (de-duped) 15,787.9 ms Non-overlapping busy time
Effective Parallelism 3.0x thread_work / wall_time
Working Threads 10 Threads with actual spans
Thread Utilization 29.9% ⚠️ Suboptimal
Total Spans 820,006 All B/E + X events
Meaningful Spans (>= 10us) 177,618 (21.7% of total)
Tracing Noise (< 10us) 642,388 (78.3% of total)

Build Phase Timeline

Shows when each build phase is active and how much CPU it consumes.
Self-Time is the time spent exclusively in that phase (excluding children).

Phase Spans Inclusive (ms) Self-Time (ms) Wall Range (ms)
Resolve 38,375 4,674.7 1,237.3 2,338.4
Parse 8,150 1,028.2 742.2 4,586.6
Analyze 114,819 31,668.4 6,551.4 4,513.4
Chunk 5,178 4,825.1 802.3 1,947.5
Codegen 8,829 1,841.2 1,147.5 1,699.1
Emit 29 32.2 16.1 7.8
Other 2,238 6,340.3 3,821.0 5,284.2

Workload Distribution by Diagnostic Tier

Category Spans Inclusive (ms) % Work Self-Time (ms) % Self
P0: Scheduling & Resolution 153,476 36,754.4 232.8% 7,977.5 50.5%
P1: I/O & Heavy Tasks 2,701 92.9 0.6% 76.8 0.5%
P2: Architecture (Locks/Memory) 0 0.0 0.0% 0.0 0.0%
P3: Asset Pipeline 20,637 7,717.7 48.9% 2,703.4 17.1%
P4: Bridge/Interop 0 0.0 0.0% 0.0 0.0%
Other 804 5,845.2 37.0% 3,560.0 22.5%

Top 20 Tasks by Self-Time

Self-time is the exclusive duration: time spent in the task itself, not in sub-tasks.
This is the most accurate indicator of where CPU cycles are actually spent.

Self (ms) Inclusive (ms) Count Avg Self (us) P95 Self (ms) Max Self (ms) % Work Task Name Top Caller
3,178.1 19,064.2 79,192 40.1 0.1 8.4 20.1% module module (60%)
2,594.5 2,693.5 16 162159.0 358.8 577.1 16.4% save snapshot persist (6%)
1,772.1 1,857.7 2,152 823.4 2.2 312.3 11.2% analyze ecmascript module module (81%)
961.5 9,950.1 25,917 37.1 0.1 5.8 6.1% process module process module (79%)
811.3 2,220.6 21,075 38.5 0.1 4.1 5.1% internal resolving internal resolving (71%)
708.1 4,668.6 3,904 181.4 0.2 68.3 4.5% chunking chunking (49%)
686.6 972.6 5,981 114.8 0.4 28.9 4.3% parse ecmascript parse ecmascript (65%)
629.7 699.8 6,753 93.2 0.4 6.1 4.0% precompute code generation generate merged code (48%)
535.6 609.0 5,359 99.9 0.3 92.0 3.4% compute async module info compute async module info (49%)
534.0 1,714.2 571 935.2 2.4 216.9 3.4% generate merged code chunking (78%)
420.8 2,449.0 16,797 25.1 0.0 4.3 2.7% resolving module (62%)
351.1 351.1 329 1067.1 0.8 204.5 2.2% generate source map code generation (83%)
312.6 642.2 131 2385.9 4.8 182.6 2.0% emit code generate merged code (41%)
247.6 423.9 1,091 226.9 0.1 144.2 1.6% write all entrypoints to disk write all entrypoints to disk (8%)
166.7 790.4 1,747 95.4 0.2 35.2 1.1% code generation code generation (82%)
100.0 100.0 8 12495.5 49.8 62.4 0.6% blocking save snapshot (62%)
91.3 153.2 1,223 74.6 0.1 8.5 0.6% compute async chunks compute async chunks (57%)
67.0 78.8 649 103.3 0.0 26.6 0.4% compute binding usage info compute binding usage info (55%)
55.6 55.6 2,166 25.6 0.0 3.8 0.4% read file parse ecmascript (91%)
25.9 49.0 1,533 16.9 0.0 10.3 0.2% collect mergeable modules collect mergeable modules (100%)

Critical Path Analysis

The longest sequential dependency chains that determine wall-clock time.
Focus on reducing the depth of these chains to improve parallelism.

Rank Self-Time (ms) Depth Path
1 639.5 3 persist → save snapshot → blocking
2 402.4 6 chunking → generate merged code → emit code → emit code → emit code → read file
3 312.5 7 module → module → process module → process module → process module → process module → analyze ecmascript module
4 282.7 9 module → module → module → ... → process module → analyze ecmascript module → analyze ecmascript module
5 237.1 4 chunking → generate merged code → emit code → generate source map

Batching Candidates

High-volume tasks dominated by a single parent. If the parent can batch them,
it drastically reduces scheduler overhead.

Task Name Count Top Caller (Attribution) Avg Self P95 Self Total Self
process module 25,917 process module (79%) 37.1 us 0.05 ms 961.5 ms
internal resolving 21,075 internal resolving (71%) 38.5 us 0.06 ms 811.3 ms

Duration Distribution

Range Count Percentage
<10us 642,388 78.3%
10us-100us 113,389 13.8%
100us-1ms 56,948 6.9%
1ms-10ms 7,110 0.9%
10ms-100ms 128 0.0%
>100ms 43 0.0%

Action Items

  1. [P0] Focus on tasks with the highest Self-Time — these are where CPU cycles are actually spent.
  2. [P0] Use Batching Candidates to identify callers that should use try_join or reduce #[turbo_tasks::function] granularity.
  3. [P1] Check Build Phase Timeline for phases with disproportionate wall range vs. self-time (= serialization).
  4. [P1] Inspect P95 Self (ms) for heavy monolith tasks. Focus on long-tail outliers, not averages.
  5. [P1] Review Critical Paths — reducing the longest chain depth directly improves wall-clock time.
  6. [P2] If Thread Utilization < 60%, investigate scheduling gaps (lock contention or deep dependency chains).

Report generated by Utoopack Performance Analysis Agent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Refactor Pack]: replace internal nodePolyFill with node-stdlib-browser

1 participant