feat: WASI port V2 — combined branch: #2's runtime + #1's test scenarios + API-contract tests

[scottmarchant]

Overview

This PR is the complete WASI port of libdispatch. It makes import Dispatch work on wasm32-wasip1, which runs on one thread with no way to block. The port contains the C runtime, the Swift overlay, file-descriptor and signal event sources, and a 52-test suite that includes 13 unmodified upstream tests.

This PR stays open as the single view of the full change set against main. We will not merge it as one unit. We will submit the work upstream as five small, independent PRs. Each upstream PR gets an internal draft in this fork first.

#	Change	Size	Depends on	Status
A	Shared-code preparation: poke-defer hooks and a shared main-queue drain. No behavior change on any current platform.	~90 lines	nothing	Drafted: #5
B1	WASI build system and the core cooperative runtime: toolchain file, CMake arms, module map, lock encoding, cooperative waits, drain backend, dispatch_main	~1,100 lines	A	not cut
B2	The test suite on WASI: the upstream harness compiled for wasm, a pluggable runner, 13 upstream tests, and the focused WASI tests	~500 lines	B1	not cut
C	The Swift overlay: overlay CMake, os(WASI) gates, consumer link tests	~250 lines	B1	not cut
D	File-descriptor and signal event sources: poll integration, harvest, signal latch, EOF policy, their tests	~900 lines	B1	not cut

Order: A lands first and alone. B1 lands next. B2, C, and D land in any order after B1. Fork-only material (runner self-checks, audit prose, extended README semantics) stays out of the upstream slices.

Threads research

The branch feat/scottm/libDispatchWasmThreads (four commits off this PR's tip) answers the multi-threading question. libdispatch now compiles for wasm32-unknown-wasip1-threads behind an experimental DISPATCH_WASI_THREADS=ON CMake knob. In that mode it behaves like a generic POSIX platform: a pthread worker pool, real blocking waits on a wasm futex, POSIX semaphores, and a manager thread on a condition variable. The proof runs under wasmtime v24 (-S threads) and prints PASS: async=8/8 on-worker-thread=8/8 sync=1 timer=1: every dispatch_async block ran on a worker thread, semaphores blocked and woke across threads, dispatch_sync funneled, and dispatch_after fired.

Two results matter for this series. First, the slice A seams need no change and no API signature moves: the poke-defer hooks compile to no-ops in threads mode, exactly as on every threaded platform. Second, the cooperative backend stays the default: current wasmtime has removed wasi-threads support (v24 LTS still runs it), so threads mode ships as an experiment. Not implemented there yet: fd and signal sources, dispatch_main(), and the Swift overlay. Full findings, the gate audit, and reproduction commands live in tests/wasm/THREADS-RESEARCH.md on that branch.

How the port works

libdispatch is mostly data structures: queues, a state machine per queue, a timer heap, and unified event sources. Threads only drive that machinery. The port keeps the machinery and replaces the driver with a cooperative drain on the only thread there is.

• dispatch_async enqueues through normal upstream code. The poke that would wake a worker instead drains pending work on the spot, unless a drain is already running.
• Blocking waits (semaphores, groups, contended sync) make progress by draining other pending work, by sleeping until the next timer, or by waking on file-descriptor readiness. A wait that nothing can ever satisfy crashes at once, with a message that names the wait. The port never hangs and never spins silently.
• Pokes from inside a caller-held critical section (a sync body, a dispatch_once initializer, an object dispose) defer until the section exits. Blocking waits issued inside such sections still pump; that is required for progress and is documented as the contract.
• Read, write, and signal sources ride preview1 poll_oneoff through wasi-libc poll(2). Signal sources cover in-process raise(). Sources deliver during blocking waits and dispatch_main().

How we validated it

• 52 of 52 ctest cases pass. Both authors reproduced the number independently. Later history rewrites changed only commit metadata (co-author trailers, author emails); the validated tree is unchanged.
• The suite includes 13 unmodified upstream libdispatch tests, 32 focused WASI tests, 2 zero-flag consumer link tests (C and Swift), and 5 runner self-checks.
• Linux container builds prove native neutrality: main and this branch pass the identical 23 of 23 upstream tests.
• The CMake floor (3.31.0) is verified by an exact-version build.
• Behavior is spot-verified on four runtimes: wasmtime, Node, WasmKit, and a browser WASI shim.

History and key decisions

Two review rounds ran against the full diff, one from each author's side. Details live in the comments on this PR. Key decisions on record:

• Eager submission is specified behavior, pinned by test: a top-level dispatch_async runs the block before it returns. This is what makes the library work in hosts that never call dispatch_main().
• Wall-clock timers anchor to the uptime clock when armed. WASI has no clock-change notification, so this ships as a documented limitation.
• An fd closed while its source is armed produces a named crash on both host error shapes. A pipe-EOF source that never cancels produces a named crash instead of a silent hot loop.
• The timed-semaphore race that looked like a port bug is upstream working as designed: _dispatch_semaphore_wait_slow re-checks the semaphore value on timeout and delivers the raced signal. The reproduction test stays in the suite as a pin.
• The two misattributed deadlock diagnostics land in B1.
• The host-callable pump entry point (for hosts that never block) stays a later PR. The limitation is documented.

Remaining work

A - shared-code preparation (#5)

• @krodak @scottmarchant Review the draft in the fork
• @scottmarchant @krodak Decide the author split for the slice and finalize the commit history (two commits: hooks and drain by @scottmarchant, privdata-funnel bracket by @krodak; each commit co-authors the other)
• @scottmarchant Copy the branch to upstream, open the upstream PR, and raise the Apple CI question

B1 - build system and core runtime

• @scottmarchant Add the two WASI-specific deadlock diagnostics (pumped work that re-enters the locked queue; a re-entered in-progress dispatch_once)
• @scottmarchant @krodak Confirm wall-clock timer anchoring ships as a documented limitation
• @krodak Write the toolchain-file placement answer and the SDK-generator sketch
• @scottmarchant Cut B1 from the frozen integration state

B2 - test suite

• @krodak Rewrite the focused tests onto the bsdtests macros where they are not already
• @krodak Trim crash tests to exit mode plus one stable message substring
• @scottmarchant Keep runner self-checks fork-only; move audit and README prose to companion docs
• @scottmarchant @krodak Prepare the CI question for upstream: who runs ctest until a WASI lane exists

C - Swift overlay

• @scottmarchant @krodak Decide Swift consumer test placement with reviewers (in-tree, or SDK-integration CI outside the repo)
• @scottmarchant Cut C from the frozen integration state

D - event sources

• @scottmarchant @krodak Agree the EOF rate-guard constants, or make them configurable
• @krodak Confirm the pump entry point stays a later PR
• @krodak Cut D from the frozen integration state

Cross-slice

• Close feat: Temporary PR to view accumulated changes to compile libDispatch to Wasm #1 and feat: Temporary PR to review changes to compile libDispatch to Wasm #2 with pointers; retarget this PR to main
• Record the B1 placement for the deadlock diagnostics
• @scottmarchant Merge this branch to a fork integration branch, then cut every slice from that frozen state
• @krodak Validate GRDB and PowerSync against this branch

Research and ecosystem

• Investigate wasm pthread support with the threads wasm SDK (wasm32-wasip1-threads). PROVEN on branch feat/scottm/libDispatchWasmThreads: libdispatch builds with -DDISPATCH_WASI_THREADS=ON and the smoke test reports PASS: async=8/8 on-worker-thread=8/8 sync=1 timer=1 under wasmtime v24. The slice A seams need no change and no API signature moves, so Prepare shared code for cooperative single-threaded event backends #5 is unblocked. Details: tests/wasm/THREADS-RESEARCH.md on that branch.
• Research Swift Embedded support
• See if any tests from dispatch-async should be ported over
• Make a public test app and usage example. Bundle many ideas in the one example: sqlite, vapor, elementary-ui, dispatch, and more
• Update SwiftWasm/uwasi to support libdispatch, then consume it in the example app to prove it works
• Research and plan the work to update WasmKit to support libdispatch
• Research the blast radius in swiftlang repos where #if canImport(Dispatch) is mis-used: find the WASI code paths that become wrong, or break, when real Dispatch support for wasm rolls out

[scottmarchant]

Comparison: PR #1 vs PR #2 vs PR #3 (libdispatch → Wasm/WASI)

(Updated after two review rounds on #3: a full-branch review with fixes, then an adversarial probe round from the #2 side with a hardening series. Every row is verified empirically — built and executed. Cross-posted on #1, #2, and #3.)

All three branches were built for wasm32-unknown-wasip1 on macOS arm64 and their test suites run. Toolchain: swift.org 6.3.3 toolchain + Swift 6.3.3 Wasm SDK (#2/#3), wasi-sdk 33 (#1), wasmtime 47, Node 26, CMake 4.4 (floor verified at exactly 3.31.0). #3's event-source and re-entrancy behavior additionally verified under WasmKit (built from source) and @bjorn3/browser_wasi_shim; native neutrality proven with Linux container builds (identical 23/23 upstream tests on main and #3).

Branch summary

	#1 (feat/scottm/libDispatchWasm)	#2 (krodak/libdispatch-wasm)	#3 (feat/scottm/libDispatchWasmV2)
Runtime design	defer-everything: work runs only under dispatch_main(); blocking traps	cooperative eager drain; blocking waits drain other work	#2's runtime + fd/signal event sources + poke-defer brackets: pokes never eager-drain beneath caller-held locks (blocking waits still pump there, by documented contract)
Diff vs main	+874 / −26 (36 files)	+2,148 / −75 (55 files)	+4,861 / −81 (76 files; stacked on #2)
ctest	3/3	30/30	52/52
Swift overlay	manual out-of-tree recipe, not in CI	CMake-integrated, autolink modulemap, consumer tests	same as #2

Dispatch API support

Legend: ✅ works · ⚠️ works with caveats · ⏱ contract violation (returns instantly instead of waiting) · 💥 traps · ❌ broken/unsupported.

Queues & submission

API	#1	#2	#3
Queue create/attrs/label/target/specifics, global & main queues	✅	✅	✅
dispatch_async / barrier_async	⚠️ nothing runs until dispatch_main()	✅ eager	✅ eager; pinned by test
dispatch_sync / barrier_sync	✅ inline; re-entrancy detection unreliable (tid bug)	✅ inline; but a poke from inside a sync body eager-drains under the held barrier lock — sync(qA){async(qB){sync(qA)}} crashes where threaded platforms complete	✅ inline; pokes inside sync bodies, dispatch_once initializers, dispose, and set_specific defer and flush after the critical section — that program now completes (tested on 3 runtimes)
dispatch_after, timers (uptime & wall)	✅ (wall deadline movable by host clock)	✅ (wall anchored at arm; tested)	✅
dispatch_apply	✅ inline-serial	✅ inline-serial	✅ inline-serial
dispatch_main	⚠️ busy-spins at 100 % CPU when idle	✅ traps loudly when idle forever	✅ parks in host poll on armed timers and fd sources; a signal-source-only park still traps as truly idle (in-process raise() cannot reach a parked sole thread — deliberate, documented)
QoS order & fairness	❌ inverted + starvation	✅ tested	✅
dispatch_assert_queue / _not	❌ inverted by tid-encoding bug	✅	✅ + regression test

Synchronization

API	#1	#2	#3
dispatch_once	✅	⚠️ initializer that submits work eager-drains under the once gate (drained item re-entering the once crashes; threaded platforms wait)	✅ deferred under the gate
Semaphore create/signal/uncontended wait	✅	✅	✅
semaphore_wait(FOREVER)	💥 traps	✅ drains cooperatively	✅ also wakes on fd events (tested)
semaphore_wait(timeout)	⏱ instant	✅ honors deadline	✅ + regression test for the timeout-vs-signal race (see the "working as designed" note in #3's description)
group_wait (timed / FOREVER)	⏱ instant / ❌ returns −1	✅ / ✅	✅ / ✅
dispatch_block_wait	⏱ instant	✅	✅
Provable single-thread deadlocks	💥 (all blocking)	💥 named diagnostics; contended unfair lock/once gate spins silently at 100 % CPU (empty _dispatch_thread_switch)	💥 named diagnostics everywhere — the lock/gate spin is now a named crash

Sources

API	#1	#2	#3
Timer sources	✅	✅	✅
User-data sources	✅	✅	✅
READ / WRITE fd sources	❌ silently never fires	💥 traps at registration	✅ via poll_oneoff fd subscriptions; armed-source count now unbounded (growable poll set; previously capped at 64 with a load-dependent trap)
SIGNAL sources	❌ silent	💥 traps	✅ in-process raise() semantics, count-accurate; registration saves and unregistration restores the app's own signal() disposition (previously reset to SIG_DFL, so the app's next raise() terminated the process); SIG_ERR checked
PROC / VNODE / Mach / memory-pressure	❌	❌	❌ — no WASI facility exists
fd closed while source armed	(n/a — sources never fire)	(n/a)	💥 named crash on both harvest shapes (per-subscription POLLNVAL and wasmtime's whole-call EBADF), each pinned by a limited-host runner mode. Deliberate: kqueue logs a vanish, epoll silently never fires again. Caveat: on Node, the guest closing an armed stdio fd aborts the host inside libuv before libdispatch can see it
pipe EOF with an un-canceled source	(n/a)	(n/a)	✅ readable-at-EOF delivered per the Darwin contract (handler observes the 0-byte read and cancels); a handler that never cancels on a host without the poll hangup flag becomes a named crash via a windowed rate guard instead of a silent ~500k-fires/sec spin

Data & I/O

API	#1	#2	#3
DispatchData (all operations)	✅	✅	✅
dispatch_read/write, DispatchIO on regular files	✅ verified	✅ verified	✅ verified
Same on non-regular fds (streams)	❌ hangs silently	💥 traps	✅ tested: a DispatchIO stream channel on stdin completes a read whose payload arrives 250 ms into the wait

Test coverage

Suite	#1	#2	#3
Upstream libdispatch tests	—	13	13
Focused WASI semantics tests	3 bespoke	13	32 — #2's set + the earlier contract/event-source/review-regression tests + the probe-round additions: async-and-wait (privdata funnel), pipe-EOF in both host shapes, close-while-armed, group wait woken by an fd source, payload-asserted regular-file IO
Swift consumer tests (in ctest)	0 (2 manual)	2	2
Runner self-checks	—	3	5 (early-guest-exit robustness; a limited-host mode that denies fd poll subscriptions, pinning the ENOTSUP capability crash)
Total in CI	3	30	52

Harness note (#3): Node is now optional — without it (or below 19.8) everything still builds and all tests register as visible-but-DISABLED, the same shape as the missing-WASI_TEST_RUNNER path (#1's design, adopted per the two-ports report).

Runtime support

Capability	wasmtime 47	Node node:wasi	WasmKit	browser_wasi_shim	uwasi
Timers / queues / sync (all three PRs)	✅	✅	✅	⚠️ busy-wait sleeps, single-subscription	❌ no poll_oneoff
fd read/write sources (#3)	✅	✅ (regular files via always-ready path)	✅ (bounded poll slices dodge its infinite-timeout host trap)	❌ loud named crash at registration	❌ same
Signal sources (#3)	✅	✅	✅	✅ (pure libc)	✅

Review round (what changed since the previous version of this comment)

Two rounds. First, a high-effort review of #3's full diff vs main (i.e. including the #2 base) produced 10 findings; each was reproduced with a test before fixing. Fixed: eager-drain-under-held-locks (the biggest semantic gap vs threaded platforms — see the sync row above), signal disposition save/restore, the 64-fd poll cap, the signal-pending latch protocol, a 63-line main-queue-drain fork (now hoisted to a shared function with DISPATCH_COCOA_COMPAT-gated divergences), silent lock-contention spins (now named crashes), and test-runner robustness. Two findings were refuted by their own reproduction tests — most instructively, the timed-semaphore "dropped signal": the low-level early-timeout path exists, but upstream's _dispatch_semaphore_wait_slow re-checks the semaphore value on timeout and delivers the signal — pre-existing upstream defense-in-depth working as designed (details in #3's description). No pre-existing upstream libdispatch bug needed fixing.

Second, an adversarial probe round from the #2 side found one bypassed funnel in the poke-defer fix (dispatch_async_and_wait with a dispatch_block_create block — what Swift's asyncAndWait(execute:) produces — crashed with an internal diagnostic; fixed with the same bracket plus a four-shape regression test) and hardened three edges: the pipe-EOF hot loop became a named crash, close-while-armed got its named diagnostic on wasmtime's whole-call-EBADF shape, and the signal-restore path checks SIG_ERR. It also independently re-verified the "working as designed" semaphore analysis and the Darwin-identity of the main-queue-drain hoist, and added tests for two behaviors previously verified only by probe. 46/46 became 52/52, independently reproduced on both sides.

Bottom line

#1 is the minimal seed but breaks contracts on APIs it nominally supports. #2 made the supported set behave per spec and the unsupported set fail loudly; the review round found its one systemic gap — eager drains running client code beneath caller-held locks — plus a silent-spin path and the signal/cap issues, all inherited by and now fixed in #3. #3 is #2's design carried to completion: fd and signal sources work, the re-entrancy divergence from threaded platforms is confined to documented, pinned-by-test semantics, and every failure mode is a named crash. The remaining unsupported surface (process/vnode/memory-pressure/Mach, cross-process signals, true parallelism) is bounded by WASI itself.

🤖 Generated with Claude Code

[krodak]

Pushed 82ff658 fixing review finding F1.

What was bypassed: _dispatch_async_and_wait_block_with_privdata (the entry for dispatch_async_and_wait / dispatch_barrier_async_and_wait with a dispatch_block_create() block, which is what DispatchQueue.asyncAndWait(execute: DispatchWorkItem) produces) called _dispatch_async_and_wait_recurse directly, skipping the poke-defer bracket that 74689b8 added to the plain-block funnels.

The crash: a dispatch_async inside such a block eager-drained inside the _dispatch_fake_wlh ANON region and hit the internal crash BUG IN LIBDISPATCH: Lingering DISPATCH_WLH_ANON. Reproduced on wasmtime and WasmKit before the fix.

The fix: the same defer/undefer pair _dispatch_async_and_wait_f uses, around the recurse call. The macros are ((void)0) off WASI and the return-of-void to bare-call change is codegen-identical, so non-WASI builds are unchanged. A sweep of the recurse/wait funnels found no other unbracketed entry.

New test: tests/wasm/async-and-wait.c (ctest name dispatch_wasi_async_and_wait), written first and observed failing with the ANON crash. It covers a plain-block control with a nested async-then-sync-back plus privdata blocks in benign-async, full-nesting, and barrier shapes.

Fresh ENABLE_SWIFT=YES build with the 6.3.3 toolchain/SDK pair: 47/47 ctest cases pass (46 base + the new test).

[krodak]

Review summary from the #2 side

Reproduced the headline (46/46, now 47/47 with the async_and_wait fix pushed as 82ff658), then ran an adversarial probe round against this branch, most probes also against the #2 base. Verdict: the architecture is right and nothing we found changes the design. The findings are contract statements that promise more than the code delivers, one hot-loop hazard, and some coverage gaps.

1. "Never beneath caller-held locks" is only true for pokes

Blocking waits still pump other queues' work under the caller's lock: dispatch_sync(qA) { dispatch_async(qB, ...); sem_wait(...) } runs qB's block during the wait, under qA's barrier. That pumping is load-bearing, not a bug: deferring it would break exactly this shape, which GRDB-style consumers depend on. Asks: promote this to the contract docs ("pokes defer; waits still pump under the caller's lock"), and give the re-entrant corner a WASI-specific diagnostic. Today, pumped work that syncs back into the locked queue dies with "already owned by current thread", and a re-entered in-progress dispatch_once with "trying to lock recursively": correct crashes, wrong stories. The once-gate itself checked out clean (submit-then-wait completes, no regression vs #2).

2. Event sources need a blocking wait or dispatch_main, and nothing says so

Timers merge in drain_one, so any submit fires them. fd and signal harvesting happens only in _dispatch_wasi_wait_for_events, reached from the parks alone. So in a JavaScriptKit-style embedding (never blocks, never calls dispatch_main), fd and signal sources register successfully and silently never fire, even when ready by construction. A quiescent module's armed timer never fires either. Asks: one contract sentence ("fd and signal sources require a blocking wait or dispatch_main to deliver"), a host-callable pump entry point as follow-up, and a correction to event_wasi.c:166 (the latch does not merge at drain points, only waits).

3. Pipe EOF turns a park into a hot loop

When the write end of a polled pipe closes, POLLHUP makes the fd permanently ready: measured ~548k handler fires in one second from a parked dispatch_main, no diagnostic. Any client disconnect triggers this. Needs an EOF story (unregister, cooldown, or named diagnostic) before external eyes.

4. Close-while-armed does not survive Node

On wasmtime the crash is the generic EBADF poll() failed for armed... path; the advertised POLLNVAL message is dead code. On Node, closing an armed stdio fd trips libuv's own assertion and aborts the host before libdispatch sees anything. The "named crash, both backends" claim needs a runtime qualifier, and the POLLNVAL branch wiring up or removing.

5. Smaller items

• Signal-source-only dispatch_main counts as "truly idle" and traps; deliberate, but the table says sources keep it parked.
• Two claimed-verified behaviors have no in-tree test: group_wait woken by an fd-source leave, and read/write plus DispatchIO on regular files. Both hold (we probed, payload-asserted); tests are cheap to add.
• Split plan: B needs A's hoisted drain, so A then B, not "either order".
• Stale bits: WAIT-PUMPING-AUDIT.md still describes the pre-revert set_specific fix; tests/wasm/README.md:146 references PR feat: Temporary PR to view accumulated changes to compile libDispatch to Wasm #1; signal() return unchecked at disposition restore.

What checked out under fire

The defer macros are no-ops off WASI, the Darwin main-queue hoist is statement-identical, the wait-park consolidation preserves the base wait semantics, the sema-late-signal "working as designed" analysis is correct (we traced the _dispatch_semaphore_wait_slow re-check independently), QoS fairness is pinned by a test covering the starvation shape, and Node 22 suffices.

Nothing blocks using this branch as the unified one. Items 1 and 2 are wording plus a diagnostic each; item 3 is the one behavior fix; the rest is tests and docs.

[krodak]

H9 follow-ups landed on the branch (82ff658..b7a1004); WASI ctest is now 52/52 (-DENABLE_SWIFT=YES, fresh build, Node runner + wasmtime spot checks).

• Pipe-EOF hot loop (89b9486): hosts that never report poll_oneoff's hangup flag (wasmtime 47 for pipes — and preview1 offers no other EOF signal: nbytes is a constant 1, fd_filestat_get has no pipe fill) make EOF indistinguishable from readiness, so a source whose handler never cancels turned a parked dispatch_main()/wait into a silent ~500k fires/sec spin. The harvest now converts that into a named crash via a windowed rate guard (100000 ready-polls within 2 s — a rate only an instantly-ready descriptor can sustain; a per-poll elapsed cutoff loses to host scheduling jitter). The readable-at-EOF fire is preserved (Darwin contract: observe the 0-byte read(), cancel), and hangup-reporting hosts keep the existing EPOLLHUP-style source drop. Pinned both ways by pipe-eof-source.c via the runner's new --suppress-poll-hangup.
• close-while-armed (136902e): a whole-call EBADF from the harvest poll (wasmtime's shape) now crashes with the named file descriptor closed while dispatch source is armed instead of the generic poll() failed message; the POLLNVAL branch stays for hosts that report per-subscription badf (wasi-libc maps it). Pinned via the runner's new --fd-poll-ebadf-after; the README documents the Node caveat (guest-closing an armed stdio fd aborts the host inside libuv before libdispatch sees anything).
• signal restore (e755797): unregistration now checks SIG_ERR from signal() like registration does.
• New coverage (ff59de7): blocking dispatch_group_wait(FOREVER) woken by an fd-source handler, and payload-asserted dispatch_read/dispatch_write/DispatchIO on a regular file (runner grew --preopen).
• Docs (b7a1004): signal-source-only dispatch_main() idle-trap rationale (in-process raise() only — nothing can signal a parked sole thread); the two boundary contracts stated explicitly (pokes defer under brackets but blocking waits still pump under the caller's lock; fd/signal sources deliver only at blocking waits or dispatch_main()); WAIT-PUMPING-AUDIT.md now describes the shipped set_specific shape (push under dqsh_lock inside the poke-defer bracket); signal latch comment fixed.

New tests: dispatch_wasi_pipe_eof_hangup, dispatch_wasi_pipe_eof_spin_crash, dispatch_wasi_fd_closed_while_armed, dispatch_wasi_group_fd_wakeup, dispatch_wasi_regfile_io (47 → 52).