fix(fspy): collect file accesses without waiting for traced processes by wan9chi · Pull Request #675 · voidzero-dev/vite-task

wan9chi · 2026-08-14T08:13:31Z

Motivation

Collecting a task's file accesses waited for every traced process to release a file lock. That broke three ways:

#544: a process that closes inherited file descriptors drops the lock while it can still write. The reader then raced it and read path bytes as if they were a record header, panicking the runner.
A process that outlives the task, such as a dev server, held collection open for as long as it kept running.
#533: a task that made more accesses than the region held was killed, because the preload panicked inside an intercepted call, where a panic cannot unwind and aborts the process instead.

Whether the result is correct cannot depend on how long a file descriptor lives, or on a dying process running cleanup code.

What changed

Records go into a table of fixed slots instead of a stream with headers in front of each one. README.md beside the code describes the whole thing; what matters here:

Writing a record is two atomic adds and one store. No lock and no retry loop.
A writer that dies mid-record leaves a slot the receiver skips. Nothing cleans up after it, because there is nothing to clean up.
Closing reads one counter and sets one bit. It never waits for a writer, and costs the same for one record as for ten million.

All of it rests on one rule: a record is written before the operation it describes is performed. A record that never arrives then describes something that never happened, and one refused after closing describes something that happened after the runner stopped looking. Both are safe to drop.

When the region fills up

The writer skips the record and carries on, because recording must never stop the task doing the work. It also sets the bit that tells the receiver these are not all the accesses, and the runner reports the task as not cached rather than caching one whose inputs it half knows.

That replaces a panic which killed the traced process, and which a large enough build could reach without anything being wrong.

Known regression

Task launch on Linux costs a millisecond or two more, from first touching the region's pages. The benchmark's launch rows read +158% and +221% because its target opens no files, so a millisecond is most of what they measure; against a real task it is invisible. It is paid once per task, though, so a build spawning hundreds pays it hundreds of times. Putting the region on /dev/shm, which does not journal, would remove it rather than hide it, and is worth measuring on its own.

Closes #544. Fixes #533. Supersedes #577.

…cation The IPC channel previously required writer quiescence before reading: a file lock (or #577's active-writer gate) had to drain before the receiver could parse the inline frame stream. A traced process that closed the lock descriptor while keeping the mapping writable corrupted parsing (#544), and one that never exited (a daemon) or died mid-record could block collection or poison the writer count forever. The shared memory now uses a two-ended layout: an allocator word admits claims and closes the channel, a descriptor table grows from the front, and payloads grow from the back. Each frame commits by publishing its descriptor with a release CAS; closing atomically aborts every unfinished slot and copies committed payloads out with relaxed atomic loads, so the receiver never waits for a writer, never trusts payload bytes for traversal, and never holds a reference into memory another process may mutate. Loss of a record by a live writer (capacity, abandonment) flags the trace incomplete so the run is not cached from an under-reporting trace; process death needs no cleanup because records are published before the recorded operation is performed. Closes #544. Supersedes #577. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

github-actions · 2026-08-14T08:15:50Z

fspy benchmark

linux

dynamic/launch             change +152.39%  [+121.28% .. +175.59%]  overhead  +285.03%
dynamic/access             change  +0.96%  [ -0.48% ..  +2.13%]  overhead    +6.64%
dynamic/access-relative    change  +1.53%  [ +0.28% ..  +1.99%]  overhead   +48.88%
dynamic/access-contended   change  +2.01%  [ -1.51% .. +10.10%]  overhead    +9.80%
static/launch              change +224.83%  [+199.91% .. +245.41%]  overhead  +700.56%
static/access              change  +0.23%  [ -1.08% ..  +1.04%]  overhead  +812.29%
static/access-relative     change  +0.20%  [ -0.65% ..  +1.45%]  overhead +1315.69%
static/access-contended    change  -0.11%  [ -2.28% ..  +1.50%]  overhead +3512.52%

macos

dynamic/launch             change  -0.58%  [ -5.08% ..  +5.15%]  overhead  +228.68%
dynamic/access             change  +2.44%  [ -3.73% .. +33.79%]  overhead    +3.53%
dynamic/access-relative    change  +1.58%  [ -7.51% .. +11.80%]  overhead  +267.50%
dynamic/access-contended   change  -1.06%  [ -8.15% ..  +2.59%]  overhead    -0.09%

windows

dynamic/launch             change  -1.59%  [ -6.19% ..  +2.15%]  overhead   +26.01%
dynamic/access             change  +0.00%  [ -3.22% ..  +1.62%]  overhead    +1.08%
dynamic/access-relative    change  -0.18%  [-11.48% ..  +2.14%]  overhead    +1.25%
dynamic/access-contended   change  -0.53%  [ -4.76% ..  +5.13%]  overhead    +5.83%

Temporary stderr phase timings to locate the CI launch regression on the Linux benchmark runner. Will be dropped before merge. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

This reverts commit d2fa1de.

…path The crash-tolerant close moved two hidden costs into the tracked child's launch window on journalling filesystems: the first write to the sparse 4 GiB backing file (a millisecond-scale block allocation, previously paid lazily or never) and unmapping the receiver's view (previously after access collection). The Linux benchmark runner priced them at ~2.2 ms and ~0.6 ms per launch. Pre-fault the header page on a background thread at channel creation — a protocol-neutral compare-exchange of zero with zero, run concurrently with process startup — and release the receiver's mapping on a detached thread after the frames are copied out. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Replace the packed allocator word and its compare-and-swap loop with two monotonic counters over a fixed table/payload partition. A claim is two wait-free fetch_adds validated against the fixed region bounds; failed claims overshoot the counters harmlessly because committed descriptors are self-describing and readers clamp to the region capacities. The close boundary becomes a snapshot load: claims that arrive later land in slots the receiver never visits and are dropped under the same publish-before-perform argument as freeze-race losses. The CLOSED gate — whose write materializes the counter page, a millisecond-scale first-block allocation on journalling filesystems when the trace is empty — moves onto the deferred teardown thread, which lets the pre-fault machinery from the previous commit be deleted outright. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The wait-free rework deleted the pre-fault thread on the theory that a write-free close no longer needs the page. The benchmark disagreed: on the Linux runner the first touch of the sparse backing file costs milliseconds whether it is a write (a sender's first claim) or a read (close's snapshot), so Linux launches regressed right back. Windows meanwhile improved once the thread was gone — its first touch is cheap and the spawn was the cost. Restore the concurrent header-page warm-up, gated to Linux. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

… a thread Reserving one block at the header and one at the payload-region start with fallocate(KEEP_SIZE) is a cheap metadata-only operation at channel creation, so the milliseconds of journalled block allocation that some filesystems charge for the first touch of each area no longer need a background thread to hide them — and the payload area, which the thread could not safely touch, is now covered too. KEEP_SIZE because growing the file would desynchronize mapping sizes across processes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…lting on a thread" This reverts commit 528d1d1.

The fallocate experiment showed the first-touch cost is in the fault path, not block allocation, so only a real touch helps. Give the pre-fault thread a second target: a protocol-owned warm word between the table and the payload data, so the page where the first payloads land is materialized without racing any writer's payload bytes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

This reverts commit c4ba8dd.

…opying Frames now owns the mapping and lazily hands out per-span borrows of the validated committed payloads. A committed span is immutable under the protocol and disjoint from everything a live writer may still touch, so the borrows are sound without a copy; the trust argument lives in the reader module docs. This also collapses the close-time machinery: the CLOSED gate returns inline into close (its page is pre-warmed on Linux where first touches are expensive), the deferred-teardown thread is gone, and the mapping is released when Frames drops — naturally off the collection path. The receiver-side frame validation pass in the supervisor is dropped with it; committed frames are complete by protocol. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Nothing is collected anymore — frames are borrowed in place — and the async wrapper descended from the file-lock era, when acquiring the trace could block until every sender exited. Closing is now bounded by the number of reported records and runs inline, so the type becomes ChannelAccesses with a TryFrom<Receiver> conversion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The protocol layer should not know its consumer: describe the publish- before-perform rule as the intended usage contract and the incomplete flag as a property of the channel, with no mention of what sits on top. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The descriptor table's length becomes the protocol's const-generic parameter, so the header and table are one repr(C) struct — offsets become field accesses, the table a real array, and the per-accessor unsafe pointer derivations collapse into one borrow. The payload area stays outside the struct deliberately: writers hold exclusive borrows into it that must not alias the shared region borrow. The channel names its layout the same way: channel::<SLOTS>() sizes the backing file to capacity_for_slots(SLOTS) (the exact inverse of the slots_for_capacity sizing rule), every process names one shared SHM_SLOTS constant, and a sender now rejects a region whose size disagrees with the layout instead of panicking inside geometry assertions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The const-generic table size taxed every signature and call site, and it bought a property the mapping already provides: with the layout derived from the mapping length alone, the region is self-describing — writers and the receiver compute identical bounds from the size of the file they mapped, with no shared constant to agree on and no size handshake to get wrong. What the struct experiment taught survives: one unsafe borrow now builds three typed views — the repr(C) header, the descriptor table as a slice of atomics, and the raw payload area — so counters are named fields, slots are bounds-checked indexes, and only payload spans remain pointer arithmetic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The incomplete flag had one real writer — capacity exhaustion — and the counters already record that: a failed claim's bumps push a counter past its limit, counters never move backwards, and the bump precedes the operation whose record was lost, so the close snapshot either sees the overshoot or the loss belongs past the boundary. The flag's other writers were bug-only paths. So the flag word, FrameMut's Drop, and the write_encoded flagging wrapper are gone; abandoning a frame now leaves exactly what dying does — an unfinished slot the receiver ignores — and acting on an abandoned record is outside the usage contract. Oversized frames become an asserted precondition: a caller error, and the one loss counters could not record. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The protocol had two ways to write a record — write_encoded, with an error enum only one caller half-used, and the hand-rolled claim/serialize/finish sequence in the Unix client. Both callers want the same thing: serialize the record into a frame and skip it on any failure, because an intercepted call must proceed no matter what. That helper now lives once, on the channel's Sender; shm_io keeps only claim, fill, and finish. Also: a plain-words README section on how a full region is handled, and a mermaid dependency graph of the module files as a reading order. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The six-way split was designed around machinery the simplifications have since deleted, and its narrative had drifted: writer and reader derive their own payload references, so state was never the only file touching shared memory. Merge to the boundary that still earns its keep — pure integer math versus code that touches the mapping: - layout.rs absorbs the descriptor codec (both plain arithmetic) - shared.rs is state + writer + reader in reading order, with the reservation types and SharedState going private to it - mod.rs keeps the surface, overview docs, and integration tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Completeness by counter overshoot had two holes. A frame larger than a descriptor can describe (> i32::MAX bytes) could not be reported at all, so claiming one panicked — reachable in the preloads, whose record lengths come from path strings the traced program controls, breaking the promise that a preload never panics its host. And a writer killed inside a failed claim left an overshot counter behind, marking a channel incomplete over a record whose operation never ran. Replace the overshoot rule with a loss flag in the header's reserved space: every failed claim stores it before the writer moves on, and the receiver reads it once at close. The non-overflow path is untouched — the flag's cache line is only written by a claim that is already failing. The report-before-perform order gives the same rule-1 guarantee as commit-before-perform: a report the receiver misses belongs to an operation performed after close, and a writer that dies before reporting never performed the operation at all. Oversized frames now fail like any other refused claim, without poisoning the counters, so the channel stays usable for the records after them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

HEADER_LEN predates the typed header: when layout.rs was offset math only, there was no struct to measure, so the size was a literal and a const assert tied the struct to it after the fact. Move the Header struct into layout.rs — it describes the region's shape, which is that file's job — and derive HEADER_LEN from size_of. The sizing math now follows the struct automatically; the one remaining literal is an assert pinning the header to a single cache line, which is a design intent no struct can express. CLOSED moves along with it, keeping all the bit meanings next to the slot codec. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Every one of these is a leftover from a deleted design, kept alive only by habit or by tests: - ReserveError duplicated ClaimError variant for variant; try_claim now returns ClaimError directly and the mapping in claim_frame goes away. - Reservation was a named pair passed once between two functions in the same file; a destructured tuple says the same thing. - SharedState::mapping_len wrapped a field its one caller can read. - The close/pre_fault wrappers in mod.rs re-stated shared's docs to delegate one call; the functions are now re-exported like the rest of the surface, with the wrapper's doc text folded into the real ones. - Sender's Deref to ShmWriter served only tests, which now reach the writer field directly; FrameMut, ClaimError, and ProtocolError are no longer nameable outside the channel (production never names them), the error types staying test-visible for assertions. - into_memory is gated to the non-miri test that is its only caller. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

SlotState classified every slot value the receiver could read, but its only consumer treats all invalid classes identically, and the validation that followed already refuses every pattern the classification singled out: an unfinished zero decodes a zero length, and any value carrying the aborted bit decodes a length beyond the 31-bit limit. Collapse decode and validate into one step returning Option<PayloadSpan>; the freeze loop keeps a slot's span, skips ABORTED, and calls everything else corrupt. The three-state table stays as the comment documenting what slot values mean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The free unsafe close made the module's unsafe boundary uneven: the writer pays its contract once at attach and operates safely, while the receiver re-asserted the same contract at every close call — in the channel, far from the creation-time facts the SAFETY comment cites. ShmReceiver mirrors ShmWriter: one unsafe constructor with the identical contract, eager geometry validation, and a safe consuming close. The channel constructs it where the region is created, so Receiver::close is now safe code. pre_fault stays a free function: it is a creator-side warm-up on a throwaway view, belonging to neither endpoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Win32 spells `ERROR_FILENAME_EXCED_RANGE` without the second E, and the comment naming it is more use to a reader than the spelling checker is, so the word joins the allowed list beside the other Windows one. The `shm_capacity` field needed no musl exemption after all. It is read there, by the setter, so claiming it is dead made the expectation unfulfilled instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Two CI fixes from the base. The musl exemption on the size field goes with them: the setter reads it there too, so claiming it is dead only made the expectation unfulfilled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Windows never overran the small channel the first version of this case set up, because a path record cannot get large enough there. A path reaches the tracer through a `UNICODE_STRING`, whose length field is a `u16`, so however long a name the caller asks for, no single record exceeds 64 KiB. Its 1 MiB channel had room to spare, tracking came back complete, and the run cached. Record count is the portable lever, and the slot table makes it exact: one slot per 64 bytes of the region, so a channel of a given size admits a known number of records whatever their paths look like. `vtt stat-many` makes as many accesses as asked for, under distinct names so none can fold into one record, and prints last to show the process outlived them. The case skips musl, which has no preload: those builds collect through the seccomp supervisor, on the runner's own side of the boundary, so there is no shared-memory channel there to fill. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The case now drives the channel by record count rather than record size, because a Windows path record cannot get large enough to overrun one. With the counting lever it can do what it was written for: a 64 KiB channel holds a thousand records and the task makes twenty thousand, so the snapshot shows the task printing its last line and exiting cleanly, with the run reported as not cached. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`4 << 30` says how the number is built; `4 * 1024 * 1024 * 1024` says what it is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The size arrived through a builder on `fspy::Command`, a public default constant, and a `LazyLock` in the runner that read the override and passed it down. Three places to look, for a number with exactly one consumer. It now reads the override next to the `channel` call that uses it, and falls back to the default there. `Command` goes back to what it was, and so do the e2e tool, the examples, the benchmark launcher and fspy's own tests, none of which ever wanted a say in the size. The runner no longer names the variable at all, which also settles the musl question: `fspy::ipc` is already `cfg(not(target_env = "musl"))`, so the size lives behind the same gate as the channel it sizes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Two commands stat generated names to be tracked; one varied the name's length and the other how many names. They are now `stat-many <count> [name-length]`, which also puts the name in kebab case with every other subcommand. Count leads because it is the knob that travels. A long name only fills a channel on unix: on Windows a path reaches the tracer through a `UNICODE_STRING` whose length is a `u16`, so no single record there exceeds 64 KiB however long a name the caller asks for. Names now carry their index, so a run of them cannot collapse into one record, and padding fills out whatever length is asked for. The `/dev/shm` case keeps its one 1 MiB name as `stat-many 1 1048576`, and gains the trailing line that reports the process survived its accesses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…error The base moved the channel's size to a single read beside the `channel` call, so `Command` keeps nothing about it and `fspy::ipc` picks the slot count out of the byte count there. A lost record was an `Option<FrameReader>` behind an `is_complete()` that callers were free to ignore. It is a `TrackingIncomplete` error now, carried in `ChildTermination::path_accesses`, so nothing reads the accesses without meeting the failure first. The runner still declines to fail the task over it: it turns the error into a not-cached reason, which is the one place that policy belongs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Both entries described the mechanism rather than what a user sees. Crashing "mid-record", closing "inherited file descriptors" and keeping "every completed record intact" are things the tracker does; what a user hit was `vp run` hanging or failing, and a task getting killed partway through. The second entry also cites #533, the report of the abort. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`sender` decided for itself: a missing backing file came back as an error, and every other failure panicked inside the call. That put the policy in the one place that cannot know the caller's situation. It now returns every failure. Two error kinds say the channel is simply over — the receiver removed the file, or sealed it just before this call — and a caller meeting those has lost nothing by recording nothing. Both preload clients skip on those and panic on anything else, which is the same behaviour as before, now written where the decision belongs. Neither client prints on the skip. A preload library writing to the traced process's stderr corrupts whatever that process is printing, and a channel that closed before this process started is not news. The benchmark launcher gains a small shim over `ChildTermination`'s accesses field. The benchmark compiles that one source against both this revision's fspy and the merge base's, so a change to the field's type stops the base arm building; the shim spans both shapes. CI caught this, and the fix is verified by compiling the head launcher against main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The table's `SAFETY` comment said its alignment "follows from the start address being aligned for `Counters`, whose size is a multiple of that alignment". The claim is true, but its second half was nowhere asserted: it came out of `size_of::<Counters>() == 2 * size_of::<AtomicU64>()` and the fact that an `AtomicU64` is never aligned more strictly than its own width. A reader had to reconstruct that, and a later field could quietly break it. There is now an assert for exactly the step in question, and the comment names both asserts it stands on rather than restating the argument. A test checks the pointers `new` actually builds, across slot counts, since the asserts argue about the types and not the arithmetic. `meta_len` was left over from the `Meta` struct that used to hold both parts. It is `payloads_at` now, which is what it measures. `table_len` became `table_bytes`, so it stops reading like the slot count that `table().len()` returns. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Handing every failure to the caller gave both preload clients the same fifteen lines: skip on two error kinds, panic on the rest. Two copies of one policy, and a signature that made a caller re-derive it from `io::ErrorKind` before it could act. `sender` returns `Option<Sender>` now and decides for itself. `None` means the channel is already over, which is the only outcome a caller can do anything about, and it does the same thing either way: record nothing. Everything else stops the process where the cause is known, with the error in the message. Both clients are one line and a comment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…region `channel` attaches a throwaway writer to prove the region can host the protocol. Nothing tested it, and two panics depend on it: `sender` and `Receiver::close` both treat a region that cannot hold the protocol as impossible, on the grounds that creation already refused it. Removing the check and running this test shows what it buys: `channel` accepts the size, and the panic lands in `sender` instead — which runs in every traced process's preload, where the cause is furthest from the config that caused it. The size comes from an environment variable, so that is reachable by configuration, not only by a bug. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`tracking_fell_short` re-derived a condition its own caller already implied. fspy is attached only when `input` or `output` asks for inferred paths (`CacheState::new`), and the spawn flag is that same `fspy.is_some()`, so `path_accesses` is `Some` exactly when the task infers. The function tested `fspy.is_some() && infers && ...` where the first two are one fact. That mattered beyond the redundancy: because the check claimed to let a short trace past for a task that declares everything, `observe_fspy` had to be ready for an `Err` it could never see, and answered it by treating a lost trace as an empty one. A wrong trace and no trace are not the same thing, and nothing should have to decide that twice. One `let ... else` takes the accesses and turns the run away if they are not all of them. `observe_fspy` now receives what it uses, an `Option<&PathAccessIterable>`, and has nothing left to interpret. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The slot count travelled: chosen in `fspy`, passed into `channel`, written into the `ChannelConf` for senders to read back, and kept on the `Receiver` so it could seal. Both ends have to agree on it, and every hop was a chance to disagree. It is a constant in the channel layer now. `ChannelConf` carries only the shared-memory id again, `Receiver` only the mapping, and `channel` takes the capacity by itself. `ChannelSize` and `from_usize` existed to carry the number and are gone with it; `shm_io` still takes the count as an argument, since a protocol module should not know one caller's number. The value is the one the ratio produced at the size a tracked run gets: 2^26 slots, one per 56 payload bytes of 4 GiB. The table is half a gibibyte of sparse address space, untouched until slots are claimed, so a channel still pays only for the slots it uses. What changes is that a region now has to clear that table before it can hold anything: the e2e case asks for 512 MiB plus the 64 KiB of records it means to overrun, where it used to ask for 64 KiB flat. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The thread count was one constant shared by every suite, fixed at two — enough to represent a normal tracked process, but not enough to make writers fight over the channel's counters. Each record costs two atomic read-modify-writes on words every other thread is touching, and two threads barely provoke that. The count moves onto the suite, so the existing rows keep their two threads and their comparability, and a new `access-contended` row runs the same opens under eight. It halves the iterations over half the opens, so four times the threads cost about the same wall clock. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ublication-design-76917a

The thread count was one constant shared by every suite, fixed at two — enough to represent a normal tracked process, but not enough to make writers fight over the channel's counters. Each record costs two atomic read-modify-writes on words every other thread is touching, and two threads barely provoke that. The count moves onto the suite, so the existing rows keep their two threads and their comparability, and a new `access-contended` row runs the same opens under eight. It halves the iterations over half the opens, so four times the threads cost about the same wall clock. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The stack puts the contended access row underneath, so the rows this change's own benchmark run reports include it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The benchmark's contended row moved to the bottom of the stack, under the channel-sizing change, so this branch picks both up through its base. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The two branches below moved onto current main, which brings in #682's vt_server fix along the way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> # Conflicts: # crates/fspy/src/ipc.rs # crates/fspy/src/unix/mod.rs # crates/fspy/src/windows/mod.rs # crates/vt_bin/tests/e2e_snapshots/fixtures/fspy_shm_capacity/snapshots.toml # crates/vt_bin/tests/e2e_snapshots/fixtures/fspy_shm_capacity/snapshots/shm_capacity_env_sizes_the_tracking_channel.md

"tracking ran out of room for this task's file accesses" gave a reader three problems: `tracking` arrives as a bare noun for anyone who does not know Vite+ watches files, "ran out of room" reads as disk or memory and invites buying a bigger machine, and it left them nowhere to go. It now names the cause in the reader's own terms and points at the way out, the same shape as the message beside it for an OS without auto-inference. The advice works: a task that declares `input` and `output` never has tracking attached, so it cannot meet this at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

## Motivation The benchmark's thread count was one constant shared by every suite, fixed at two. Two threads represent a normal tracked process, but they barely make writers fight over the IPC channel's counters, and every recorded access costs two atomic read-modify-writes on words every other thread is touching. Changes to how a record is claimed can therefore look free here while costing real time under a parallel build. The count moves onto the suite, so the existing rows keep their two threads and stay comparable with earlier runs, and a new `access-contended` row runs the same opens under eight. It halves the iterations over half the opens, so four times the threads cost about the same wall clock. Split out of [#675](#675), where it was used to measure claim-path changes. Replaces [#679](#679), which GitHub auto-closed and deleted the branch of when its base briefly absorbed this commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

## Motivation The shared memory a tracked run reports its file accesses through is four gibibytes, fixed. Nothing could ask for a smaller channel, so no test could put a task in front of one too small to hold its records — and that path decides whether a run may be cached. It had no coverage at all. ## What changes `VP_RUN_INTERNAL_FSPY_SHM_CAPACITY` overrides the size, read next to the `channel` call that uses it and falling back to the same four gibibytes when unset. Nothing about a normal run moves. The variable is internal: it exists so a test can shrink the channel until a task overruns it, and nothing outside this repository should set it. The read sits at the point of use rather than travelling there. A size threaded through `Command` would put a builder method, a public default constant and a lookup in the runner between the variable and its one consumer, and would drag every other caller — the benchmark launcher, the e2e tool, the examples, fspy's own tests — into a decision none of them want to make. It also keeps the whole thing behind one `cfg`: `fspy::ipc` is already `cfg(not(target_env = "musl"))`, so the size lives behind the same gate as the channel it sizes. ## The test `vtt stat-many <count> [name-length]` stats generated names to be tracked, under distinct names so none can fold into a single record, and prints its last line afterwards to show the process outlived them. It absorbs `stat_long_filename`, which did the same thing along the other axis; the `/dev/shm` case that used it now says `stat-many 1 1048576`. The e2e case makes twenty thousand accesses under a 64 MiB channel, which holds every one, so the run caches like any other — that is what tells us the size arrived where it was meant to. Record *count* is the lever rather than record size, because size cannot be pushed far enough on every platform. A path reaches the tracer on Windows through a `UNICODE_STRING`, whose length field is a `u16`, so no single record there exceeds 64 KiB however long a name the caller asks for — an earlier version of this case tried one 2 MiB path and Windows had room to spare for it. The case skips musl, which has no preload: those builds collect through the seccomp supervisor, on the runner's own side of the boundary, so there is no shared-memory channel there to fill. The case worth testing, a channel too small for the task, has to wait for [#675](#675). On this base a full channel aborts the task process, and the panic it prints carries a thread id, a toolchain path, a backtrace and a platform's own abort code, so there is nothing there that snapshots the same way twice. #675 makes that a skipped record and a reported reason instead, and updates this snapshot to show it. Split out of #675. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

wan9chi and others added 2 commits August 14, 2026 16:13

docs: add changelog entry for crash-tolerant file-access tracking

2f5d389

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

wan9chi and others added 12 commits August 14, 2026 16:35

temp(fspy): benchmark phase instrumentation

d2fa1de

Temporary stderr phase timings to locate the CI launch regression on the Linux benchmark runner. Will be dropped before merge. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Revert "temp(fspy): benchmark phase instrumentation"

ab2749f

This reverts commit d2fa1de.

Revert "perf(fspy): preallocate first-touch blocks instead of pre-fau…

3e5bea8

…lting on a thread" This reverts commit 528d1d1.

Revert "perf(fspy): warm the first payload page alongside the header"

916e292

This reverts commit c4ba8dd.

docs(fspy-shm): add a protocol README to shm_io

0a80d2b

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

wan9chi force-pushed the claude/fspy-shm-publication-design-76917a branch from e560a84 to 0a80d2b Compare August 15, 2026 01:05

wan9chi and others added 14 commits August 15, 2026 09:10

docs(fspy-shm): say u64 and AtomicU64 instead of word

2d6aafa

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(fspy-shm): update the README for dynamic typed-view sizing

907a55d

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(fspy-shm): plainer wording in the README

c07dea9

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

wan9chi and others added 20 commits August 17, 2026 14:28

style(fspy): spell the default capacity as a byte count

a781d16

`4 << 30` says how the number is built; `4 * 1024 * 1024 * 1024` says what it is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Merge branch 'claude/fspy-bench-contended-row' into claude/fspy-shm-p…

a0ff60e

…ublication-design-76917a

Merge the benchmark's contended row below this change

187994d

The stack puts the contended access row underneath, so the rows this change's own benchmark run reports include it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Merge the restacked base

ade97bc

The benchmark's contended row moved to the bottom of the stack, under the channel-sizing change, so this branch picks both up through its base. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

wan9chi mentioned this pull request Aug 18, 2026

test(fspy-benchmark): add a contended access row #683

Merged

wan9chi merged commit c83bad8 into main Aug 18, 2026
19 checks passed

wan9chi deleted the claude/fspy-shm-publication-design-76917a branch August 18, 2026 03:28

wan9chi changed the title ~~fix(fspy): replace quiescence locking with crash-tolerant shared-memory publication~~ fix(fspy): collect file accesses without waiting for traced processes Aug 18, 2026

wan9chi mentioned this pull request Aug 18, 2026

fix(fspy): replace the IPC file lock with an in-mapping close gate #577

Closed

Navigation Menu

Uh oh!

fix(fspy): collect file accesses without waiting for traced processes - #675

fix(fspy): collect file accesses without waiting for traced processes#675
wan9chi merged 101 commits into
claude/fspy-shm-capacity-envfrom
claude/fspy-shm-publication-design-76917a

wan9chi commented Aug 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 14, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

wan9chi commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

What changed

When the region fills up

Known regression

Uh oh!

github-actions Bot commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

fspy benchmark

linux

macos

windows

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

wan9chi commented Aug 14, 2026 •

edited

Loading

github-actions Bot commented Aug 14, 2026 •

edited

Loading