CI: gc_write_barrier_stress red on main (missing old→young remembered-set edges → segfault)

[proggeramlug]

Summary

cargo-test is red on main: the GC write-barrier stress tests segfault deterministically.

test tenured_mutation_stress ... FAILED
test structured_clone_gc_churn_stress ... FAILED
  panicked at crates/perry/tests/gc_write_barrier_stress.rs:124:
  compiled binary failed (signal/segfault = missed write barrier)
  old-young-edge-verifier failed: checked_old_objects=3664 checked_remembered_pages=127
  checked_old_to_young_edges=37714 missing_edges=7710
  first_missing_parent=0x… first_missing_slot=0x… first_missing_child=0x…

Reproduced locally (release): cargo test --release -p perry --test gc_write_barrier_stress → 2 failed.

Diagnosis

The old→young edge verifier finds ~7710 remembered-set edges missing — i.e. some object-field stores that create an old→young pointer are not emitting/registering a write barrier, so a minor GC can miss roots and free live young objects → segfault. The stress harness covers structuredClone object fields and tenured-object mutation (cf. #4886 / #4956 GC barrier work). Likely a codegen write-barrier emission gap (or remembered-set registration gap) on a store shape the audit doesn't cover.

Scope

Pre-existing on main, independent of any in-flight feature PR. Surfaced while landing #5016 (PR #5022); its diff touches no GC code. Needs dedicated GC-context investigation. Distinct from #4985 (typed_shape_descriptors).

[proggeramlug]

Root-cause investigation (bisect + instrumentation results)

Bisect

git bisect between #4994 (da4f8be7f, last green cargo-test) and main, predicate cargo test --release -p perry --test gc_write_barrier_stress:

GOOD 5ce0d676f  (#4996)
BAD  44fd5c3c7  (#4998 — explicit gc() forces the conservative native-stack scan)
BAD  651181710, 0db801936, f6888301d, ... (everything after)

#4998 is the trigger, not the defect. On the GOOD commit 5ce0d676f, forcing the same scan via env reproduces the identical failure:

PERRY_CONSERVATIVE_STACK_SCAN=full PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1 ./clone_stress
→ old-young-edge-verifier failed: ... missing_edges=7710   (same signature as main)

So the defect is a pre-existing Full-conservative-scan × forced-evacuation remembered-set bug; #4998 merely made manual gc() use the Full scan by default, putting the default stress-test config on the buggy path.

It is real corruption, not just a verifier artifact

Several runs fail with CLONE_STRESS_CORRUPT 301 (every clone field bad) without the verifier panicking — live data is actually lost. One instrumented run dumped the first missing edge's child header as recycled garbage (obj_type=208 size=2147419223 flags=0xff): the old parent's slot points into swept/reused nursery memory.

Knob matrix (binary from current main)

knobs	result
FORCE_EVACUATE=0 VERIFY=1	OK
CONSERVATIVE_STACK_SCAN=0 FORCE=1 VERIFY=1	OK
default (#4998 path) FORCE=1 VERIFY=1	verifier panic or CORRUPT
CONSERVATIVE_STACK_SCAN=full FORCE=1 VERIFY=1	CORRUPT 301

The measured invariant violation

Instrumented remembered_dirty_page_count() at minor-cycle entry vs after the cycle's remembered_set_clear + sticky restore:

[minor prelude]  dirty_pages=119..130   ← mutator write barriers are fine
[after restore]  dirty_pages=9..10     ← cycle drops ~120 pages of coverage

The per-cycle remembered_set_clear() + rebuild (live_old_to_young_sticky from OldToYoungRememberedRebuildState + evacuation_sticky from rebuild_evacuated_old_to_young_remembered_set) restores only ~10 pages. Whatever young children survive that cycle without being evacuated lose their coverage; the next minor never scans those slots, sweeps the children, and the old parent (kept alive/marked by the Full conservative scan — cons_pinned=36k-40k objects per cycle in the diag) is left with dangling slots. Force-evacuation's rewrite then walks through garbage headers → corruption.

First missing parent (every run)

A 262160-byte array backing born directly in old-gen (large-object allocation in arena_alloc_gc routes >threshold allocations to arena_alloc_gc_old + TENURED) — the stress test's 30000-element garbage array. Its old→young edges depend entirely on barrier dirty pages (born old, never "promoted" through a scanning path), which makes it the first victim of the coverage drop. [rebuild-diag] shows the rebuild does walk it but adds ≤1 page while the verifier later counts 7710 uncovered edges on it.

Notable structural gap found while reading

The cycle.rs fallback minor has no equivalent of copying.rs's re-remember during the dirty-slot scan (visit_slot_with_parent → collector.sticky.remember_slot, copying.rs:554-557). In the fallback path the post-cycle remembered set is solely rebuild + evacuation stickies. Suspect ordering/coverage of that rebuild (it runs inside AtomicFinalize relative to evacuation/marking) is where the ~120 pages go missing — that is where I'd focus next.

Repro artifacts

• Standalone repro: extract structured_clone_gc_churn_stress's TS into a file, compile, run with the knobs above (fails in <10s, deterministic signature missing_edges=7710).
• tenured_mutation_stress fails on Linux CI as well; locally on macOS it passes — the clone test fails on both.

Risk note

This is not test-only: policy evacuation can enable organically under nursery/RSS pressure (no PERRY_GC_FORCE_EVACUATE needed), and #4998's Full scan now runs on every explicit gc(). A production workload with explicit gc() + large old-born arrays + policy evacuation can hit the same lost-coverage → swept-live-children corruption. I'd treat this as a correctness bug, not just a red gate.

Options to ungreen CI while the GC fix is developed

1. Land the real fix (re-remember surviving old→young slots in the fallback minor path / fix the rebuild coverage) — right answer, needs GC-owner-level care.
2. Temporarily #[ignore] the two stress tests with a reference to this issue — unblocks cargo-test but hides a real corruption bug.
3. Do not revert fix(gc): explicit gc() forces the conservative native-stack scan (#4977) #4998 — it fixes real reclaim-live-locals corruption (GC: explicit gc() (full collect, default auto stack-scan) reclaims live top-level locals — string fields read back as garbage #4977) and is only the exposure vector.

(Companion gate fixes already up: #5031 for the node_stream cargo-test red, #5032 for the h1_buffer_alias_negative compiler-output-regression red.)