ltm: link-score generation materializes one full guard-form arm per element of an arrayed target (quadratic in element count) #977

Summary

For a per-element (Ast::Arrayed) target, a link score is shaped over the whole target: one arm
per target element. An arm whose element equation does not reference the link's source with the
link's access shape has every source reference frozen by the ceteris-paribus wrap, so it is intended
to evaluate to zero — but it is still materialized as a full guard form wrapping a full copy of that
element's equation, and it is generated, printed, parsed, lowered, and executed every timestep like
any other arm.

With N distinct source elements over an N-slot target that is N^2+N+1 arms for a single causal
edge, of which N^2-N are the intended-zero kind. (The title's older (N+1)^2 figure is off by
the cross terms; the measured count is N^2+N+1.)

Scale

Synthetic fixture — dimension Wide with N elements; growth[Wide] a per-element
Equation::Arrayed flow whose element e equation is pop[e] * rate * 0.01; pop[Wide] an array
stock fed by growth; rate a scalar aux. Release build, timing db::model_ltm_variables:

Clean 4x per doubling. The salsa queries the emission reads are linear and negligible: at N=400,
model_ltm_reference_sites 5.2 ms + model_element_causal_edges 1.16 ms +
model_loop_circuits_tiered 0.81 ms = ~7 ms against 2,304 ms. The cost is in the emission body.

Real model — C-LEARN v77, LTM enabled (discovery mode; 7,001 synthetic variables). Of the 12,364
emitted guard-form arms, 10,570 (85.6%) come from build_arrayed_link_score_equation, and 9,514
(77.0%) have no live source reference at all — 7.63 MB of 9.63 MB of arm text (79.3%). LTM takes
the root flow program from 31,525 to 1,208,106 opcodes, and these arms are the bulk of it.

Scalar models are unaffected. WORLD3-03 has zero such arms; its own LTM cost is entirely
per-link guard scaffolding, which is a different problem.

Why the obvious collapse is unsafe, and what the safe form is

Dropping an arm is sound exactly when the frozen partial recomputes the target from the same inputs
that produced PREVIOUS(target) — i.e. when the numerator partial(frozen deps) - PREVIOUS(target)
is identically zero. That is what an absent Ast::Arrayed slot lowers to in
compiler::expand_arrayed_with_hoisting: a single AssignCurr(off, Const(0.0)).

Testing "the link's source stayed frozen" does not establish that, and this is the trap the
problem keeps setting. That test says nothing about everything else the arm reads. Measured on
C-LEARN, omitting on that criterion changes 187 result slots across 35 link-score variables, 151
of them by ≥ 1.0, with a worst case of 8,086.97 → 0. A ±1 link score is the canonical
single-input attribution and sets its loop's score, so this can drop loops out of the ranking. It is
also unstable: fixing any of the wrap defects below silently changes which arms it selects.

The safe test is the positive one: is the transformed partial provably PREVIOUS(target)? —
i.e. outside every PREVIOUS(...)/INIT(...) subtree the arm contains only literals, operators,
keywords and pure builtins, with no bare variable reference and no time-dependent builtin. This asks
the question the soundness condition actually poses, and it keeps asking it after the wrap defects
are fixed. Measured on C-LEARN: 4,036 arms, −17.4% of the flow program, 0 of 30,123 result slots
differing over 251 steps.

A prior attempt built the negative-criterion collapse and withdrew it after seven adversarial review
rounds, each finding a different model whose "trivial" arm was not zero. Those seven escapes are all
instances of one root cause — the wrap does not freeze everything the arm reads that varies —
and the positive predicate excludes every one of them by construction rather than by enumeration.
That history yields two standing constraints for any future work here:

• A predicate arm that matches a node without positively establishing a verdict is the failure
  mode. Make it unrepresentable: every arm returns a named Reach { Established, NotEstablished },
  the BuiltinContents walk goes through a verdict-returning fold rather than a unit-returning
  callback (so => {} is a type error), and matches are exhaustive with no catch-all (so a new
  Expr2/IndexExpr2 variant is a compile error). This makes roughly half the escape class
  unwritable.
• The other half cannot be caught by any type, because the arm's justification is a claim about
  what the wrap — two modules away — does to the node. Only a fixture catches those. Keep a
  differential harness comparing whole result slabs over every slot as the primary evidence;
  comparing only offsets' base slots is blind to exactly the per-element arms at issue, which is
  how an over-eager collapse first passed review.

What blocks the remaining 5,478 arms

Decomposition of the 9,514 no-live-source arms on C-LEARN:

The unwrapped-source row was 322 when this issue was written and measures 0 today, with the
arithmetic closing exactly (+299 provable, −12 time-only, −322 unwrapped, −2 live = −37, against a
total that moved 9,514 → 9,477). Roughly 299 of those arms are now fully frozen and therefore
provably PREVIOUS(target), consistent with the raw-vs-canonical spelling mismatch having been
fixed since — #986 is the obvious candidate. Do not budget work against the 322.

That zero was controlled for rather than assumed: every residual arm carrying a live head was
recorded, and all 14 are one shape — lookup(rs_co2_ff[g77_china], time() / PREVIOUS(one_year)) / PREVIOUS(rs_ff_co2_ff_aggregated[aggregated_regions·developing_a_countries]). The source is
demonstrably frozen there; what is live is a lookup table and the time() in its index, i.e.
#984 plus #1016, not an unwrapped source. Both spellings #977 originally described are present in
that sample — the raw one is the lookup table's own static element selector, the canonical one sits
inside the freeze — so the spelling pair is real and reachable; the consequence attributed to it is
not. Scope caveat: the probe sits where the omission policy is live (Ast::Arrayed targets with
apply_default_to_missing == false), so EXCEPT-default and whole-variable paths are outside it —
at most the 37-arm gap. Measured on C-LEARN only, which is where the 322 was measured.

Relaxing the predicate does not recover the residual: treating lookup(...) as pure in its
arguments buys exactly zero additional arms (those arms hit a live time() inside the lookup's
own index immediately afterwards), and no other pure builtin appears. An AST-walk implementation is
preferable to a text scan for provability and maintenance, but it will not classify materially more
on this model.

TIME. db/ltm/compile.rs skips time/dt/initial_time/final_time during dependency
processing, and the wrap freezes dependencies — so where a target reads time(), a "fully frozen"
partial still advances the clock and the target's autonomous time evolution is attributed to the
link. docs/reference/ltm--loops-that-matter.md:231-233 defines the partial as re-evaluating with
"the previous values of all other inputs"; the formula only closes if that covers everything f
reads that varies. Two independent confirmations: all 35 link-score variables whose values move
under the naive collapse carry a live time-dependent term, and 26 of them sit at exactly ±1.0 —
the "all of Δz came from the clock" case. Under a TIME-freezing convention these 5,035 arms are zero
by construction and omission stops being an approximation.

That is a scoring-semantics change, not a refactor: 6,975 of 16,967 link-score arms (41.1%)
retain a live time-dependent term, so it moves roughly two in five link scores on C-LEARN and needs
dominance-ranking validation. It is tracked as #1016; #763 is the reducer-body instance of the
same root cause. Nobody has yet measured what the new scores would be — only how many arms it
unblocks and how many partials it touches.

Unwrapped bare variables. A raw-vs-canonical element-spelling mismatch leaves the live source
unwrapped in the partial (raw [developing_b_countries] against canonical
[aggregated_regions·developing_b_countries]) while the shape match records no live reference.
#984 tracks the LOOKUP-table-index instance of a live read surviving a partial; whether these 322
share that root cause is unverified.

Components

• src/simlin-engine/src/ltm_augment.rs — build_arrayed_link_score_equation,
  shaped_guard_form_text, wrap_non_matching_in_previous
• src/simlin-engine/src/db/ltm/link_scores.rs — emit_per_element_link_scores
• src/simlin-engine/src/db/ltm/equation.rs — LtmEquation::Arrayed, arm representation
• src/simlin-engine/src/compiler/mod.rs — expand_arrayed_with_hoisting (the absent-slot lowering
  the omission relies on)

Constraints on the fix

• An intentional omission must be recorded by a distinct marker, never by an empty arm.
  equation.rs deliberately distinguishes "legitimately empty ⇒ drop ⇒ zero-fill" from "failed to
  parse ⇒ reject"; reusing the empty channel makes a generation bug indistinguishable from an
  intended zero slot. ltm-503-cross-element-agg.AC1.3 guards against a generator giving up and
  emitting a literal "0" — point that guard at the marker, not at the arm text.
• Omission is sound only where apply_default_to_missing == false; an EXCEPT-default target's
  missing slot takes the default rather than zero.
• Two disclosed value-neutrality qualifiers: an omitted slot's Const(0.0) is +0.0 where a
  materialized trivial arm ends in ... * SIGN(dx) and can be -0.0 (inert — ltm_post
  accumulates |score|); and a materialized NaN - NaN = NaN slot becomes 0, which removes an
  engine-manufactured NaN while leaving the modeller's own on live arms.
• A regression gate must assert on arm count / structure, not wall-clock, per the test-time
  budgets in docs/dev/rust.md. db::ltm_tests::per_element_generation_scaling (#[ignore]d) is
  the scaling harness:
  cargo test -p simlin-engine --release --lib -- --ignored --nocapture per_element_generation_scaling
• There is no value-level LTM gate at any scale: clearn_residual_exactness never enables LTM,
  and clearn_ltm_var_count_guardrail pins variable count and slot width, neither of which arm
  omission moves. A change that rewrote 149 C-LEARN LTM slots to zero passed every named C-LEARN
  gate. Any work here needs its own value-level guard.

What remains quadratic afterwards

With the collapse in place at N=400, generation was 519.7 ms, of which 439 ms is
db::ltm::compile's two reconstruct_single_variable calls, which parse and lower the whole
N-slot target from scratch once per score. Do not over-attribute a collapse's win.

Related

• ltm: residual ~30k synthesized PREVIOUS helper-aux vars on C-LEARN threaten the VM 65k-slot ceiling #654 — residual synthesized helper-aux count vs the VM slot ceiling; quadratic arm emission feeds it
• ltm: should a ceteris-paribus partial freeze TIME? (today it does not, and the target's autonomous time evolution is attributed to the link) #1016 — should a ceteris-paribus partial freeze TIME? gates the 5,035-arm residual here
• ltm-augment: TIME-bearing reducer bodies break the PREVIOUS(agg) anchor in body-aware partials (nonlinear arm attributes full time-drift to every row) #763 — TIME-bearing reducer bodies breaking the PREVIOUS(agg) anchor: same root cause, reducer-body site
• ltm: the ceteris-paribus wrap leaves a LOOKUP table argument's runtime index live in every link-score partial #984 — LOOKUP table argument's runtime index left live in every partial
• epic: Loops that Matter (LTM) -- tracking known issues and improvements #488 — LTM epic