Repository navigation
engine: LTM cross-element aggregate scoring (aggregate nodes) - #519
Merged
Merged
Conversation
added 30 commits
May 9, 2026 10:57
The LTM synthetic-variable record carried its equation as a bare `String` plus a separate `dimensions: Vec<String>` field. Phase 1's feature work needs a link-score variable to be able to hold an `Equation::Arrayed` (per-element partial equations for arrayed-target link scores), so the `equation` field becomes a `datamodel::Equation`. This is a representation prerequisite only -- no behavior change. The link-score generators now return `datamodel::Equation` (Scalar for scalar targets, ApplyToAll for arrayed targets; Arrayed is added in Task 2), the shared link-score-guard wrapping is factored into one helper, and every `LtmSyntheticVar` constructor keeps the equation's dimension names in lockstep with the retained `dimensions` field (which `compute_layout` and `parse_link_offsets` still key off of). `parse_ltm_equation` / `compile_ltm_equation_fragment` take the typed equation instead of a `(text, dims)` pair. `LtmSyntheticVar`/`LtmVariablesResult` drop `Eq` and gate `Debug` on `debug-derive` (the embedded `GraphicalFunction` carries f64 points and datamodel types only derive `Debug` under that feature); salsa only needs `PartialEq` and nothing uses either type as a hash key.
When a link-score target's AST is `Ast::Arrayed` (per-element-equation form), the link-score generator previously fell through to a `"0"` placeholder partial -- so a cross-element aux like `migration_pressure[NYC] = (population[NYC] - population[Boston]) * 0.01` got an `ApplyToAll` equation whose partial was the constant `0`, and the link score was meaningless. Now `generate_auxiliary_to_auxiliary_equation` and `generate_stock_to_flow_equation` route an `Ast::Arrayed` target through `build_arrayed_link_score_equation`, which emits an `Equation::Arrayed` over the target's dimensions whose per-element slot equation is the standard link-score guard form with that element's *own* partial: the slot for `migration_pressure[NYC]` carries `(population[nyc] - PREVIOUS(population[boston])) * 0.01` when the link score's shape is `FixedIndex(["nyc"])`. An element whose equation doesn't reference the source with this shape gets its source refs frozen, so that slot evaluates to ~0 -- correct, because that source-element's influence flows through a different `(from[other], to)` link-score variable. The per-element dep set is computed by walking each slot's expression alone (the union over all slots would over-freeze) with the target's dimensions passed in so literal element-name subscripts are not mistaken for variable references. The shared "wrap a partial in the guard form" logic is `link_score_guard_form`; the scalar/A2A equation-text extraction is factored into `scalar_or_a2a_target_equation_text`. Output slots are sorted by element name so the salsa-tracked result is deterministic.
`test_stock_to_flow_link_score_handles_apply_to_all` pinned the `Equation::ApplyToAll` flow case; this adds the sibling for the `Ast::Arrayed` (per-element-equation) flow case that the prior commit fixed. A `population[Region]` stock with a per-element-equation `births[Region]` inflow now yields an `Equation::Arrayed` link score whose every per-element slot references the flow's actual equation contents and carries no `(0)` placeholder partial. The non-shaped `link_score_equation_text` scalarizes its result, so the test goes through `link_score_equation_text_shaped` with a `FixedIndex` shape (each `births[e]` references `population[e]`, which is how the per-shape emission keys these link scores).
End-to-end check on the `cross_element_ltm` fixture: the `migration_pressure[boston] -> migration_in` link score (where `migration_in` is a per-element-equation flow) now carries a meaningful per-element partial instead of the pre-fix `"0"`-partial-derived value that was far from 1. `migration_in[NYC] = MAX(migration_pressure[Boston] * -1, 0)`, so the partial w.r.t. live `migration_pressure[boston]` is exactly all of `migration_in[NYC]` -- `Δpartial == Δmigration_in[NYC]` and `ABS(SAFEDIV(Δ, Δ)) == 1` at every step where `Δ != 0` (which holds for t >= 2, since `migration_pressure[Boston] < 0` throughout and the population gap keeps growing under the uniform birth rate). The Boston slot references only `migration_pressure[nyc]` (frozen at PREVIOUS under the `FixedIndex(boston)` shape) and `migration_in[Boston]` is constantly 0, so that slot is identically 0.
…est)
Two follow-ups from the Phase 1 code review of the LTM cross-element
aggregate scoring work:
- Document why `scalar_or_a2a_target_equation_text` falls back to the
raw datamodel `eqn` text (and ultimately `"0"`): the path is reached
only when the target variable failed to lower its equation to an AST,
in which case using whatever scalar equation text it still carries is
strictly more useful than a `"0"` partial -- the stock-to-flow
generator has always done this for the same variable shape. The
duplicated `Variable::{Stock,Var}` match arm is extracted into
`scalar_eqn_text_or_zero` so the rationale lives in one place.
- Add unit tests for `Equation::source_text()` covering the scalar,
Apply-to-All, and Arrayed variants, including the Arrayed-with-EXCEPT-
default branch that no prior test exercised.
No behavior change.
`retarget_ltm_equation_dims`'s doc comment promises that empty `dims` collapses the equation to `Scalar`, and the call site in `emit_per_shape_link_scores` documents that an incompatible-dimensions arrayed-target edge produces a scalar link score. The `Arrayed` arm, however, unconditionally rebuilt an `Arrayed` with `dims.to_vec()` -- so for empty `dims` it yielded a degenerate zero-dimension `Arrayed` rather than a `Scalar`. That arm assumed a per-element link-score equation is only ever emitted for a target with non-empty dims, but the arrayed-target / incompatible-source-dims edge (whose target also has per-element equations) reaches it: `try_cross_dimensional_link_scores` declines non-scalar targets, `link_score_dimensions` returns `[]` for the incompatible-dims case, and `build_arrayed_link_score_equation` produces an `Equation::Arrayed`. Route the empty-`dims` `Arrayed` case through the existing `scalarize_ltm_equation` helper so the function matches its own contract. A zero-dimension `Arrayed` is meaningless -- its per-element partials have no target dimension to index -- so collapsing is the correct choice.
Adds a direct unit test for the previously untested Equation::Arrayed arm of scalarize_ltm_equation (multi-slot picks the first slot, no slots with an EXCEPT default picks the default, no slots and no default falls back to "0"), plus quick guards for the Scalar/ApplyToAll arms. The function now has a second caller (retarget_ltm_equation_dims, for collapsing a degenerate zero-dimension Arrayed); its rustdoc previously named only the legacy link-score path, so add a note explaining why that second caller collapses to scalar.
`resolve_link_score_name_for_loop` gains a `target_element` parameter and
`generate_loop_score_equation` learns to emit a subscripted reference
`"$⁚ltm⁚link_score⁚{from}→{to}"[e]` when a loop link's `to` node carries
an element subscript -- which happens (after Phase 2's loop-builder
rewrite) when the target link-score variable is A2A and the cross-element
loop visits a single element of it. A bracketed `from` is now resolved by
trying the per-source-element FixedIndex name first and the variable-level
(Bare / Wildcard / DynamicIndex) name as a fallback, so the structural
flow-to-stock link score on a cross-element loop edge resolves to the
variable-level A2A name it is actually emitted under. `find_fixed_index_
emitted_name` prefers an exact `{from}[{e}]->{to}` match when the visited
element is known instead of guessing alphabetically.
No behavior change on its own: with `target_element = None` everywhere
(the only callers until the loop builder produces element-subscripted
`Link.to`s) the output is byte-identical to before.
`build_element_level_loops`'s `is_cross_element` branch no longer collapses
a cross-element circuit onto the diagonal apply-to-all link scores. The
"shortest unique cycle" stripping and the diagonal `circuit_to_links`
call are gone; instead each circuit becomes its own scalar `Loop` whose
`Link`s keep the element subscripts (`population[nyc]`,
`migration_pressure[boston]`, ...) -- `from` is kept subscripted when the
source node is subscripted, and `to` is kept subscripted when the target
node is subscripted and its variable is dimensioned (so the loop visits a
single element of an A2A link score). `generate_loop_score_equation` then
emits `"$⁚ltm⁚link_score⁚{from}->{to}"[e]` for each such hop, so a loop
like `population[nyc] -> migration_pressure[boston] -> migration_in[nyc] ->
population[nyc]` is scored as the product of the actual per-element link
scores along its path. A new `build_element_subscripted_links` helper
factors out the per-link rule; the element-level stock collection (needed
by `partition_for_loop`, which keys element-level) is retained.
The link-score emission loop for exhaustive mode strips the subscript off
both ends of each loop `Link` before calling `try_cross_dimensional_link_
scores` / `emit_per_shape_link_scores` (which key on variable-level
names), so the full per-element link score for each `(var_from, var_to)`
edge is emitted and the loop-score equation's `[elem]` subscript picks the
slot the loop visits.
Pure-A2A loops (the dimensioned-loop-score branch) and the mixed/scalar
branch are unchanged. Cross-element circuits are no longer deduped into a
single Loop, so a model like the `cross_element_ltm` fixture gets one
loop-score variable per cross-element circuit; the postscript-measurement
fixtures count element-level circuits (unchanged), not Loops.
The Phase 2 rewrite left two minor papercuts. The `build_element_level_loops` visibility comment still claimed unit tests inspect a per-link `shape` field; there has never been one -- the element-subscripted access is encoded in the `Link.from` / `Link.to` strings -- so the comment now describes that accurately. And the rewrite added a second copy of `strip_subscript` (plus a sibling `split_node_subscript`) in `ltm_augment.rs` alongside the existing one in `db::db_ltm`. Both modules already lean on `crate::ltm`, which is where the small LTM string/cycle helpers live (`canonical_rotation`, `lex_smallest_rotation_start`), so the two subscript helpers move there as `pub(crate)` and both call sites import them. The `db_analysis` mirror-comment is updated to point at the new home.
These two pure helpers moved into crate::ltm in the Phase 2 cross-element aggregate work and previously had no direct coverage (they were exercised only via callers in db_ltm / ltm_augment). Add example-based tests next to the existing canonical_rotation helper tests covering bare names, single-element subscripts, and multi-dimensional subscripts so the identifier-parsing semantics are pinned independently of any caller.
A scalar-source -> arrayed-target link score (e.g. `total_pop -> migration`
where `total_pop` is scalar and `migration[Region]` is arrayed) was emitted
as a single Bare-A2A `LtmSyntheticVar` with `dimensions = [target_dims]`.
That form is undiscoverable: `parse_link_offsets`'s `expand_a2a_link_offsets`
subscripts *both* sides over `target_dims`, inventing a phantom
`total_pop[nyc]` node that doesn't match the unsubscripted `total_pop` node
the reducer edges (`pop[d] -> total_pop`) produce, so any loop through
`total_pop` is unreachable in the strongest-path search graph.
Emit one scalar `LtmSyntheticVar` per target element instead, named
`$:ltm:link_score:{from}->{to}[{elem}]` with `dimensions: vec![]`, mirroring
the arrayed->scalar `{from}[{elem}]->{to}` convention from
`try_cross_dimensional_link_scores`. `parse_link_offsets`'s `[`-in-`to`
single-passthrough branch parses these to `(from, to[elem])` with no parser
change. New `try_scalar_to_arrayed_link_scores` in `db_ltm.rs` is routed
ahead of `emit_per_shape_link_scores` in both link-score emission loops;
`link_score_dimensions` now returns `vec![]` for the scalar-source case so
the routing-bypass fallback degrades to a harmless scalar Bare var rather
than a phantom-node-bearing A2A one. Per-element equations come from
`generate_scalar_to_element_equation` / `subscript_idents_at_element` in
`ltm_augment.rs`: the partial of `to[elem]`'s equation w.r.t. `from` live,
arrayed deps pinned to `elem`, wrapped in the standard link-score guard
form. `assemble_module` compiles any link score whose name carries a `[`
on either side of the arrow directly from `ltm_var.equation` rather than
through the `(from, to)`-keyed salsa path (which can't round-trip a
bracketed name back to a user variable).
So the system stays consistent, this also threads the per-target-element
name through `build_element_level_loops`'s mixed/scalar branch (keep the
`to[e]` subscript when the target is arrayed) and the loop-score-equation
generator (`loop_link_score_ref`: reference the per-element scalar variable
bare when it exists, else fall back to the dimensioned-A2A subscript-after-
quote form); the Bare-A2A link-score name is no longer emitted for these
edges, so loop-score equations that used to reference it now reference the
per-element scalar variables instead.
The per-target-element emission and the loop-score-equation wiring landed
together in the previous commit (a green build requires both: emitting the
new `$:ltm:link_score:{from}->{to}[{elem}]` names without updating
consumers leaves loop-score equations referencing a no-longer-emitted
Bare-A2A name). This adds the dedicated coverage:
- `scalar_reducer_loop_score_uses_per_element_link_scores` (unified tests):
the loop `population[nyc] -> total_pop -> migration[nyc] ->
population[nyc]` -- a scalar reducer (`total_pop = SUM(population[*])`)
factored out of the per-element migration flow -- has its loop-score
equation built from exactly the three per-element link-score references
along its element-level path: `"...population[nyc]->total_pop"`,
`"...total_pop->migration[nyc]"` (the new scalar->arrayed per-target-
element name, referenced bare), and `"...migration->population"[nyc]`
(the structural flow->stock A2A link score, subscripted-after-quote).
It also asserts no loop score references the retired Bare-A2A
`total_pop->migration` name.
- `test_scalar_reducer_loop_score_value_matches_hand_calc` (simulate_ltm):
the same loop's `loop_score` series equals the product of its three
per-element link-score slots at every step t >= 2 (within 1e-6) and is
non-zero at some step.
Discovery-mode end-to-end coverage for the cross-element loop-finding the preceding two commits enable (no new production logic): - `test_cross_element_ltm_discovery` (tightened): on the `cross_element_ltm` fixture, assert that some discovered loop carries the genuine cross-element edge `population[nyc] -> migration_pressure[boston]` (or the symmetric `population[boston] -> migration_pressure[nyc]`) -- the FixedIndex-source A2A link score `population[nyc]->migration_pressure` expands via `expand_fixed_from_a2a_link_offsets` to per-target-element edges, keeping that edge in the search graph -- not merely "some subscripted loop". - `test_scalar_reducer_loop_discovery` (new): on a model that factors a scalar reducer (`total_pop = SUM(population[*])`) out of the per-element migration flow, assert discovery finds the loop `population[*] -> total_pop -> migration[r] -> population[r]` with `total_pop` *unsubscripted* on both incident edges (`(population[nyc], total_pop)` and `(total_pop, migration[nyc])`), and that no loop endpoint is a phantom `total_pop[nyc]`. Pre-fix this loop was silently undiscoverable: `total_pop -> migration` was a Bare-A2A link score with `dimensions = ["Region"]`, so `parse_link_offsets`'s `expand_a2a_link_offsets` subscripted both sides and the invented `total_pop[nyc]` node never matched the unsubscripted `total_pop` from the reducer edge. `discovery_loops_have_link` / `discovery_loops_debug` test helpers added for these assertions. No defensive guard added in `expand_a2a_link_offsets`: it only has the link-score var's `dimensions`, not the source variable's actual shape, so it can't tell a true Bare-A2A edge from a scalar->arrayed one -- and Task 1 stops emitting Bare-A2A names for scalar->arrayed edges, so the mis-expansion path is unreachable from emission.
Two Minor items from the Phase 3 (LTM cross-element aggregate scoring) code review: - src/simlin-engine/CLAUDE.md described only the arrayed-source -> scalar-target reducer link score helper (try_cross_dimensional_link_scores) in the db_ltm.rs paragraph; add the symmetric clause for the new sibling try_scalar_to_arrayed_link_scores (scalar source -> arrayed target, element in the `to` side), including why a single Bare-A2A var would be undiscoverable. - try_scalar_to_arrayed_link_scores recomputed the target's identifier set (and its equation text) inside the per-element loop for the ApplyToAll arm, where both are element-invariant. Split the per-element loop by AST variant: the ApplyToAll case computes body/deps/deps-to-pin once before the loop; the Arrayed case keeps per-element recomputation, since each element genuinely has its own slot expression.
The reducer link-score machinery in `try_cross_dimensional_link_scores` previously handled only full reduces -- an arrayed source feeding a *scalar* target through SUM/MEAN/MIN/MAX/STDDEV/RANK -- and early-returned None whenever the target was arrayed. That left partial reduces (`agg[D1] = SUM(matrix[D1,*])`, which collapses only the D2 axis) unscored even though `link_score_dimensions` already contained the partial-collapse branch. A partial reduce is recognized by the target's dims being a strict subset of the source's (matched by name); the implied reduced axes are the difference, so the co-reduced `matrix[d1,*]` slice for each row is derived directly from the source element tuples without re-deriving the implicit `[D1,*]` subscript. For each `(d1, d2)` pair the relevant target is only `agg[d1]`, so the link score is a per-`(d1, d2)` *scalar* variable named `$⁚ltm⁚link_score⁚matrix[d1,d2]→agg[d1]` (both axes ride in the source subscript, only the surviving axis in the target subscript), `dimensions: vec![]`. That naming is consistent with the existing full-reduce per-source-element naming and rides `parse_link_offsets`'s element-level-on-both-sides single-passthrough branch, so no parser change is needed -- the alternative (`...→agg` with `dimensions = ["D1"]`) would route through `expand_fixed_from_a2a_link_offsets`, broadcasting over D1 and producing wrong edges. `generate_element_to_reduced_equation` is a sibling of `generate_element_to_scalar_equation` that subscripts the target reference by the result-axis element; both now share the wrapping body, so the full-reduce path is byte-identical. STDDEV/RANK and nested reducers keep their delta-ratio fallback against `agg[d1]`, unchanged (out of scope: #483).
End-to-end coverage for AC4.6: a `matrix[D1,D2]` stock feeds `row_sum[D1]
= SUM(matrix[D1,*])` (a partial reduce) which feeds back into the stock, so
the loop `matrix[d1,d2] -> row_sum[d1] -> ... -> matrix[d1,d2]` runs over
the reduced axis. The test asserts the per-`(d1,d2)` partial-reduce link
scores are present with non-degenerate values (a SUM partial reduce splits
the row delta, so the per-element magnitudes are a fraction strictly
between 0 and 1 and sum to ~1), a loop-score equation references them, and
a hand-calc value relation holds.
Two gaps in loop participation surfaced and were fixed (the phase plan
anticipated these as minimal fixes the integration test would surface):
- The mixed/scalar branch of `build_element_level_loops` stripped the
source subscript whenever the target node was arrayed, which collapsed
`matrix[d1,d2] -> row_sum[d1]` to `matrix -> row_sum[d1]` -- but the
partial-reduce link score is named with both axes in the source
subscript. A new `is_partial_reduce_edge` predicate (same dims-difference
rule `try_cross_dimensional_link_scores` uses) keeps both subscripts for
that edge.
- `loop_link_score_ref` only matched the per-target-element scalar link
score name (`$⁚ltm⁚link_score⁚{from}→{to}[{e}]`) when `link.from` was
unsubscripted (the scalar-source -> arrayed-target case). The
partial-reduce link score has the same name shape but an element-level
`link.from`, so the guard was dropped -- the name is now matched
verbatim for any `link.from`, which is safe because FixedIndex ->
arrayed edges never emit that element-in-name form.
The conservative full-cross-product element graph for `SUM(matrix[D1,*])`
still produces spurious cross-element loops (Phase 5 tightens that); the
assertions only require that a real partial-reduce link score is emitted,
non-degenerate, and referenced -- not that the loop set is exactly the four
clean 3-cycles.
The part-(b) comment in test_partial_reduce_cross_element_loop described the loop through row_sum as a 3-cycle (matrix -> row_sum -> growth -> matrix), but growth references the scalar full-reduce total, not row_sum directly, so the actual cycle is the 4-cycle the helper's doc comment already describes (matrix -> row_sum -> total -> growth -> matrix). Correct the comment to match, and reword the analogous mention earlier in the test to not pin the cycle length -- the assertions only require that some loop_score references a partial-reduce link score, independent of which cycle it lands in. Comment-only change; no test logic touched.
Phase 5 of the LTM cross-element-aggregate-scoring design treats each
maximal inlined array-reducer subexpression (SUM(pop[*]), MEAN(...), ...)
as an implicit "aggregate node" so causality can route source[d] -> agg ->
target instead of all-pairs source[d] -> target[e]. This adds the shared,
salsa-tracked enumerator that both consumers (model_element_causal_edges
and model_ltm_variables, wired in later subcomponents) will use to agree on
a deterministic agg name for each subexpression.
A variable whose entire dt-equation is exactly one reducer call is its own
aggregate node (no synthetic minted); a reducer used as a sub-expression
mints $:ltm:agg:{n}; AST-identical subexpressions dedupe via printed
equation text (Expr2 is not Eq). Deterministic by construction: variables
visited in canonical-sorted order, AST walked left-to-right depth-first,
synthetic names assigned 0, 1, ... in first-encounter order.
Reducers over slices/partial reduces used as sub-expressions are
deliberately not recognized yet (the conservative full-cross-product
element graph stays in place for those); whole-RHS partial reduces
(row_sum[D1] = SUM(matrix[D1,*])) are recognized as variable-backed aggs
carrying their result dims.
No consumers yet -- the enumerator is registered in lib.rs and exported for
the following Phase 5 subcomponents.
…nt graph Phase 5 of the LTM cross-element-aggregate-scoring work: `model_element_causal_edges` now consults `enumerate_agg_nodes` so a Wildcard/DynamicIndex reducer reference to `from` inside target `to` -- where `to`'s equation hoists a synthetic aggregate node reading `from` -- emits `from[d] -> $⁚ltm⁚agg⁚n` (per source element) and `$⁚ltm⁚agg⁚n -> to[e]` (per target element / scalar) instead of the all-pairs cross-product. A reference to `from` that is not inside a hoisted reducer (a bare numerator `pop[r]`, a FixedIndex `pop[NYC]`, a non-hoisted slice) still emits its own edges normally; variable-backed aggs like `total_population = SUM(pop[*])` are already real nodes and are not rerouted. `model_loop_circuits_tiered`'s slow-path subgraph projection now keeps synthetic agg nodes (they have no variable-level counterpart, but a cross-element loop through a hoisted reducer genuinely traverses the agg, so dropping it would hide that loop). A slice-reducer subexpression (`x[r] = ... + SUM(pop[NYC, *])`) is deliberately not hoisted as a synthetic agg: the agg descriptor only carries the source variable name, not which elements the slice reads, so the reroute and the per-element reducer link scores would over-approximate the unread rows with nonzero garbage. Such a subexpression stays conservatively Wildcard-classified (tracked as tech debt). A whole-RHS slice/partial reduce (`agg[D1] = SUM(matrix[D1,*])`) is still recognized, but as a variable-backed agg.
Phase 5: `model_ltm_variables` now emits one `$⁚ltm⁚agg⁚{n}` synthetic
auxiliary per synthetic aggregate node from `enumerate_agg_nodes` -- a plain
computed aux whose equation is the maximal inlined reducer subexpression
(`SUM(pop[*])`, ...). The simulation evaluates it each timestep (so
`PREVIOUS(agg)` is available via the regular `prev_values` snapshot) and the
two link-score halves (`source[d] -> agg`, `agg -> target`) reference it.
Model equations are not rewritten -- the agg aux yields the same value as the
inline reducer. A whole-RHS reducer (`total_population = SUM(population[*])`)
mints no synthetic; the variable already is the aggregate node.
The agg vars get a sort category strictly before link-score vars in the
returned `vars` list, and the category function uses the `$⁚ltm⁚agg⁚` prefix
(not a `⁚agg⁚` substring search) so the `agg -> target` link score
(`$⁚ltm⁚link_score⁚$⁚ltm⁚agg⁚{n}->{to}`, which contains `⁚agg⁚`) stays a
link score that runs after the agg aux. `compute_layout` section 3 re-sorts
LTM vars purely by name -- `$⁚ltm⁚agg⁚{n}` < `$⁚ltm⁚link_score⁚...` lexically
-- so the agg gets its layout slot first there too; the runlist order (the
same-timestep ordering hazard) comes from the `model_ltm_variables` sort and
the `ltm_flow_names` iteration that follows it.
(The `agg -> target` link score equation emitted here still goes through the
generic per-shape fallback, which yields a zero partial because the reducer
subexpression isn't substituted by the agg name yet -- fixed in the next
commit.)
Phase 5: a Wildcard/DynamicIndex reducer reference to `from` inside target
`to` -- where `to` hoists a synthetic aggregate node `$⁚ltm⁚agg⁚n` reading
`from` -- now produces two link-score halves instead of a `…→to⁚wildcard`
per-shape link score:
(a) `$⁚ltm⁚link_score⁚{from}[{d}]→{agg}` per source element of `from`: the
source element's fractional contribution to the aggregate's velocity.
The agg's own equation is exactly the reducer, so the "bare" algebraic
shortcut applies (varying `from[d]` changes the agg by exactly its own
delta regardless of what else the reducer combines).
(b) `$⁚ltm⁚link_score⁚{agg}→{to}[{e}]` per target element (or a single
scalar `$⁚ltm⁚link_score⁚{agg}→{to}` for a scalar `to`): the partial of
`to`'s equation w.r.t. `agg` held live, with every hoisted reducer
subexpression in `to` first AST-substituted by its agg name (so `agg`
appears live where `SUM(...)` was and any other hoisted reducers end up
as `PREVIOUS(agg_j)`).
The AST-subtree substitution (`substitute_reducers_in_equation`) matches on
the parsed-AST subtree, not a substring of the equation text, so a reducer
text that is a textual prefix of a different reducer subexpression
(`sum(p[*])` vs `sum(p[*] + 1)`) is never falsely matched.
The Bare numerator / FixedIndex references of `from` in `to` still get their
own (non-reducer-shape) link scores via `emit_per_shape_link_scores`. The
agg-routed loop links (`X -> agg`, `agg -> Y` -- the original `(X, Y)` causal
edge never appears in an element-level loop once routed) drive the same
emission from the loop-iteration path. No `…⁚wildcard`/`…⁚dynamic` link-score
vars are emitted when the reference routes through an agg (the `RefShape`
suffix arms and `link_score_var_name`'s wildcard/dynamic cases survive as
code -- Phase 6 deletes them; a non-hoisted slice/dynamic reference with no
agg keeps the conservative `…⁚wildcard` emission for now).
Phase 5 (Task 5): a cross-element feedback loop that runs through a hoisted reducer visits the aggregate node more than once, so Johnson -- which enumerates only elementary circuits -- never emits it directly. Each agg-touching elementary circuit contributes one "petal" `agg → ... → agg`; `build_element_level_loops` now combines `k ≥ 2` petals of the same agg whose internal nodes are pairwise disjoint into the non-elementary cross-agg loops (capped at MAX_AGG_PETALS=8 petals → ≤ 219 recovered loops per agg, to keep the combinatorics bounded on wide arrays). The recovered loops' links and polarities come from `build_element_subscripted_links`, so their loop-score equations reference the per-element link scores along the un-trimmed path -- including the `pop[d]→agg` and `agg→share[e]` halves with `d ≠ e` (the cross-element coupling through the aggregate, exactly what the design's AC4.2 asks for). The polarity of an agg hop is currently `Unknown` (the variable-level polarity graph doesn't carry agg nodes), so a recovered loop classifies as Undetermined -- improving that means teaching polarity analysis about reducer monotonicity, which is out of scope (tech-debt #21). The user-facing reported loops (`model_detected_loops`, variable-level) never include synthetic agg nodes -- the aggregate is "trimmed" from the displayed loop in the sense that the only graph that surfaces it is the element-level one used internally to build the loop-score equations (and discovery's element-level graph, where the agg node stays visible -- AC4.7's test checks discovery finds a loop traversing it). A standalone cosmetic `[X→agg, agg→Y] → [X→Y]` collapse of the element-level `Loop.links` is not implemented (it would break the loop-score equations, which need the agg halves, unless `Loop` carried a separate untrimmed-links field); the "trim for reporting" contract is met by the variable-level report not showing agg nodes.
Fixes the five items from the Phase 5 review of the reducer-hoist / aggregate-node implementation: - Trim synthetic `$⁚ltm⁚agg⁚n` nodes out of *reported* loops in `discover_loops_with_graph` (AC4.2). A chain `[X→agg, agg→Y]` collapses to `[X→Y]` with composed polarity; only synthetic agg names are trimmed (variable-backed whole-RHS-scalar reducers are real nodes). The loop- score equation still references the un-trimmed agg-traversing path, so the trim is reporting-only. `LinkPolarity::compose` replaces the two ad-hoc sign-multiplication matches in `ltm::polarity`. The AC4.7 discovery test now asserts both that the cross-element loop is still found and that the reported FoundLoop's links are the trimmed form. - Fix the `agg → target` link score when a target hoists 2+ reducers (`x = SUM(a[*]) / SUM(b[*])`): the other aggs in the reducer- substituted equation must be in the dep set so they get PREVIOUS- wrapped, otherwise the numerator collapses to Δtarget and the magnitude pins to ±1. Also fixes the Pass-3 fragment dispatch: an `agg → scalar_target` link-score name has no bracket or shape suffix, so the legacy `(from, to)`-keyed salsa path used to claim it, `reconstruct_single_variable` the synthetic agg name, get `None`, and emit a degenerate equation against the target's original (reducer- bearing) equation -- collapsing the score to zero. Any agg-node link score now compiles from its already-substituted `ltm_var.equation`. - Dedupe agg link-score emission in the exhaustive loop-link path: when a target references a source both directly and via a hoisted reducer, the loop-link emitter visited the agg-routed loop links *and* re-emitted the agg halves through the direct edge's `emit_link_scores_for_edge`, pushing duplicate `LtmSyntheticVar`s into the `Vec`. The loop-link caller now passes `skip_agg_halves`. - Derive an aggregate node's `result_dims` from the reducer's result shape, not the owning variable's: `share[Region] = SUM(pop[*])` is a full reduce broadcast to `[Region]`, so `result_dims` is `[]`; only a partial slice-reduce that genuinely varies per element keeps the variable's dims. - `substitute_reducers_in_expr0` now recurses into subscript index expressions (`stock[SUM(idx[*])]`) -- such a reducer is hoisted by the agg walker, so the substituter must mirror that descent -- and the `slot.unwrap()` in the arrayed-target branch is restructured to thread the slot expression through directly. The Pass-3 LTM fragment dispatch in `compile_project_incremental` is also de-duplicated (one `compile_direct` closure for the four verbatim-compile cases) to keep `db.rs` under the per-file line cap.
…-score product)
The Phase 5 review flagged `test_discovery_finds_cross_element_through_agg_loop`
as a near-no-op: its `has_trimmed_agg_loop` assertion only checked that *some*
discovered loop contained a `pop[d] -> share[?]` link and an `update[?] -> pop[?]`
link, which the plain numerator-diagonal loop `pop[big] -> share[big] ->
update[big] -> pop[big]` already satisfies whether or not the `SUM(pop[*])`
reducer (hence the synthetic aggregate node) is present. The accompanying
comment ("a model without the agg-traversal would not produce this loop") was
therefore false: both models produce that loop; only its *score* differs.
Replace it with `test_discovery_loop_through_agg_scored_on_untrimmed_path`,
which compiles the heterogeneous-share fixture in discovery mode, simulates it,
runs `discover_loops_with_graph` directly, and then (1) keeps the strong AC4.2
trim assertion (no reported `FoundLoop.link` references `$:ltm:agg:0`), and
(2) reproduces the discovered `pop[big] -> share[big] -> update[big] ->
pop[big]` loop's `loop_score` series, at every step, as the product of the
four *un-trimmed* per-element link scores read straight from the result
matrix -- `pop[big]->agg * agg->share[big] * share->update[big] *
update->pop[big]` -- with a `saw_nonzero` guard so the equality is not vacuous
and so it pins that discovery routed through the aggregate node (the bare
numerator product is identically zero for this fixture). Update phase_05.md's
AC4.7 wording and done-when checklist to note that strongest-path discovery
reports one loop per stock node, so AC4.7's intent is "loop-finding is rerouted
through the agg node and scored on the un-trimmed path; the agg node is trimmed
from the reported links" -- not "the cross-element loop is the reported strongest
loop".
No production code changed.
Phase 5's aggregate-node reroute (each maximal inlined reducer is hoisted
into a synthetic $⁚ltm⁚agg⁚{n} auxiliary; pop[d]→agg→target is scored by
the chain rule) made the ad-hoc ⁚wildcard / ⁚dynamic per-shape link-score
machinery dead. This removes it:
- link_score_var_name no longer appends the ⁚wildcard / ⁚dynamic
suffixes; the LINK_SCORE_WILDCARD_SUFFIX / LINK_SCORE_DYNAMIC_SUFFIX /
LINK_SCORE_SHAPE_SUFFIXES constants are gone.
- ShapeRank shrinks to Bare / FixedIndex (still needed for the
Bare-beats-FixedIndex dedup tie-break in parse_link_offsets);
strip_to_shape_suffix_with_rank and its callsite are gone.
- resolve_link_score_name_for_loop loses its Wildcard / DynamicIndex
lookups; the assemble pass loses the LINK_SCORE_SHAPE_SUFFIXES check.
- shape_aware_source_ref's TODO about the aggregate denominator is
replaced with a note that the agg node now realizes it (the body is
unchanged -- it already special-cased only FixedIndex).
The RefShape::Wildcard / DynamicIndex enum variants survive: they remain
the reference-site walker's "route through an agg" markers, and the
conservative-slice reducer (SUM(pop[NYC,*]) inside a larger expression)
that enumerate_agg_nodes deliberately doesn't hoist still classifies as
Wildcard. emit_per_shape_link_scores now dedups by the resulting name so
such a slot collapses onto the canonical Bare link score rather than
minting a duplicate.
Dead tests pinned to the suffix machinery are removed; a positive AC5.1
guard (no ⁚wildcard / ⁚dynamic var emitted for a few reducer-bearing
fixtures) is added.
Updates the LTM docs to describe the aggregate-node treatment (each
maximal inlined reducer hoisted into a synthetic $⁚ltm⁚agg⁚{n} aux,
pop[d]→agg→target scored by the chain rule, the agg trimmed from reported
loops) and the element-level cross-element loop scoring landed by the
2026-05-09-ltm-503-cross-element-agg design plan:
- docs/design/ltm--loops-that-matter.md: adds $⁚ltm⁚agg⁚{n} and the
per-element link-score names to the naming table; rewrites the
element-graph truth table's Wildcard/DynamicIndex rows for the agg
reroute; adds an "Aggregate Nodes" subsection; updates the Link Score
Classification, Loop Scores, and Discovery sections; retires the
per-shape Wildcard link-score description.
- src/simlin-engine/CLAUDE.md (= AGENTS.md): adds ltm_agg.rs to the
module map; updates ltm_augment.rs / db_analysis.rs / db_ltm.rs /
ltm/types.rs bullets for enumerate_agg_nodes, the retired Wildcard
path, LtmSyntheticVar.equation: datamodel::Equation, element-
subscripted cross-element loops; adds a "Last updated" freshness date.
- docs/design-plans/2026-04-25-ltm-per-ref-elem-graph.md: adds a
re-measured post-Phases-1-5 measurement postscript subsection
(cross_element_ltm, arrayed_population_ltm, hero_culture_ltm, WRLD3-03,
plus share/mean reducer-with-feedback fixtures showing the N²→N+M
element-edge win).
- simulate_ltm.rs: updates the measurement_postscript_* rustdocs to
match the re-measured numbers (the loose asserts are unchanged).
- docs/tech-debt.md: forward-pointers + implementing commit hashes on
items #20 / #26 / #34 (the per-shape Wildcard link score was
superseded; cross-element assertions tightened).
- db_ltm.rs: a doc-comment on is_partial_reduce_edge noting the agg
reroute only covers scalar synthetic aggs (the whole-RHS arrayed-result
reducer is a variable-backed agg the predicate still handles).
Phase 6 retired the per-shape wildcard/dynamic link-score path, but in doing so it dropped the routing that sent a non-Bare-shaped *scalar* link score through direct (non-salsa) compilation. For a DynamicIndex/Wildcard reference into a scalar target -- e.g. `total = arr[idx]` inside a feedback loop -- the `(from, to)`-keyed salsa path (link_score_equation_text, hard-coded to RefShape::Bare) re-derives the partial with the whole subscript wrapped in PREVIOUS(), collapsing the ceteris-paribus numerator to ~0 and zeroing the link score (and any loop score that multiplies it). Untested, so the suite stayed green. Fix: add a `compile_directly` flag to LtmSyntheticVar; emit_per_shape_link_scores sets it for any non-Bare shape, and assemble_module routes those vars to the verbatim-equation compile path. Separately, the DynamicIndex/Wildcard partial that path produces was itself uncompilable: shape_aware_source_ref spelled the bare *arrayed* source in the scalar guard equation (`source - PREVIOUS(source)`), a dimension error -> no bytecode -> still zero. Capture the live source slice the partial isolates (`arr[PREVIOUS(idx)]`, `pop[NYC,*]`) during the PREVIOUS-wrapping transform and wrap it in SUM() for the guard's source side; SUM of a scalar is the identity and SUM of a slice is scalar, and the result only feeds the SIGN factor and the =0 guard, so substituting SUM for the reducer's own algebra is harmless. Applied to the aux->aux, stock->flow, and arrayed-per-element link-score generators (same latent bug in all three). Also retires a stale db_ltm_unified_tests section comment that still described the removed `⁚wildcard` naming convention.
…r_name The post-Phase-6 `link_score_var_name` bullet said only the not-hoisted conservative-slice reducer reaches that function as a Wildcard/DynamicIndex shape. The bare-dynamic-index case (`arr[idx]`, `arr[i+1]`) is a sibling that also reaches it -- `source_ref_for_guard` handles both -- so spell both out to match the code's own docstrings.
The doc comment claimed strip_element_subscript "mirrors" crate::ltm::strip_subscript, but it truncated at the first `[` (find) while strip_subscript truncates at the last `[` (rfind). For the single-comma-subscript element-node names LTM actually produces (e.g. "x[a,b]", "population[nyc]") and the bracket-free synthetic agg names, the two are equivalent; they would only diverge on a hypothetical nested-bracket name the codebase never generates. Switch to rfind so the "mirrors" claim is literally true and behavior matches strip_subscript's documented "last `[` is the truncation point", and note the nested-bracket semantics in the doc comment.
Human verification plan for the completed 6-phase GH #503 work (aggregate nodes, element-level cross-element loop scoring, discovery alignment, partial-reduce reducer machinery, the retired wildcard/dynamic link-score path). Backend engine change with no UI, so the plan is a manual verification procedure: build the share[r]=pop[r]/SUM(pop[*]) motivating case, simulate with --ltm, and inspect the synthetic LTM variables to confirm per-element chain-rule attribution; plus the WRLD3 / logistic-growth no-regression checks. Includes a traceability table mapping each acceptance criterion to its automated test and manual step.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3efb982044
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+1643
to
+1645
| let route_through_agg = !routed_aggs.is_empty() | ||
| && matches!(site.shape, RefShape::Wildcard | RefShape::DynamicIndex); | ||
| if route_through_agg { |
There was a problem hiding this comment.
Guarding reroute on !routed_aggs.is_empty() makes every Wildcard/DynamicIndex site from from_name to to_name go through synthetic aggs whenever the target equation contains at least one hoisted reducer, even when a given site is not inside that reducer. In mixed equations (for example one hoisted SUM(pop[*]) plus a non-hoisted slice/dynamic reference to pop), the non-hoisted reference loses its direct from->to edges and is incorrectly rewritten as from->agg->to, which changes the element graph and can drop or distort loop detection/scoring for those terms.
Useful? React with 👍 / 👎.
Review: LTM cross-element aggregate scoringReviewed the core engine changes ( [P2] Agg-node reroute also captures non-hoisted Wildcard/DynamicIndex references to the same source
let route_through_agg = !routed_aggs.is_empty()
&& matches!(site.shape, RefShape::Wildcard | RefShape::DynamicIndex);The comment just above (lines 1633-1636) states that a reference to Overall correctness: correct — the one issue above is an edge case; the patch otherwise appears sound and existing tests/behavior are preserved. |
ReviewI went through the LTM cross-element aggregate-scoring change in depth — I traced the major paths end-to-end:
A couple of accepted trade-offs are clearly documented in comments rather than being defects: cross-element loops routed through an agg report No blocking issues found. Overall correctness: correct |
added 4 commits
May 9, 2026 19:11
`70f34eae` ("doc: remove implementation plan") deleted the phase_NN.md
files but left `test-requirements.md`, which linked to phase_01.md and
phase_06.md -- so `scripts/check-docs.py` (and the pre-commit hook) failed
on a clean tree. Rephrase line 10 to refer to "phases 1 through 6 of the
now-removed implementation plan" instead of linking the deleted files.
The element-graph reroute (`model_element_causal_edges`) and the
link-score suppression (`emit_per_shape_link_scores(skip_reducer_shapes)`)
both keyed off the access *shape* (`RefShape::Wildcard | RefShape::DynamicIndex`)
to decide whether a reference to `from` in `to` should collapse into a
hoisted `$⁚ltm⁚agg⁚{n}` aggregate node. But a `to` equation that mentions
`from` both inside a hoisted reducer (`SUM(pop[*])`) *and* via a direct
dynamic index (`pop[idx]`) produces a `DynamicIndex` site for the *direct*
`pop[idx]` reference too -- so that direct reference was wrongly folded
into the agg (losing its conservative `pop[d] -> to` element edges) and its
`pop -> to` link score was dropped, leaving the dynamic-index dependency
unscored.
Add `ReferenceSite::in_reducer`, set true iff the site is syntactically
inside an array-reducing builtin argument (the same `SUM`/`MEAN`/`MIN`/
`MAX`/`STDDEV`/`RANK` recognition set `enumerate_agg_nodes` hoists; `SIZE`
and the 2-arg `MIN`/`MAX` are excluded). The reroute now keys on
`in_reducer`, and `skip_reducer_shapes` only suppresses the `Wildcard`
shape (the hoisted reducer's argument, already scored by the agg's two
halves) -- `DynamicIndex` keeps its conservative Bare-named link score.
`register_agg` deduped aggregate nodes purely by canonical reducer text,
regardless of `AggKind`. So when a whole-RHS reducer (`denom = SUM(pop[*])`,
a *variable-backed* agg with `is_synthetic = false`) was registered before
an inline use of the same text (`share[r] = pop[r] / SUM(pop[*])`, which
should mint a *synthetic* `$⁚ltm⁚agg⁚{n}`), the inline use reused the
variable-backed agg -- no synthetic was minted, so the downstream
`is_synthetic` filters in `model_element_causal_edges` and
`model_ltm_variables` skipped it, and whether the inline reducer got proper
aggregate-node routing depended on canonical variable name order.
Make the by-key dedup apply only to `Synthetic` aggs; `VariableBacked` aggs
are never deduped (two scalars can each be `SUM(pop[*])` and remain
distinct nodes). Rename `AggNodesResult.by_key` to `synthetic_by_key` and
narrow `agg_for_key` to resolve only synthetic aggs (variable-backed aggs
are found via `aggs_in_var`); `by_var` still indexes both kinds.
The doc comments on `emit_per_shape_link_scores` (db_ltm.rs) and the Phase-5 reroute block (db_analysis.rs) cited `SUM(pop[idx, *])` / `SUM(pop[NYC, *])` as the construct that produces a `DynamicIndex` reference site. That's wrong: `classify_subscript_shape` checks for a wildcard first and returns `Wildcard` for any index list containing one, regardless of a sibling dynamic index. The construct that actually yields a `DynamicIndex` site is a single non-literal index with no wildcard -- a direct `pop[idx]`, or `SUM(pop[idx])` inside a reducer (which isn't a full reduce, so `enumerate_agg_nodes` doesn't hoist it anyway). The claim each comment supports -- a `DynamicIndex` site reaching that code is from a non-hoisted construct, so its conservative link score must survive -- was already correct; only the worked example was mislabeled. Also add `ref_site_bare_arrayed_arg_is_in_reducer` to the `collect_reference_sites_tests` module: a bare arrayed argument `SUM(pop)` is a full reduce that gets hoisted, and its AST reference is a bare `Var`, so the site must come back as `RefShape::Bare` with `in_reducer == true` -- the existing tests covered `SUM(pop[*])` (Wildcard) but not this bare-arg form.
Review: PR #519 — LTM cross-element aggregate scoringI reviewed the new One minor observation: [P3] Distinct loops that trim to the same edge chain are reported as separate loops with identical links
Overall correctness verdict: correctNo blocking issues. Existing behavior and tests are preserved; the patch is free of bugs that would break the build or simulation results, modulo the cosmetic P3 above. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #519 +/- ##
==========================================
+ Coverage 82.38% 82.45% +0.06%
==========================================
Files 241 242 +1
Lines 61200 63290 +2090
==========================================
+ Hits 50419 52185 +1766
- Misses 10781 11105 +324 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Two LTM (Loops That Matter) follow-ups surfaced while reviewing the cross-element aggregate-scoring work. GH #516: a feedback loop that routes through a synthetic $⁚ltm⁚agg⁚{n} node always classified as Undetermined. The loop builders derive every link's polarity from the variable-level causal graph, but agg nodes exist only in the element graph, so analyze_link_polarity found no reference to the agg's name in either endpoint and the hop came back Unknown -- which forces the whole loop to Undetermined. The polarity is derivable for the common cases: source[d] -> agg is Positive whenever the reducer is monotone non-decreasing in each element (SUM/MEAN/MIN/MAX), and agg -> consumer is the polarity of the consumer's equation with respect to the reducer subexpression, computed by substituting the reducer with the agg name and running the ordinary static polarity analysis (CausalGraph::agg_consumer_polarity). model_ltm_variables now patches these hops on the detected loops and re-derives each affected loop's polarity (and re-assigns IDs, whose r/b/u prefix is polarity-derived). STDDEV/RANK aggs are not monotone and stay Unknown. GH #517: the ceteris-paribus partial for a Bare reference whose target also contains an inlined reducer (share[r] = pop[r] / SUM(pop[*]) -> the pop -> share link score) wrapped the array view *inside* the reducer in PREVIOUS -- pop[r] / SUM(PREVIOUS(pop[*])) -- which is silently 0.0 at every step under an active apply-to-all dimension because codegen has no LoadPrev-of-array-view path. wrap_non_matching_in_previous now wraps the whole reducer App -- pop[r] / PREVIOUS(SUM(pop[*])), which is PREVIOUS of a scalar and evaluates correctly -- unless the reducer itself carries the live reference (the test-only Wildcard live_shape). This is the correct partial (Delta pop[r] / D(t-1) over Delta share[r]); the general codegen fix for LoadPrev-of-array-view stays open for the not-yet-hoisted slice cases. With the numerator path's link score now real, it correctly competes with the aggregate path in discovery's strongest-path heuristic, so the test_discovery_loop_through_agg_scored_on_untrimmed_path assertion is relaxed to accept whichever path discovery surfaces. References GH #516 and GH #517.
ReviewI reviewed the cross-element aggregate-node changes ( One minor doc-comment note (P3): [P3]
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements GH #503 (which was closed manually as part of this work — the repo forbids auto-close keywords). Design:
docs/design-plans/2026-05-09-ltm-503-cross-element-agg.md. Updated user-facing behavior doc:docs/design/ltm--loops-that-matter.md("Aggregate Nodes" section). Manual verification plan:docs/test-plans/2026-05-09-ltm-503-cross-element-agg.md.Fixes: #516
Fixes: #517
Summary
LTM ("Loops That Matter") scored cross-element feedback loops from the diagonal apply-to-all link scores instead of the actual per-element path, and lumped inlined array reducers into a single "Wildcard" link score that omitted each source element's fractional contribution to the aggregate's velocity. This branch fixes that:
SUM(pop[*]),MEAN(...),SUM(m[D1,*]), ...) is hoisted into a synthetic auxiliary$⁚ltm⁚agg⁚{n}(enumerate_agg_nodes, newsrc/simlin-engine/src/ltm_agg.rs). Causality routessource[d] → agg → target(O(N+M)) instead of an all-pairssource[d] → target[e]cross-product; both halves get real per-element link scores that compose by the chain rule; the agg node is trimmed from a loop when the loop is reported (like a macro's hidden internal nodes). A variable whose entire RHS is one reducer maps to itself — no synthetic minted. Model equations are not rewritten. This resolves the diagonal conflation (share[r] = pop[r]/SUM(pop[*]): the numerator path and the SUM path are now distinct loops, scored separately) and fixes heterogeneous-magnitude scoring (the per-element|Δpop[d]/Δagg|factor is present and non-constant acrossd).Ast::Arrayed(per-element-equation) link-score targets now carry meaningful per-element partial equations derived from each element's own equation, instead of a"0"placeholder.LtmSyntheticVar.equationchanged fromStringtodatamodel::Equation."$⁚ltm⁚link_score⁚{from}→{to}"[e]), in both the exhaustive and discovery modes.$⁚ltm⁚link_score⁚{from}→{to}[{elem}]), so discovery's per-timestep search graph traverses the correct element graph (previously it invented ascalar_source[elem]phantom node and the loop was unreachable).agg[D1] = SUM(matrix[D1,*])) are supported by the reducer link-score machinery — per-(reduced-elem, result-elem)scalar link scores.⁚wildcard/⁚dynamicper-shape link-score path is retired (link_score_var_namesuffix arms,strip_to_shape_suffix_with_rank,ShapeRank::Wildcard/::DynamicIndex,shape_aware_source_ref's TODO). TheRefShape::Wildcard/DynamicIndexenum variants survive as the reference-site walker's "route through an agg node" markers. Also fixed a latent identical-zero bug whereshape_aware_source_refspelled a bare arrayed source in a scalar guard equation (a dimension error → uncompilable fragment).$⁚ltm⁚agg⁚{n}node used to classifyUndetermined: the loop builders derive every link's polarity from the variable-level causal graph, but agg nodes exist only in the element graph, so the hop came backUnknownand forced the whole loop toUndetermined.model_ltm_variablesnow patches these hops on the detected loops —source[d] → aggisPositivefor a monotone reducer (SUM/MEAN/MIN/MAX);agg → consumeris the polarity of the consumer's equation w.r.t. the reducer subexpression, computed by substituting the reducer with the agg name and running the ordinary static polarity analysis (CausalGraph::agg_consumer_polarity) — and re-derives each affected loop's polarity (and re-assigns IDs, whoser/b/uprefix is polarity-derived). STDDEV/RANK aggs stayUnknown(not monotone).share[r] = pop[r] / SUM(pop[*])→ thepop → sharelink score) wrapped the array view inside the reducer inPREVIOUS—pop[r] / SUM(PREVIOUS(pop[*]))— which is silently0.0at every step under an active apply-to-all dimension because codegen has no LoadPrev-of-array-view path.wrap_non_matching_in_previousnow wraps the whole reducer App —pop[r] / PREVIOUS(SUM(pop[*])), which isPREVIOUSof a scalar and evaluates correctly — unless the reducer itself carries the live reference. This is the correct ceteris-paribus partial (Δpop[r] / D(t-1) over Δshare[r]); it also makes the numerator-path loop's link score real, so it correctly competes with the aggregate path in discovery's strongest-path heuristic. (The general codegen fix for LoadPrev-of-array-view stays open for the not-yet-hoisted slice cases.)simulates_population_ltm(vstest/logistic_growth_ltm/ltm_results.tsv), the WRLD3 LTM smoke, and the arrayed-population LTM tests pass with unchanged expected values; no golden TSV was modified.Implemented in 6 phases (each individually code-reviewed) plus the two follow-up fixes above; 37 commits. All 22 acceptance criteria (
ltm-503-cross-element-agg.AC1.1..AC7.2) are covered by passing unit/integration tests or completed verifications; the GH #516 / #517 fixes add their own tests (db_ltm_unified_tests::cross_element_loop_through_agg_is_recoveredpolarity assertion;ltm_augment::test_partial_equation_*for the whole-reducerPREVIOUSwrapping).cargo test --workspaceis green in ~20s, well under the 3-minute cap; the pre-commit hook passed on every commit.Remaining pre-existing limitations discovered along the way are filed (not fixed — out of scope): #510, #511, #514, #515 (LTM follow-ups, referenced from epic #488 and/or
docs/tech-debt.md), #518 (db.rsis at the 6000-line lint cap and could use a split), and #520 (unify the LTM element-graph and link-score reference-site walkers behind one shared classification IR — the current "they agree" invariant is enforced by convention + tests rather than shared code). Epic #488's#503checklist box is ticked; #516 and #517 are addressed here.Test plan
cargo test --workspacegreen (also the pre-commit hook end-to-end).cargo test -p simlin-engine --features file_io --test simulate_ltmgreen.cargo clippy --workspace --all-targets -- -D warningsclean;cargo fmt -- --checkclean.docs/test-plans/2026-05-09-ltm-503-cross-element-agg.md— build theshare[r] = pop[r]/SUM(pop[*])model with heterogeneous inits,simulate --ltm, and confirm the$⁚ltm⁚agg⁚0aux, the per-source-elementpop[d]→agglink scores (whose|Δpop[d]/Δagg|magnitudes differ markedly between the large and small region), theagg→share[e]link scores, the Bare numeratorpop→sharelink score (nowpop[r] / PREVIOUS(SUM(pop[*]))— non-zero), no⁚wildcard/⁚dynamiccolumns, and the loop scores = product over the un-trimmed agg-traversing path.model_detected_loopssurfaces no$⁚ltm⁚agg⁚{n}node in any reported loop (agg trimmed from reporting), and that a cross-element-through-agg loop classifiesR/B(notU) when the reducer is SUM/MEAN/MIN/MAX.