You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ltm: PREVIOUS-captured array-valued non-reducing builtin (RANK) scalarizes into ill-typed stubbed helpers, corrupting link scores #742
When LTM scores a link whose target equation contains an array-valued, NON-reducing builtin subtree (RANK demonstrated; VECTOR SORT ORDER / VECTOR ELM MAP / ALLOCATE are plausibly the same class), the ceteris-paribus partial wraps that subtree atomically in PREVIOUS(...), builtins_visitor::make_temp_arg captures the complex PREVIOUS argument into per-element scalar helper auxes, the helper equation is ill-typed in scalar context, the helper fragments fail to compile and are silently stubbed to constant 0, and the consuming link score is corrupted -- with no diagnostic anywhere.
Demonstrated repro
With arrayed pop[region] and a scored feedback-loop edge scale -> grow:
grow[region] = scale[region] * RANK(pop, 1)
The ceteris-paribus partial for the scale -> grow link embeds PREVIOUS(rank(pop, 1)) (the RANK subtree contains no live scale reference, so wrap_non_matching_in_previous wraps it atomically).
builtins_visitor::make_temp_arg captures the complex PREVIOUS arg into per-element scalar helper auxes (...⁚arg0⁚north / ...⁚arg0⁚south) whose equation is rank(pop, 1).
That equation is ill-typed in a scalar helper: RANK is array-valued and non-reducing, Pass 1 of the builtins visitor never decomposes RANK args, and the dimension-bounds machinery cannot rescue it (verified identical behavior at multiple commits).
The scale -> grow link score reads a corrupted -1000 instead of the true value. No Warning fires.
Root cause
Two interacting pieces:
src/simlin-engine/src/ltm_augment.rs -- is_array_reducer_name (~line 299) is a thin reader of ltm_agg::reducer_kind_from_name, which includes RANK. The wrap_non_matching_in_previous call site (~lines 639-652) uses it to decide "this subtree reduces to a scalar, so wrapping it atomically in PREVIOUS(...) is safe." That is true for SUM/MEAN/STDDEV/SIZE/MIN/MAX, but RANK does not reduce -- RANK(arr, n) is array-valued -- so the resulting PREVIOUS(<array-valued expr>) has no scalar meaning.
src/simlin-engine/src/builtins_visitor.rs -- make_temp_arg (~line 694) synthesizes the captured PREVIOUS argument as scalar helper auxes. For an array-valued non-reducing builtin subtree there is no per-element scalar projection it can emit (unlike the bare-arrayed-name case engine: bare arrayed name inside nested PREVIOUS() in an apply-to-all equation fails to compile #541 fixed via arrayed helper synthesis), so the helper lands ill-typed.
The failure then disappears into the model_ltm_implicit_var_info silent-stub path tracked separately as #741.
Relationship to existing issues
engine: bare arrayed name inside nested PREVIOUS() in an apply-to-all equation fails to compile #541 (closed): the same class -- scalar helper capture of array-shaped PREVIOUS args -- for a bare arrayed name. Its fix (arrayed helper synthesis / subscripting the bare reference) does NOT cover an array-valued builtin application: there is no name to subscript, and Pass 1 never decomposes RANK args. Verified byte-identical broken behavior at commits bd4376e and 9187da4.
docs/tech-debt.md entry 27 covers RANK's delta-ratio fallback when RANK is the scored reducer itself -- a different path; this issue is about RANK appearing as a frozen (non-live) subtree inside someone else's scored partial.
Why it matters
Silent wrong link scores (and therefore loop scores / dominance attribution) for any model where an array-valued non-reducing builtin appears inside a scored equation on a loop edge. Severity low-medium: the shape is uncommon (no current corpus model hits it), but when hit the corruption is invisible to the user.
Components affected
src/simlin-engine/src/ltm_augment.rs -- is_array_reducer_name (~299) and its wrap_non_matching_in_previous call site (~639-652): the atomic-PREVIOUS wrap is only sound for genuinely reducing builtins
src/simlin-engine/src/builtins_visitor.rs -- make_temp_arg (~694): scalar capture of array-shaped PREVIOUS args
src/simlin-engine/src/db/ltm/compile.rs -- the fragment compile path where the helpers fail
Possible approaches
In ltm_augment.rs, distinguish reducing from array-valued builtins at the wrap site: exclude RANK (and audit VECTOR SORT ORDER / VECTOR ELM MAP / ALLOCATE) from the atomic-PREVIOUS shortcut, instead recursing into the args and PREVIOUS-wrapping the scalar-shaped leaves (subscripting array refs by the active A2A dimension first, mirroring the engine: bare arrayed name inside nested PREVIOUS() in an apply-to-all equation fails to compile #541 generator-side workaround), so every captured helper is genuinely scalar.
Identified during adversarial review on branch ltm-core-batch; pre-existing on main, NOT a regression of that branch's work (verified byte-identical at bd4376e and 9187da4). Part of LTM tracking epic #488.
Summary
When LTM scores a link whose target equation contains an array-valued, NON-reducing builtin subtree (RANK demonstrated; VECTOR SORT ORDER / VECTOR ELM MAP / ALLOCATE are plausibly the same class), the ceteris-paribus partial wraps that subtree atomically in
PREVIOUS(...),builtins_visitor::make_temp_argcaptures the complexPREVIOUSargument into per-element scalar helper auxes, the helper equation is ill-typed in scalar context, the helper fragments fail to compile and are silently stubbed to constant 0, and the consuming link score is corrupted -- with no diagnostic anywhere.Demonstrated repro
With arrayed
pop[region]and a scored feedback-loop edgescale -> grow:scale -> growlink embedsPREVIOUS(rank(pop, 1))(the RANK subtree contains no livescalereference, sowrap_non_matching_in_previouswraps it atomically).builtins_visitor::make_temp_argcaptures the complexPREVIOUSarg into per-element scalar helper auxes (...⁚arg0⁚north/...⁚arg0⁚south) whose equation isrank(pop, 1).assemble_modulesilently drops them (the silent drop itself is ltm: failed implicit-helper fragment compile emits no diagnostic (model_ltm_fragment_diagnostics skips model_ltm_implicit_var_info) #741); the helpers read constant 0 at runtime.scale -> growlink score reads a corrupted -1000 instead of the true value. NoWarningfires.Root cause
Two interacting pieces:
src/simlin-engine/src/ltm_augment.rs--is_array_reducer_name(~line 299) is a thin reader ofltm_agg::reducer_kind_from_name, which includes RANK. Thewrap_non_matching_in_previouscall site (~lines 639-652) uses it to decide "this subtree reduces to a scalar, so wrapping it atomically inPREVIOUS(...)is safe." That is true for SUM/MEAN/STDDEV/SIZE/MIN/MAX, but RANK does not reduce --RANK(arr, n)is array-valued -- so the resultingPREVIOUS(<array-valued expr>)has no scalar meaning.src/simlin-engine/src/builtins_visitor.rs--make_temp_arg(~line 694) synthesizes the capturedPREVIOUSargument as scalar helper auxes. For an array-valued non-reducing builtin subtree there is no per-element scalar projection it can emit (unlike the bare-arrayed-name case engine: bare arrayed name inside nested PREVIOUS() in an apply-to-all equation fails to compile #541 fixed via arrayed helper synthesis), so the helper lands ill-typed.The failure then disappears into the
model_ltm_implicit_var_infosilent-stub path tracked separately as #741.Relationship to existing issues
PREVIOUSargs -- for a bare arrayed name. Its fix (arrayed helper synthesis / subscripting the bare reference) does NOT cover an array-valued builtin application: there is no name to subscript, and Pass 1 never decomposes RANK args. Verified byte-identical broken behavior at commits bd4376e and 9187da4.wrap_non_matching_in_previous-> ill-typed-PREVIOUS-arg family, but for partially-iterated subscripts (a hard compile failure, not a silent stub).Why it matters
Silent wrong link scores (and therefore loop scores / dominance attribution) for any model where an array-valued non-reducing builtin appears inside a scored equation on a loop edge. Severity low-medium: the shape is uncommon (no current corpus model hits it), but when hit the corruption is invisible to the user.
Components affected
src/simlin-engine/src/ltm_augment.rs--is_array_reducer_name(~299) and itswrap_non_matching_in_previouscall site (~639-652): the atomic-PREVIOUSwrap is only sound for genuinely reducing builtinssrc/simlin-engine/src/builtins_visitor.rs--make_temp_arg(~694): scalar capture of array-shapedPREVIOUSargssrc/simlin-engine/src/db/ltm/compile.rs-- the fragment compile path where the helpers failPossible approaches
ltm_augment.rs, distinguish reducing from array-valued builtins at the wrap site: exclude RANK (and audit VECTOR SORT ORDER / VECTOR ELM MAP / ALLOCATE) from the atomic-PREVIOUSshortcut, instead recursing into the args andPREVIOUS-wrapping the scalar-shaped leaves (subscripting array refs by the active A2A dimension first, mirroring the engine: bare arrayed name inside nested PREVIOUS() in an apply-to-all equation fails to compile #541 generator-side workaround), so every captured helper is genuinely scalar.make_temp_argto synthesize an arrayed helper aux when the captured expression is array-shaped (the generalization of engine: bare arrayed name inside nested PREVIOUS() in an apply-to-all equation fails to compile #541's fix from bare names to arbitrary array-shaped expressions), then subscript the helper reference at the capture site.Warningrather than nothing (and consider refusing to emit the partial with an unscoreable-edge warning instead of a corrupt score).Discovery context
Identified during adversarial review on branch
ltm-core-batch; pre-existing on main, NOT a regression of that branch's work (verified byte-identical at bd4376e and 9187da4). Part of LTM tracking epic #488.