Skip to content

ltm: reducer beside a dynamic-index co-source is un-hoisted and every row's link score is the silent delta-ratio stand-in of +-1 (no diagnostic) #1042

Description

@bpowers

Summary

A reducer whose argument list holds a dynamic-index read beside a statically describable [Iterated, Reduced] read -- y[d1] = SUM(m[d1, *] * n[d1, idx]) * 0.01 with idx a time-varying aux -- is not hoisted into an aggregate node (the dynamic index is not statically describable, the documented carve-out). Its edges then fall to the legacy cartesian derivation in try_cross_dimensional_link_scores, where the row partial cannot be built and degrades to the delta-ratio stand-in: the partial is the target itself, so every emitted per-row link score is |dy/dy| = +-1 -- for the describable m rows, for the dynamic n rows, and for n rows the equation does not read at that step. No diagnostic is emitted. The loops that survive (fm -> m -> y, fn -> n -> y) consume these scores and report loop scores of exactly +-1, including a wrong-signed -1 at the step where idx switches.

This violates epic #488's standing invariant ("No silent wrong numbers. An un-scoreable shape degrades to one loud warning plus dropped loop scores -- never garbage values"). docs/design/ltm--loops-that-matter.md, "Residual array carve-outs", records the behavior verbatim ("the link score keeps the delta-ratio stand-in with no diagnostic") and calls the carve-out "tracked tech debt", but nothing tracks the link-score half: docs/tech-debt.md entry 20 covers only the element graph's conservative cross-product (a sound superset), and no open GH issue names the shape.

Pre-existing: the V9b adversarial review of PR #1040 ran its base (615526fe) and tree binaries on the three probe models below and got byte-identical output (the stand-in of 1 on both); re-run today on compiler-unification-v2 at 78d0195a with the same result. The code path (NotDescribable -> cartesian arm -> delta-ratio) has been in place since the GH #514 carve-out list.

Reproduction (probe p13a_dynamic_index_reducer.xmile; sim 0..4, dt 1, Euler)

d1 = {a, b}, d2 = {c, d}

m[d1, d2]   stock, init {a,c: 1, a,d: 2, b,c: 3, b,d: 5},     inflow fm
n[d1, d2]   stock, init {a,c: 7, a,d: 11, b,c: 13, b,d: 17},  inflow fn
idx         = IF TIME > 2 THEN 1 ELSE 2
y[d1]       = SUM(m[d1, *] * n[d1, idx]) * 0.01
fm[d1, d2]  = y[d1] * 0.1
fn[d1, d2]  = y[d1] * 0.2

Executed read: y[a] = 0.01 * n[a, idx] * (m[a,c] + m[a,d]) -- two m rows and ONE n row (n[a,d] while idx = 2, n[a,c] once idx = 1) feed each y slot.

Full XMILE:

<?xml version="1.0" encoding="utf-8"?>
<xmile version="1.0" xmlns="http://docs.oasis-open.org/xmile/ns/XMILE/v1.0" xmlns:simlin="https://simlin.com/XMILE/v1.0"><header><name>probe</name><vendor>v9b-review</vendor><product version="1.0">v9b-review</product><options namespace="std"></options></header><sim_specs method="Euler" time_units="Month"><start>0</start><stop>4</stop><dt>1</dt></sim_specs><dimensions><dim name="d1"><elem name="a"/><elem name="b"/></dim><dim name="d2"><elem name="c"/><elem name="d"/></dim></dimensions><model name="main"><variables><stock name="m"><element subscript="a,c"><eqn>1</eqn></element><element subscript="a,d"><eqn>2</eqn></element><element subscript="b,c"><eqn>3</eqn></element><element subscript="b,d"><eqn>5</eqn></element><inflow>fm</inflow><dimensions><dim name="d1"/><dim name="d2"/></dimensions></stock><stock name="n"><element subscript="a,c"><eqn>7</eqn></element><element subscript="a,d"><eqn>11</eqn></element><element subscript="b,c"><eqn>13</eqn></element><element subscript="b,d"><eqn>17</eqn></element><inflow>fn</inflow><dimensions><dim name="d1"/><dim name="d2"/></dimensions></stock><aux name="idx"><eqn>IF TIME > 2 THEN 1 ELSE 2</eqn></aux><aux name="y"><eqn>SUM(m[d1, *] * n[d1, idx]) * 0.01</eqn><dimensions><dim name="d1"/></dimensions></aux><flow name="fm"><eqn>y[d1] * 0.1</eqn><dimensions><dim name="d1"/><dim name="d2"/></dimensions></flow><flow name="fn"><eqn>y[d1] * 0.2</eqn><dimensions><dim name="d1"/><dim name="d2"/></dimensions></flow></variables></model></xmile>

Run: simlin simulate --ltm p13a_dynamic_index_reducer.xmile.

Observed

Emitted link scores (every m[*,*] -> y[*] and n[*,*] -> y[*] row is identical):

t m[a,c]->y[a] m[a,d]->y[a] n[a,c]->y[a] n[a,d]->y[a] loop_score r13 (fm->m->y) loop_score r18 (fn->n->y)
1 1 1 1 1 0 0
2 1 1 1 1 1.000 1.000
3 -1 -1 -1 -1 -1.000 -1.000
4 1 1 1 1 1.000 1.000

stderr carries ONLY the 16 loop-drop warnings for the cross-element circuits (r1..r12, r14..r17) whose off-diagonal hops (m[a,c] -> y[b]) the cartesian arm never emits -- those are the element graph's conservative DynamicIndex cross-product being dropped loudly, which is correct. Nothing names m -> y or n -> y; the surviving loops r13/r18 (the A2A pure-dimension loops, statically reinforcing) are scored from the stand-in.

Exact ceteris-paribus per-row shares computed from the recorded series (other sources, including idx, held at t-1), against the emitted value:

t dy[a] m[a,c] m[a,d] n[a,c] n[a,d] emitted (all four)
1 +0.00928 0.391 0.391 0.000 (unread, idx=2) 0.213 1
2 +0.00964 0.390 0.390 0.000 (unread) 0.216 1
3 -0.11814 +0.033 +0.033 0.000 (unread) 0.019 -1
4 +0.00483 0.345 0.345 0.306 0.000 (unread, idx=1) 1

Three distinct wrongs, none warned: (a) magnitude -- every row claims the whole of dy, so the shares of one slot sum to 4 instead of ~1; (b) unread rows -- n[a,c] at t<=3 and n[a,d] at t=4 are not read by y[a] at all yet score 1; (c) sign -- at t=3 dy is negative because idx switched rows (the idx -> y link's doing), and the stand-in reports every m and n row as negative, so the reinforcing loops r13/r18 (r prefix in the enumeration) read as balancing at that step.

Controls

  • p13d: same model with y[d1] = SUM(m[d1, *] * n[d1, idx]) (reducer is the whole RHS): stand-in 1 on every row at every step -- so it is the un-pinnable body, not the surrounding * 0.01, that dooms the partial.
  • p13e: same model with n[d1, *] in place of n[d1, idx]: hoisted as $⁚ltm⁚agg⁚0[d1], per-row scores 0.290 / 0.456 / 0.083 / 0.166 (sum ~1), 8 loops all scored, zero warnings. The dynamic index is the sole trigger.
  • p13b (SUM(m[d1, *] * SUM(n[d1, idx])) * 0.001, nested) and p13c (SUM(m * n[d1, idx]) * 0.01, bare m beside the dynamic index): identical stand-in of +-1 on every row.

Mechanism

  1. ltm_agg.rs::compute_read_slice -> classify_axis_access: the index idx is a non-literal Expr, so the read of n is None (not statically describable) and no aggregate node is minted for the reducer (the documented carve-out; db/ltm_ir.rs reclassifies the site DynamicIndex, the element graph takes the conservative cross-product).
  2. db/ltm/link_scores.rs::try_cross_dimensional_link_scores: no variable-backed or synthetic agg backs m -> y or n -> y, so both reach the legacy cartesian partial-reduce branch. decline_unhoisted_reducer_edge -> ltm_agg::unhoisted_reducer_source_read -> classify_expr_source_read returns FullExtent for m (its own read [Iterated(d1), Reduced] covers every row) and NotDescribable for n; neither verdict declines (the NotDescribable rustdoc: "The conservative cartesian cross-product is the DOCUMENTED behavior for the scalar/A2A dynamic-index family, so the caller keeps it (no decline)"). The verdict for m looks only at m's OWN slice; it does not see that a sibling read in the same reducer body is un-pinnable.
  3. The cartesian arm calls ltm_augment.rs::generate_element_to_reduced_equation -> build_element_reducer_link_score with a ReducerBodyCtx. generate_linear_body_partial -> pin_body_to_row returns None on n[d1, idx] ("an index that doesn't correspond to the row's axes"), and the ReducerKind::Linear arm does .unwrap_or_else(|| target_ref.to_string()) -- the partial becomes y[a] itself. (With surrounding arithmetic the _ if !is_bare arm returns target_ref even earlier.) The enclosing formula then reads |(y - PREVIOUS(y)) / (y - PREVIOUS(y))| * sign(...) = +-1.
  4. No warning is accumulated anywhere on this chain; the only loud surface is the loop builder's missing-name drop for circuits routed through off-diagonal hops.

Why it matters

Severity: medium. It is exactly the class the epic's invariant forbids: a plausible-looking, confidently wrong number with no diagnostic, on a shape a modeler writes without trying (a per-row product with a switch-selected column). The dominance surface is corrupted twice over -- rel-loop scores split the stand-in evenly across the surviving loops regardless of the true shares, and the wrong sign at the switch step flips the reported runtime polarity of a structurally reinforcing loop. Every other un-hoisted family (StrictSlice GH #791, PerElementReducerRead GH #792, the GH #780 doom, the GH #778/#785 duplicated dim) has been moved onto the loud-skip contract; the dynamic-index family is the one still documented as "no diagnostic".

Expected behaviour

Per the invariant, one of:

  1. Loud decline (the minimum). m -> y and n -> y recorded in unscoreable_edges with one warning naming the dynamic index, no link-score variable, r13/r18 dropped like the other sixteen -- the GH ltm: declined element-mapped sliced reducer emits a non-compiling scalar link score on the conservative path (loop scores stub to 0) #758/ltm: PartialEquationError link-score skips don't drop dependent loop scores via unscoreable_edges #780 contract.
  2. Exact description (better). The direct-reference PerElement path already lowers a frozen dynamic-index read as PREVIOUS(n[region·r, PREVIOUS(idx)]) (GH ltm: first-live-step NaN in dynamic-index per-element link scores (uninitialized PREVIOUS(index) capture helper) #975). The same form lets pin_body_to_row pin m to its row and keep the dynamic sibling frozen, so m[a,c] -> y[a] is 0.01 * (m[a,c] * PREVIOUS(n[a, PREVIOUS(idx)]) + PREVIOUS(m[a,d]) * PREVIOUS(n[a, PREVIOUS(idx)])) -- the 0.391 in the table. The dynamic source's own edge (n -> y) reads a row selected at runtime; it can be described per row with the selection live (IF PREVIOUS(idx) = 1 THEN ... ELSE PREVIOUS(y[a]), giving 0 for the unread row) or declined loudly on its own while its describable siblings are scored exactly.

Either way the silent degradation point must go: build_element_reducer_link_score's unwrap_or_else(|| target_ref.to_string()) and its !is_bare arm are the one place several un-hoisted shapes fall to the delta-ratio unwarned. Returning None there and letting try_cross_dimensional_link_scores take the unscoreable_edges skip closes the whole class; the GH #744 self-reference family (SUM(pop[*] * pop[north]), "deliberately conservative delta-ratio") lands on the same point and needs an explicit decision (keep with a pin that names it, or decline) rather than riding along silently. RANK's delta-ratio is separately documented and pinned (test_generate_rank_keeps_delta_ratio) and is not this issue.

Components affected

  • src/simlin-engine/src/ltm_agg.rs -- compute_read_slice / classify_axis_access (the hoist decline), unhoisted_reducer_source_read / classify_expr_source_read (FullExtent for a describable read beside an un-pinnable sibling; NotDescribable explicitly keeps the cartesian), UnhoistedSourceRead::NotDescribable rustdoc.
  • src/simlin-engine/src/db/ltm/link_scores.rs -- try_cross_dimensional_link_scores cartesian partial-reduce branch, decline_unhoisted_reducer_edge.
  • src/simlin-engine/src/ltm_augment.rs -- build_element_reducer_link_score (the unwrap_or_else(|| target_ref.to_string()) and !is_bare delta-ratio arms), generate_linear_body_partial, pin_body_to_row.
  • docs/design/ltm--loops-that-matter.md -- "Residual array carve-outs" (rewrite the dynamic-index bullet once the behaviour is loud or exact) and the "carve-outs (tracked tech debt ...)" paragraph; docs/tech-debt.md entry 20's 2026-05-13 note.

Pin

A TestProject/ltm_array_agg fixture with the p13a equations asserting, per row and per step, either the exact shares above against the recorded series or the loud decline (warning text names idx; m -> y and n -> y in unscoreable_edges; no $⁚ltm⁚link_score⁚m[a,c]→y[a] variable; r13/r18 absent) -- plus the p13e control asserting the hoisted exact scores are unchanged.

Related

Discovery context

Discovery 5 of the V9b (Loops That Matter, chunk B) adversarial review of PR #1040 (branch compiler-unification-v2; design plan docs/design-plans/2026-08-25-compiler-unification.md, "Phase LTM semantic divergences (V9b)"). Probes p13a_dynamic_index_reducer.xmile, p13b_dynamic_index_nested_reducer.xmile, p13c_dynamic_index_bare_feeder.xmile (natural XMILE models) gave the stand-in of 1 per row on the base and the tree alike; the p13a equations and full XMILE are reproduced above. Not a regression of that PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions