engine: graphical-function tables with unsorted/NaN x-coordinates are accepted silently and produce undefined lookup results #715

Summary

No layer between a source file and the VM validates that a graphical function's (GF / lookup-table) x-coordinates are non-decreasing (and finite). A GF whose x points are out of order -- e.g. an XMILE <xpts>0,10,5,20</xpts> -- is accepted verbatim all the way through to simulation, where the VM's binary-search lookup assumes sorted x. On an unsorted table the binary search returns an arbitrary (though mutually self-consistent) interpolated value with no diagnostic. NaN x-coordinates are similarly unguarded at the import layers.

This is a pre-existing input-validation gap, distinct from every currently-tracked GF issue (see "Relationship to existing issues" below). It was discovered during adversarial review of the GF lookup hot-path perf optimization (#602): rather than fix the validation gap inline, the perf change gates its optimization on a per-table sortedness check at Vm::new -- so the optimization is correct, but the underlying "unsorted table reaches the VM at all" gap remains.

The problem

Every layer that constructs a GF's x-vector passes the x points through verbatim, with no check that x[i] <= x[i+1] for all i (and no finiteness check):

• XMILE <xpts> / points import: src/simlin-engine/src/xmile/dimensions.rs (~line 364, where x_points is read from gf.x_pts).
• protobuf / DB load: src/simlin-engine/src/serde.rs.
• parse_table: src/simlin-engine/src/variable.rs:295 -- the single choke point all the import paths ultimately funnel a GF through; it builds Table { x, y, .. } by cloning gf.x_points (or synthesizing them from x_scale) with no ordering check.
• Table::new: src/simlin-engine/src/compiler/expr.rs:20.
• Compiler::new: src/simlin-engine/src/compiler/codegen.rs:55.

The consumer that assumes sorted x:

• VM lookup: src/simlin-engine/src/vm.rs:3181 -- binary search (while low < high { ... if table[mid].0 < index ... }). It guards empty tables, NaN index, and below-first / above-last index, but assumes the table's x-axis is non-decreasing. On an unsorted table the binary search lands on an arbitrary segment and lookup_tail interpolates against it -- a silently wrong result, not a crash.
• wasmgen lookup helpers: src/simlin-engine/src/wasmgen/lookup.rs (lookup_interp / lookup_forward / lookup_backward) reproduce the same sorted-x assumption.

A user-authored XMILE GF with <xpts>0,10,5,20</xpts> reaches simulation as-is and produces undefined output.

Why it matters

• Correctness: silently wrong simulation output for any model whose GF has out-of-order (or NaN) x-coordinates -- no crash, no diagnostic, just wrong numbers that cascade through every downstream variable. The result is internally self-consistent (deterministic for a given table), which makes it harder to notice than a crash would be.
• Developer experience / robustness: malformed input should be rejected at the boundary with an equation-level diagnostic, not silently miscomputed deep in the VM.

Component(s) affected

• src/simlin-engine -- GF import (xmile/dimensions.rs, serde.rs), the parse_table choke point (variable.rs), the compiler GF construction (compiler/expr.rs, compiler/codegen.rs), and the VM/wasmgen lookup consumers (vm.rs, wasmgen/lookup.rs).

Suggested fix direction

Validate at the single choke point all import paths funnel through -- parse_table (variable.rs) and/or Table::new (compiler/expr.rs) -- and emit an equation-level diagnostic when a GF's x-coordinates are not non-decreasing (or contain a NaN/non-finite value). Two decisions are needed before implementing, which is precisely why this was NOT fixed inline during the perf work:

1. Warning vs Error, and conformance semantics. Check the XMILE spec's GF semantics (docs/reference/xmile-v1.0.html) to decide whether unsorted x is invalid input (emit an Error) or whether the importer should sort the points on import (and whether sorting x must co-permute y). The Vensim/XMILE conformance behavior should drive this, not an ad-hoc choice.
2. Back-compat for existing DB-stored projects. Some serialized projects in the DB may already contain unsorted GF x-points; an Error gate would suddenly fail to load them. Decide whether to sort-on-load, warn-and-sort, or hard-error, accounting for the protobuf back-compat constraint (the repo's only hard back-compat requirement is serialized protobuf instances).

A regression test should cover: (a) a sorted table still compiles/simulates unchanged, (b) an unsorted table produces the chosen diagnostic (and/or is sorted), and (c) a NaN/non-finite x is rejected or handled.

Relationship to existing issues (DISTINCT)

• engine: optimize the graphical-function Lookup hot path (~4% of the C-LEARN run) #602 (optimize the GF lookup hot path) is where this gap was discovered, but it is about lookup speed and explicitly assumes sorted x; the perf change works around this gap with a per-table sortedness check at Vm::new, it does not close it.
• engine: standalone lookup-only variable is lowered to gf(Time) (unit error; phantom Time-indexed series for a bare graphical function) #606 (bare lookup-only var lowered to gf(Time)) is a different lowering bug (wrong index expression for a bare lookup), not x-axis input validation.
• engine: per-element graphical functions are mapped by Arrayed-Vec (sorted) position instead of element-name -> dimension index #589 (per-element GF mapped by sorted Vec position vs element-name -> dim index) is about which element's table a slot reads, not about a single table's x ordering.
• ltm: graphical-function monotonicity compares the y-delta, not the slope dy/dx (non-uniform x-spacing) #536 / ltm: GF strict-monotonicity check (EPSILON=1e-10) flags numeric-import noise as Unknown #492 (LTM GF monotonicity heuristics) are about y-axis monotonicity classification for link-polarity analysis, a best-effort heuristic -- not VM lookup correctness or x-axis input validation.
• layout: dependency-graph extraction misses lookup-table references, so lookup tables appear as isolated nodes with no connectors #650 (layout dep-graph misses lookup-table references) is a layout-quality issue, unrelated to lookup numerics.

Discovery context

Identified during adversarial review of the GF lookup hot-path perf optimization (#602). The perf change correctly gates its optimization behind a per-table sortedness check at Vm::new; this issue tracks the pre-existing upstream gap that an unsorted/NaN-x GF reaches the VM at all with no diagnostic.

Severity

Medium (silent correctness gap on malformed input; no crash, requires malformed source to trigger).