Problem
Editing a dimension (renaming or adding an element) re-lowers and recompiles every fragment in the project, not just the variables that use that dimension.
The parse tier has a per-variable dimension firewall: parse_source_variable reads only the dimensions its equation names (db::query::variable_relevant_dimensions, widened by expand_maps_to_chains and filtered out of project_datamodel_dims), so a scalar variable never depends on the project's dimensions at all and an arrayed one depends only on the dimensions it declares (db/dimension_invalidation_tests.rs). variable_dimensions mirrors the same narrowing for the layout and dependency shapes.
The fragment tier does not. Five sites read the whole-project context directly, so every lowering and fragment memo depends on the project-wide dimensions input and any dimension edit invalidates all of them:
src/simlin-engine/src/db/var_fragment.rs -- implicit_dep_shape (dimensions_named(&meta.dimensions, project_dimensions_context(..))), lowered_source_variable (LoweringScope.dimensions), explicit_fragment_input (dim_context + project_converted_dimensions handed to the compiler Module)
src/simlin-engine/src/db/fragment_compile.rs -- lowered_implicit_variable (implicit_var.parsed_variable(dim_context) and LoweringScope.dimensions), implicit_fragment_input (dim_context + converted_dims into FragmentInput::new)
compile_var_fragment consumes explicit_fragment_input, so it inherits the whole-context dependency.
Measurement
ProbedDb census (db/exec_probe.rs, salsa WillExecute counts) over an LTM-enabled two-model fixture with a stdlib helper and per-element helpers: adding one element to a dimension re-executes 77 fragment compiles and 70 lowerings -- the whole project. An equation edit on the same fixture re-executes 1 lowering and 1 fragment. Pre-existing on main and unchanged by PR #1040 (measured identically on both trees).
Why it matters
Correctness is fine: AC1.5 holds through salsa backdating, and test_dimension_invalidation_different_dim_immune pins that a DimB variable's fragment is value-equal after a DimA edit. This is purely the cost of the interactive edit loop the incremental compiler exists for. Dimension edits are a normal editing action in the web editor and via MCP edit_model; on a project with many dimensions and thousands of variables (C-LEARN class) each one pays a full recompile while an equation edit pays for one variable.
What is NOT pinned today
test_dimension_invalidation_different_dim_immune asserts value equality of the fragment, which backdating satisfies whether or not the body re-ran (exec_probe.rs's module docs spell this out: "only the event says which"). There is no execution-count test for the fragment tier over a dimension edit; the parse tier has them. The fix should land with one: a ProbedDb probe over a dimension edit asserting that lowered_source_variable / compile_var_fragment / lowered_implicit_variable / compile_implicit_var_fragment re-execute only for the variables that read the edited dimension, alongside the existing parse-side rows.
Possible approach
Replace the whole-context reads with a per-variable dimensions projection, the way the parse tier already does. The projection the fragment tier needs is wider than the parse tier's, which is what makes this its own piece of work rather than a one-line swap:
- the variable's own declared dimensions (
variable_relevant_dimensions, already narrowed);
- the declared dimensions of every dependency it reads (
variable_dimensions per dep is already narrowed; the fragment constructors already compute per-dep DepShapes through it -- compiler_shapes / expr2_shapes in var_fragment.rs -- so the shapes side is done and only the DimensionsContext / converted_dims handed to lowering and the compiler Module remain whole);
- any subdimension or element named in an index expression (
x[SubDimA], x[a1]), which is not a declared dimension of either side and today resolves only through the whole context;
- the
maps_to and parent chains of all of the above (expand_maps_to_chains), so mapped and sub-range reads still resolve.
A tracked query keyed on (var, project) that returns that closure as a DimensionsContext (and the matching converted_dims slice) backdates when the variable's own closure is unchanged, which is exactly the firewall the parse tier gets from variable_relevant_dimensions. Implicit helpers lower under their parent's shapes, so the helper's projection is the parent's.
One trap: implicit_dep_shape and dimensions_named resolve a helper's declared dimension names against the context; the projection must include those names for module-typed and arrayed helpers or the shape silently degrades to scalar. Cover that arm in the probe.
Component
src/simlin-engine (db/var_fragment.rs, db/fragment_compile.rs, db/query.rs). Tracked in docs/tech-debt.md entry 18 ("Dimension-Granularity Incremental Invalidation"), which states the current state and the measure; this issue is the work item for it.
Context
Identified during the compiler-unification-v2 branch (PR #1040) by its Phase 8.2/8.3 adversarial review (finding O4). Pre-existing on main; the branch's parse-tier firewall (AC3.1) stops at the parse and does not extend to the fragment tier.
Problem
Editing a dimension (renaming or adding an element) re-lowers and recompiles every fragment in the project, not just the variables that use that dimension.
The parse tier has a per-variable dimension firewall:
parse_source_variablereads only the dimensions its equation names (db::query::variable_relevant_dimensions, widened byexpand_maps_to_chainsand filtered out ofproject_datamodel_dims), so a scalar variable never depends on the project's dimensions at all and an arrayed one depends only on the dimensions it declares (db/dimension_invalidation_tests.rs).variable_dimensionsmirrors the same narrowing for the layout and dependency shapes.The fragment tier does not. Five sites read the whole-project context directly, so every lowering and fragment memo depends on the project-wide dimensions input and any dimension edit invalidates all of them:
src/simlin-engine/src/db/var_fragment.rs--implicit_dep_shape(dimensions_named(&meta.dimensions, project_dimensions_context(..))),lowered_source_variable(LoweringScope.dimensions),explicit_fragment_input(dim_context+project_converted_dimensionshanded to the compilerModule)src/simlin-engine/src/db/fragment_compile.rs--lowered_implicit_variable(implicit_var.parsed_variable(dim_context)andLoweringScope.dimensions),implicit_fragment_input(dim_context+converted_dimsintoFragmentInput::new)compile_var_fragmentconsumesexplicit_fragment_input, so it inherits the whole-context dependency.Measurement
ProbedDbcensus (db/exec_probe.rs, salsaWillExecutecounts) over an LTM-enabled two-model fixture with a stdlib helper and per-element helpers: adding one element to a dimension re-executes 77 fragment compiles and 70 lowerings -- the whole project. An equation edit on the same fixture re-executes 1 lowering and 1 fragment. Pre-existing onmainand unchanged by PR #1040 (measured identically on both trees).Why it matters
Correctness is fine:
AC1.5holds through salsa backdating, andtest_dimension_invalidation_different_dim_immunepins that aDimBvariable's fragment is value-equal after aDimAedit. This is purely the cost of the interactive edit loop the incremental compiler exists for. Dimension edits are a normal editing action in the web editor and via MCPedit_model; on a project with many dimensions and thousands of variables (C-LEARN class) each one pays a full recompile while an equation edit pays for one variable.What is NOT pinned today
test_dimension_invalidation_different_dim_immuneasserts value equality of the fragment, which backdating satisfies whether or not the body re-ran (exec_probe.rs's module docs spell this out: "only the event says which"). There is no execution-count test for the fragment tier over a dimension edit; the parse tier has them. The fix should land with one: aProbedDbprobe over a dimension edit asserting thatlowered_source_variable/compile_var_fragment/lowered_implicit_variable/compile_implicit_var_fragmentre-execute only for the variables that read the edited dimension, alongside the existing parse-side rows.Possible approach
Replace the whole-context reads with a per-variable dimensions projection, the way the parse tier already does. The projection the fragment tier needs is wider than the parse tier's, which is what makes this its own piece of work rather than a one-line swap:
variable_relevant_dimensions, already narrowed);variable_dimensionsper dep is already narrowed; the fragment constructors already compute per-depDepShapes through it --compiler_shapes/expr2_shapesinvar_fragment.rs-- so the shapes side is done and only theDimensionsContext/converted_dimshanded to lowering and the compilerModuleremain whole);x[SubDimA],x[a1]), which is not a declared dimension of either side and today resolves only through the whole context;maps_toand parent chains of all of the above (expand_maps_to_chains), so mapped and sub-range reads still resolve.A tracked query keyed on
(var, project)that returns that closure as aDimensionsContext(and the matchingconverted_dimsslice) backdates when the variable's own closure is unchanged, which is exactly the firewall the parse tier gets fromvariable_relevant_dimensions. Implicit helpers lower under their parent's shapes, so the helper's projection is the parent's.One trap:
implicit_dep_shapeanddimensions_namedresolve a helper's declared dimension names against the context; the projection must include those names for module-typed and arrayed helpers or the shape silently degrades to scalar. Cover that arm in the probe.Component
src/simlin-engine(db/var_fragment.rs,db/fragment_compile.rs,db/query.rs). Tracked indocs/tech-debt.mdentry 18 ("Dimension-Granularity Incremental Invalidation"), which states the current state and the measure; this issue is the work item for it.Context
Identified during the compiler-unification-v2 branch (PR #1040) by its Phase 8.2/8.3 adversarial review (finding O4). Pre-existing on
main; the branch's parse-tier firewall (AC3.1) stops at the parse and does not extend to the fragment tier.