engine: lazy IF evaluation — skip the untaken branch in the VM #711

Activity

1. bpowers commented on Jun 4, 2026

   

   bpowers

   on Jun 4, 2026

   OwnerAuthor

   More actions

   Measured NO-GO for now -- census data in docs/design/engine-performance.md (round 3, 2026-06-04), summarized:

   A dispatch census (temporary eval_bytecode counters + a stack-effect branch-span reconstruction over the fused stream the VM actually executes) on the engine-vm-perf branch:

   	C-LEARN	WORLD3-03
   executed dispatches/step (exact)	30,524	1,112
   flow If sites / executed Ifs per step	1,679 / 1,873	24 / 24
   dispatches lazy-IF would skip	4,859 (15.9%)	36 (3.25%)
   instruction-share estimate of the skip	~1.5% (~35k of ~2.4M instr/step)	trivial

   The 15.9% dispatch share collapses to ~1.5% instructions because 93% of the skippable opcodes are cheap scalar loads/binops (LoadVar 30%, BinStackVar 11%, LoadConstant 11%, ...; only 3.7% Lookup + 2.9% Apply), while per-step instruction count is dominated by per-element array/lookup/module work lazy-IF never touches. ~1.5% is below the ~4% build-layout measurement floor documented in the perf doc (negative result #3's methodology note), and this is the most cross-cutting change of the round-3 candidates (forward jumps touch the stack-depth validator, peephole/fusion jump maps, the symbolic layer, and wasmgen). The lowering also adds two jump dispatches per executed If back.

   Two findings worth keeping:

   • 69.8% of the skippable dispatches sit behind constant conditions (1,300 of 1,679 flow If sites take the same branch for the entire run -- policy switches). Those are compile-time / engine: time-invariant variable hoisting — a constant phase evaluated once per run_to #712-family targets (branch specialization), not runtime-jump targets.
   • Exact dispatch counts: C-LEARN executes ~30.5k dispatches/step, far below earlier loose estimates -- useful calibration for sizing any future dispatch-level work.

   Revisit conditions: a mispredict-bound microarchitecture (the round-1 Ryzen profile, where the dispatch indirect branch dominates and the If/SetCond data-dependent selection has real misprediction cost), or as part of a threaded-dispatch rewrite (#601) where the control-flow structure changes anyway.
2. bpowers

   added

   engineIssues with the rust-based simulation engineIssues with the rust-based simulation engine

   enhancementNew feature or requestNew feature or request

   on Jun 8, 2026
3. bpowers commented on Aug 10, 2026

   

   bpowers

   on Aug 10, 2026

   OwnerAuthor

   More actions

   Correction to the measurement premise in this issue's NO-GO — the verdict is unchanged

   The decision stands. Only its stated reason is wrong, and the wrong reason is the kind that shuts down further thought, so it is worth correcting where people will read it.

   docs/design/engine-performance.md records the NO-GO as:

   the instruction share is only ~1.5% (~35k of ~2.4M instr/step): below the ~4% layout-noise measurement floor

   That applies a cycles/binary-layout noise floor to an instruction-count measurement. The two floors are three orders of magnitude apart.

   Measured on the C-LEARN run across six independent build+run pairs of identical source (Ryzen 9950X):

   channel	sd across builds
   retired instructions	0.026%
   branches	0.028%
   cycles, quiet machine	1.65%
   cycles, machine under load	9.9%–11%

   A null control makes the same point directly — the identical binary run as both sides of an interleaved A/B, 5 rounds, medians:

   instructions   -0.003%
   branches       -0.004%
   cycles         -1.540%     <- an apparent "win" from nothing
   

   Retired instructions are a property of the program; cycles are a property of the machine executing it. So ~1.5% of instructions is roughly 58 sigma on that channel — comfortably measurable, not below any floor. The ~4% figure is real, but it bounds wall-clock/cycles claims from a single build pair, not instruction counts.

   What the verdict actually rests on

   Two reasons survive, and they are sufficient:

   1. Small absolute win for the highest design cost of the three candidates. Lazy If needs forward-jump opcodes, which touch codegen, max_stack_depth join validation, the peephole/fusion jump maps, the symbolic layer, and wasmgen parity. ~1.5% of instructions does not pay for that.
   2. The cheap part of the win is already taken. Fusing SetCond;If[;AssignCurr] into conditional-select opcodes removes 12.0% of executed dispatches against this issue's projected 15.9%, with none of that machinery — no jumps, no stack-depth join reasoning, no symbolic or wasmgen changes. It rests on the pair being adjacent by construction (compiler::codegen's Expr::If arm is the sole producer of both and emits them together; measured executed counts are exactly equal at 1,874,169 each on C-LEARN).

   So the remaining upside here is the residual after that fusion, against the full forward-jump design cost.

   Why this is worth a comment rather than a silent doc edit

   A wrong measurement premise in a design doc does not get caught by review: reviewers check the reasoning against the stated premise, not the premise against the world. "Below the measurement floor" reads as a fact and ends the discussion. This one had already carried a verdict, which is the evidence it was doing damage rather than sitting inert.

   Anyone picking this issue up would have read the invalid floor claim here first, so fixing only the doc would have preserved the trap in the place most likely to be read.

   The general convention that would have prevented it: when a doc or issue records a measurement-derived verdict, state which channel the number came from. A bare "only ~1.5%" invites comparison against whatever floor the reader has in mind.
4. bpowers

   added a commit that references this issue on Aug 11, 2026

   engine: performance pass on compile, simulation, and the LTM path (#1021

   ...

   a17be13