fix(codegen): teach the in-process IR reader the vector instructions codegen emits (#8228)

[proggeramlug]

Fixes #8228.

What was broken

Perry's in-process LLVM backend round-trips its own textual IR through a
hand-written dialect reader (crates/perry-codegen/src/dialect/). That reader
accepts a closed set of instruction forms by design — anything else is
supposed to fail loudly rather than diverge silently.

#8204 (the #8122 header-shrink instruction-cost recovery) started emitting the
per-class object-header image as a <2 x i64> composed at module init:

%r7 = insertelement <2 x i64> <i64 207232172546, i64 0>, i64 %r6, i32 1

The reader has no insertelement case, so the line fell through to the
binary-op arm, which sees three operands where it wants two:

native codegen unit 2/3 failed: unit 1: in line:
  %r7 = insertelement <2 x i64> <i64 207232172546, i64 0>, i64 %r6, i32 1:
  bad binary op `insertelement` operands: <2 x i64> <i64 207232172546, i64 0>, i64 %r6, i32 1

The reader is only the default for split (multi-unit) modules
(native_emit::native_units_mode), so this hit exactly the modules big enough to
split — the five biggest of the Next App Route fixture, including
next-server/app-route.runtime.prod.js itself.

What this covers, and what it deliberately does not

git grep over crates/perry-codegen/src for every vector/aggregate form gives
two emitters, and both reach the reader as Raw text:

form	emitter
insertelement <2 x i64> <i64 W, i64 0>, i64 %hdr, i32 1	function.rs:586, codegen/string_pool.rs:514
insertelement <4 x i32> {<i32 0,…>|%reg|poison}, i32 %v, i32 N	expr/channel.rs:401,410,440
shufflevector <4 x i32> %a, <4 x i32> poison, <4 x i32> zeroinitializer	expr/channel.rs:415
extractelement <4 x i32> %v, i32 N	expr/channel.rs:469
mul / add over <4 x i32>	expr/channel.rs:421,449

All of it is now accepted:

• dialect/vector.rs (new; split out for the 2000-line cap, following
  eh.rs/types.rs) — insertelement, extractelement, shufflevector, and
  the vector arm of binary arithmetic. The last one is not cosmetic: inkwell's
  integer builders are generic over VectorValue, but the scalar arm reaches
  its operands through into_int_value(), which panics on a vector instead
  of returning an error, so mul <4 x i32> would have aborted the compiler
  rather than reported anything.
• dialect/types.rs — constant vector literals (<i64 W, i64 0>,
  <i32 0, i32 0, i32 0, i32 0>) plus poison / undef / zeroinitializer
  of vector type. channel.rs seeds a splat from poison and its shuffle mask
  is zeroinitializer, so those are emitted forms, not corners.

channel.rs's <4 x i32> family was the same latent bug — it has simply
never been reached, because a byte-channel reduction has not yet shown up in a
module large enough to split. It is covered by this fix.

Deliberately not implemented: insertvalue and extractvalue. Perry emits
neither — the tree's only extractvalue is an input fixture to
retype_landing_pads_for_statepoints, not emitted IR. Adding them would be arms
no test can reach, which is the opposite of what a closed reader is for. Vector
binary ops other than add/mul bail with a message naming the emitter, for
the same reason.

Why did this reach main?

Because the reader's only gate is a set of frozen snapshots. All three
dialect::tests::corpus_* tests round-trip .ll files checked in under
experiments/llvm-inprocess-spike/. A form the emitters started producing
after those files were captured is invisible to them by construction.

Nothing else covered it either:

• the reader is the default only on the multi-unit path, and every
  gap/parity fixture is small enough to be a single unit, which keeps the
  mature text transport;
• PERRY_LLVM_INPROCESS=0 (external clang/opt) compiles all 104 modules clean
  at the same commit, and that is the configuration
  tests/test_next_app_route_dylib.sh pins;
• the one build big enough to split is the release-tier App Route fixture,
  which is tag-gated.

So the failure needed a large in-process build to appear, and no per-PR job
performs one.

The test added

dialect::tests::compiled_module_ir_round_trips_through_the_reader replaces the
snapshot with a live emit → re-parse: it compiles a real HIR module (a class
new'd in a loop, which admits the inline bump allocator and therefore the
header image) through the real emitters, then feeds every function of the
resulting IR back through the reader and verifies the module. It asserts the
insertelement <2 x i64> <i64 compose is present before round-tripping, so a
fixture that stopped exercising the allocator fails loudly instead of passing
vacuously. This closes the class, not the instance: the next new emission form
goes red in cargo-test.

dialect::tests::channel_reduction_vector_forms_round_trip covers the
<4 x i32> family, which no HIR fixture reaches cheaply. Its IR is built from
the emitter's own format! templates rather than from duplicated literals, and
each template is asserted to still be present verbatim in expr/channel.rs — so
if the emitter's text changes, this fixture changes with it or the test fails.

Validation

All on the bench mini (M1, macOS 26.5.1 arm64, LLVM 22.1.8), perry-dev profile.

Sabotage A/B — the new tests are the detector

Reverting only the fix (dialect/{mod,types}.rs back to main,
dialect/vector.rs removed) while keeping the new tests, on the same tree:

running 5 tests
test dialect::tests::channel_reduction_vector_forms_round_trip ... FAILED
test dialect::tests::compiled_module_ir_round_trips_through_the_reader ... FAILED
test dialect::tests::corpus_exception_handling ... ok
test dialect::tests::corpus_spike ... ok
test dialect::tests::corpus_batch_kernel ... ok

---- dialect::tests::compiled_module_ir_round_trips_through_the_reader stdout ----
in line: %r10 = insertelement <2 x i64> <i64 207232172546, i64 0>, i64 %r9, i32 1:
  bad binary op `insertelement` operands: <2 x i64> <i64 207232172546, i64 0>, i64 %r9, i32 1

207232172546 is the same packed header word the production failure reports, so
the unit test reproduces the field failure exactly. And the three frozen-corpus tests
pass in both arms — the demonstration that they could never have caught this.

Test counts

• cargo build --profile perry-dev -p perry — clean, no new warnings.
• cargo test --profile perry-dev -p perry-codegen --lib — 1057 passed, 0 failed
  (1055 before; the two new tests are the delta).
• cargo fmt --all -- --check, scripts/check_file_size.sh,
  scripts/addr_class_inventory.py — clean.

crates/perry-codegen/tests/*.rs were not run: this change touches only
src/dialect/, which no integration suite covers.

Acceptance: the App Route fixture's app dylib

Same tree, same fixture, same runtime archives (the diff is codegen-only, so both
arms share one PERRY_RUNTIME_DIR); only the compiler binary differs.
PERRY_MODULE_JOBS=2 PERRY_CODEGEN_UNIT_JOBS=2, in-process backend (default),
perry compile perry-host.js --output-type dylib --no-auto-optimize --no-cache.

Before — exit 1 after 14m19s, all five modules from #8228, one signature:

Error compiling module '.next/server/chunks/2.js' … with --backend llvm:
  native codegen unit 2/3 failed: unit 1: in line:
  %r7 = insertelement <2 x i64> <i64 207232172546, i64 0>, i64 %r6, i32 1:
  bad binary op `insertelement` operands: <2 x i64> <i64 207232172546, i64 0>, i64 %r6, i32 1
… chunks/430.js            native codegen unit 4/6 failed: unit 3
… jsonwebtoken/index.js    native codegen unit 2/2 failed: unit 1
… app-page.runtime.prod.js native codegen unit 8/10 failed: unit 7
… app-route.runtime.prod.js native codegen unit 3/4 failed: unit 2

✗ 5 module(s) failed to compile — REFUSING TO LINK

After — exit 0 after 15m24s, Error compiling module count 0, all 104
modules compiled, Wrote shared library: next-app-after.dylib (87,490,488 bytes),
and nm -u still shows js_gc_init undefined (the fixture's own check that the app
dylib does not embed the Perry ABI).

Third arm: the defect is isolated to the native path

Same unfixed main compiler, same fixture, only PERRY_LLVM_INPROCESS=off added
(that selects the mature text transport, native_emit::native_units_mode):
exit 0 in 9m15s, Error compiling module count 0, dylib written. So the three
arms are

arm	compiler	backend	result
1	main	in-process (default)	exit 1 — 5 modules, bad binary op \insertelement``
2	main	PERRY_LLVM_INPROCESS=off	exit 0, dylib written
3	main + this fix	in-process (default)	exit 0, dylib written

Arm 2 is a usable interim workaround and is now noted on #8228, but it is not free:
the text path materialized 222.6 MB of IR for one module and produced a 206 MB
dylib against the native path's 87 MB (the giant modules drop to -O0 there).
It is a way to keep moving, not a reason to leave the native default broken.

Running the App Route end to end is out of scope here and owned separately; this PR
is done when the dylib links.

[coderabbitai]

Warning

Review limit reached

@proggeramlug, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 1 minute

Limit details: You’ve used all 8 included reviews currently available under your plan.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ccd548a8-4c67-4485-9132-30c64d190378

📥 Commits

Reviewing files that changed from the base of the PR and between 537f74a and a66993f.

📒 Files selected for processing (5)

• changelog.d/8241-dialect-vector-instructions.md
• crates/perry-codegen/src/dialect/mod.rs
• crates/perry-codegen/src/dialect/tests.rs
• crates/perry-codegen/src/dialect/types.rs
• crates/perry-codegen/src/dialect/vector.rs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}