fix(codegen): closes #317 — wire String.matchAll codegen + NaN-box inner arrays (v0.5.415)

CLAUDE.md

@@ -8,7 +8,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 Perry is a native TypeScript compiler written in Rust that compiles TypeScript source code directly to native executables. It uses SWC for TypeScript parsing and LLVM for code generation.
 
-**Current Version:** 0.5.414
+**Current Version:** 0.5.415
 
 
 ## TypeScript Parity Status
@@ -150,6 +150,7 @@ First-resolved directory cached in `compile_package_dirs`; subsequent imports re
 
 Keep entries to 1-2 lines max. Full details in CHANGELOG.md.
 
+- **v0.5.415** — Closes #317: `text.matchAll(re)` aborted codegen with `perry-codegen Phase 2: expression StringMatchAll not yet supported` — surfaced compiling `effect/src/internal/cause.ts` (func 140) during the #309 compat sweep. The HIR variant `Expr::StringMatchAll` and the runtime helper `js_string_match_all` already existed (the lowering rule is shared with `string.match` in `crates/perry-hir/src/lower/expr_call.rs:4001`); only the codegen arm and the runtime_decls extern were missing, plus a latent bug in the runtime helper that stored inner array pointers as raw `f64::from_bits(ptr as u64)` bits rather than NaN-boxed POINTER_TAG values — so even after wiring codegen, `for (const m of text.matchAll(re)) { m[1] }` would have crashed reading the inner-array slot through the existing IndexGet path (which expects NaN-boxed values). **Fix across three files**: (1) `crates/perry-codegen/src/expr.rs` — new `Expr::StringMatchAll` arm right after `Expr::StringMatch` that mirrors its shape exactly: `unbox_str_handle` for SSO-safe receiver unbox (same #214 SSO bug class as `StringMatch`), `unbox_to_i64` for the regex handle, `call I64 js_string_match_all`, then `nanbox_pointer_inline` on the resulting `*mut ArrayHeader`. (2) `crates/perry-codegen/src/runtime_decls.rs` declares `js_string_match_all (I64, I64) -> I64` immediately after the existing `js_string_match` decl. (3) `crates/perry-runtime/src/regex.rs::js_string_match_all` swaps the inner-array slot store from `f64::from_bits(inner_ptr as u64)` to `crate::value::js_nanbox_pointer(inner as i64)` — same NaN-boxing convention as `js_array_group` at `array.rs:1493` and as the inner string elements `js_nanbox_string` at `js_string_match` line 313. Empty-result and no-match cases preserved (matchAll returns an empty array, never null — distinct from `match` which returns null on no match). New regression test `test-files/test_issue_317_string_matchall.ts` covers: the issue's literal repro (`for...of` over `/([a-z]+)=([a-z]+)/g` with `m[1]` / `m[2]` capture access), three-capture-group dates with full-match `m[0]`, no-match returning empty iterable (count 0), spread `[...text.matchAll(re)]` with index-into-result, and single-match — matches `node --experimental-strip-types` byte-for-byte. **Verified:** cargo build clean; gap tests 27/28 = baseline (lone fail is pre-existing `console_methods` ci-env quirk); parity 179/179 (100%); regression test matches Node byte-for-byte across all 5 sections. Companion follow-up under umbrella #321 — flips Effect's `internal/cause.ts` from "rejected at codegen" to compiling cleanly.
 - **v0.5.414** — Closes #315: `String.prototype.startsWith(searchString, position)` and `endsWith(searchString, endPosition)` (the standard 2-arg ES forms) were rejected at codegen with `perry-codegen: String.startsWith expects 1 arg, got 2` — surfaced compiling Effect's `src/String.ts` (func 19) during the #309 compat sweep. The 1-arg `lower_string_method.rs` arm hard-bailed on `args.len() != 1`, and `lower_call.rs` only routed Any-typed receivers to the string dispatcher when `args.len() == 1`. **Fix in three places**: (1) new `js_string_starts_with_at` / `js_string_ends_with_at` in `crates/perry-runtime/src/string.rs` that take a position i32 and use the existing `is_ascii_string` / `utf16_offset_to_byte_offset` helpers (UTF-16 code-unit indexing per spec, position clamped to `[0, len]`). (2) `runtime_decls.rs` declares both new externs `(I64, I64, I32) -> I32`. (3) `lower_string_method.rs:548` widens the gate to `args.len() in 1..=2` and dispatches the 2-arg form to the `_at` runtime variant via `fptosi(DOUBLE → I32)` on the position arg — matches the `slice` / `substring` pattern. (4) `lower_call.rs:934` widens the Any-typed-receiver routing gate from `args.len() == 1` to `args.len() == 1 || args.len() == 2` for `startsWith / endsWith` since neither method exists on Array, so 2-arg dispatch is unambiguous. New regression test `test-files/test_issue_315_starts_with_position.ts` covers 1-arg + 2-arg forms, position clamping (negative, beyond length), and multi-byte UTF-8 / UTF-16 indexing (`αβγδε`) — matches `node --experimental-strip-types` byte-for-byte. **Verified:** cargo build clean; gap tests 27/28 = baseline (lone fail is pre-existing `console_methods` ci-env quirk); regression test matches Node byte-for-byte. Companion follow-up under umbrella #321 — flips Effect's `String.ts` from "rejected at codegen" to "compiles cleanly".
 - **v0.5.413** — Closes #324: `Array.isArray(value)` constant-folded to `TAG_TRUE` for any value statically typed as a Union with at least one Array variant — so `function hook(value: number | readonly number[]) { if (Array.isArray(value)) { ... } else { ... } }` always picked the array branch even when the runtime value was a `number`. Surfaced on @codehz's ECS demo (same issue reporter as #313). Root cause: the `Expr::ArrayIsArray` lowering at `crates/perry-codegen/src/expr.rs:6851` short-circuited to TAG_TRUE whenever `is_array_expr(ctx, o)` returned true, but that helper is deliberately loose — it returns true if *any* variant of a Union is `Array(_)` / `Tuple(_)`, which is correct for routing `.length` / `.push` / `[i]` dispatch on `T[] | null` after a truthy narrow (so `(maybeArr || []).slice()` still hits the array fast path), but wrong for `Array.isArray` which must reflect the actual runtime tag. The HIR confirms: `function hook(value: number | readonly number[])` lowered the parameter as `Union([Number, Array(Number)])`, `is_array_expr` returned true via the union arm, and `ArrayIsArray(LocalGet(0))` constant-folded to TAG_TRUE before the runtime ever saw the integer 1. **Fix in one place** (`expr.rs::Expr::ArrayIsArray`): replace the loose `is_array_expr(ctx, o)` check with a strict `matches!(ty, Type::Array(_) | Type::Tuple(_))` direct match against `static_type_of(ctx, o)`. Pure `T[]` / `[T, U]` types still constant-fold to TAG_TRUE; Union shapes (including the `T[] | null` post-narrow case) fall through to the existing runtime `js_array_is_array` dispatch which correctly inspects the NaN-box tag and the GC type. The FALSE side already correctly skipped Union (no match), so the only behavioral change is on the TRUE side. New regression test `test-files/test_issue_324_array_isarray_union.ts` covers: `number | readonly number[]` parameter in if-guard with both numeric and array call shapes (the issue's exact repro), `string | number[]` ternary form, `number[] | undefined` optional form, and a control case verifying the fast path still fires on a definitively-Array parameter. Matches `node --experimental-strip-types` byte-for-byte. **Verified:** cargo build clean; gap tests 27/28 = baseline (lone fail is pre-existing `console_methods` ci-env quirk); regression test matches Node byte-for-byte.
 - **v0.5.412** — Closes #323: `const values = new Array(4); values[1]` returned `0` instead of `undefined`, `1 in values` and `2 in values` (after `values[2] = undefined`) both returned `false` regardless of presence, and `Object.keys(values)` happened to return `[]` only because uninitialized arena bytes read as zero (a fresh `js_object_keys` call after the title-bug fix segfaulted dereferencing slot[1] as a `keys_array` pointer). Found in user @codehz's ECS demo. **Root cause** layered four overlapping defects: (1) `js_array_alloc_with_length` in `crates/perry-runtime/src/array.rs` left element bytes uninitialized — JS spec says `new Array(n)` slots are *holes* that read as `undefined`; (2) `js_object_keys` and `js_object_has_property` cast their argument to `*const ObjectHeader` regardless of the actual GC type, so passing an array walked the slot bytes as ObjectHeader fields (`object_type=length`, `keys_array=elements[1]`); (3) `Object.keys(arr)` and `n in arr` had no array-specific path — the former relied on the slot[1] zero-coincidence, the latter always returned false because the key-is-string guard rejected numeric keys; (4) the codegen's three inline IndexGet fast paths in `crates/perry-codegen/src/expr.rs` (bounded-index loop fast path, generic IndexGet, generic-Object numeric fallback) each emitted a raw `load DOUBLE, ptr` at `arr+8+idx*8` — bypassing `js_array_get_f64`'s translation entirely. **Fix across five files** with a HOLE sentinel approach: (a) `crates/perry-runtime/src/value.rs` defines `pub(crate) const TAG_HOLE: u64 = 0x7FFC_0000_0000_0010` (next free slot in the 0x7FFC singleton namespace, after UNDEFINED/NULL/FALSE/TRUE); (b) `js_array_alloc_with_length` initializes every reachable slot (`0..capacity`, the requested length — slots `capacity..actual_capacity` are unreachable through the bounds-checked accessor) to TAG_HOLE; (c) `js_array_get_f64` and `js_array_get_f64_unchecked` translate HOLE → UNDEFINED at the read site so the sentinel never leaks to user code; (d) `js_object_keys` detects ArrayHeader by `GcHeader::obj_type == GC_TYPE_ARRAY`, walks the slot bytes, and emits `js_string_new_sso` decimal indices for every non-HOLE slot — short-circuits before the ObjectHeader path; (e) `js_object_has_property` detects ArrayHeader by the same GC type byte, parses the key as a numeric index (accepting both NaN-boxed i32 and plain f64 integers in `[0, length)`), and returns true iff `slot != TAG_HOLE`; (f) `crates/perry-codegen/src/nanbox.rs` mirrors `TAG_HOLE_I64 = "9222246136947933200"` for codegen use, with a tag-strings-match-u64 unit-test pin to catch any future drift; (g) all three inline IndexGet paths in `crates/perry-codegen/src/expr.rs` emit a branchless `bitcast double→i64` + `icmp eq TAG_HOLE_I64` + `select` after the raw load, so user code reading `arr[i]` never observes the sentinel even on the hot inline path; (h) `crates/perry-codegen/src/type_analysis.rs::refine_type_from_init` adds `Expr::New { class_name: "Array", .. } => Some(HirType::Array(Box::new(HirType::Any)))` so `const xs = new Array(n)` (no annotation) gets refined to Array<Any> — without this, the local stays at type `Any`, `is_array_expr` returns false, and `xs[i]` falls through to the generic-Object numeric fallback path; (i) `crates/perry-codegen/src/lower_call/builtin.rs` updates the misleading "(zero-initialized slots)" comment. New regression test `test-files/test_issue_323_array_holes.ts` covers the issue's exact 11-line repro shape (length / hole-read / `===` / `in` / `Object.keys` / explicit-undefined-write / post-write-`in`), the MIN_ARRAY_CAPACITY=16 padding boundary (size 20), the empty-array edge case (size 0), and a function-scoped `for` loop reading `arr[i]` over a fresh `new Array(3)` (exercises the bounded-index inline fast path's HOLE→UNDEFINED translation distinct from the generic IndexGet path). Matches `node --experimental-strip-types` byte-for-byte. **Verified:** cargo build clean; gap tests 27/28 = baseline (lone fail is pre-existing `console_methods` ci-env quirk); parity 179/179 (100% on a clean re-run; one earlier run showed 2 flaky `test_net_*` failures from a stale TCP listener bind on port 8080 — not caused by this change); user's literal repro now matches Node byte-for-byte across all 11 lines. Caveat: the `--no-cache` flag is required when re-compiling cached test files after this change because the build cache keys on source bytes, not type-analysis pass output — the cached `.o` was built before #323's `is_array_expr` flip recognized `new Array(...)` as Array<Any>.

Cargo.toml

@@ -115,7 +115,7 @@ opt-level = "s"       # Optimize for size in stdlib
 opt-level = 3
 
 [workspace.package]
-version = "0.5.414"
+version = "0.5.415"
 edition = "2021"
 license = "MIT"
 repository = "https://github.com/PerryTS/perry"

crates/perry-codegen/src/expr.rs

@@ -4997,6 +4997,25 @@ pub(crate) fn lower_expr(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result<String> {
             Ok(nanbox_pointer_inline(blk, &result))
         }
 
+        // -------- string.matchAll(regex) --------
+        // Returns Array<Array<string>>, never null. Each inner array is
+        // [fullMatch, ...captureGroups], matching the shape Node produces
+        // when iterating `for (const m of s.matchAll(re))`. SSO-safe receiver
+        // unbox via `unbox_str_handle` for the same reason as `StringMatch`.
+        Expr::StringMatchAll { string, regex } => {
+            let s_box = lower_expr(ctx, string)?;
+            let r_box = lower_expr(ctx, regex)?;
+            let blk = ctx.block();
+            let s_handle = unbox_str_handle(blk, &s_box);
+            let r_handle = unbox_to_i64(blk, &r_box);
+            let result = blk.call(
+                I64,
+                "js_string_match_all",
+                &[(I64, &s_handle), (I64, &r_handle)],
+            );
+            Ok(nanbox_pointer_inline(blk, &result))
+        }
+
         // -------- obj.field++ / obj.field-- (PropertyUpdate) --------
         // Lowered as: load → fadd/fsub 1.0 → store. Same as the
         // Update variant but for a property instead of a local.

crates/perry-codegen/src/runtime_decls.rs

@@ -584,6 +584,7 @@ pub fn declare_phase_b_strings(module: &mut LlModule) {
     module.declare_function("js_path_relative", I64, &[I64, I64]);
     module.declare_function("js_object_from_entries", DOUBLE, &[DOUBLE]);
     module.declare_function("js_string_match", I64, &[I64, I64]);
+    module.declare_function("js_string_match_all", I64, &[I64, I64]);
     module.declare_function("llvm.log.f64", DOUBLE, &[DOUBLE]);
     module.declare_function("llvm.log2.f64", DOUBLE, &[DOUBLE]);
     module.declare_function("llvm.log10.f64", DOUBLE, &[DOUBLE]);

crates/perry-runtime/src/regex.rs

@@ -394,9 +394,12 @@ pub extern "C" fn js_string_match_all(
                 }
             }
 
-            // Store inner array as NaN-boxed pointer in outer array
-            let inner_ptr = inner as i64;
-            std::ptr::write(outer_elements.add(i), f64::from_bits(inner_ptr as u64));
+            // Store inner array as NaN-boxed POINTER_TAG in outer array slot —
+            // raw `inner as i64 -> f64::from_bits` would write a non-NaN-boxed
+            // double whose bits happen to alias the heap pointer; the codegen
+            // IndexGet path then reads `arr[i]` as a plain number and crashes
+            // when iterating with `for (const m of arr) m[1]`.
+            std::ptr::write(outer_elements.add(i), crate::value::js_nanbox_pointer(inner as i64));
         }
 
         outer

test-files/test_issue_317_string_matchall.ts

@@ -0,0 +1,46 @@
+// Regression for #317: String.prototype.matchAll(regex) used to fail at
+// codegen with "perry-codegen Phase 2: expression StringMatchAll not yet
+// supported". HIR + runtime helper already existed; only the codegen arm
+// and runtime_decls extern were missing. Surfaced compiling
+// effect/src/internal/cause.ts during the #309 compat sweep.
+
+// Issue's literal repro: `for...of` over matchAll, accessing capture
+// groups by index. The runtime fix also NaN-boxes inner array pointers
+// so `m[1]` / `m[2]` read back correctly through the IndexGet path
+// (without the fix the slot held raw pointer bits and `m[1]` returned a
+// nonsense double).
+const text = "key=val&foo=bar&baz=qux";
+const re = /([a-z]+)=([a-z]+)/g;
+for (const m of text.matchAll(re)) {
+  console.log(m[1], "->", m[2]);
+}
+
+// Full match at index 0 + named indices.
+const text2 = "2026-04-30 and 2026-05-01";
+const dateRe = /(\d{4})-(\d{2})-(\d{2})/g;
+for (const m of text2.matchAll(dateRe)) {
+  console.log(m[0], m[1], m[2], m[3]);
+}
+
+// No matches: matchAll returns empty iterable, never null.
+const noMatch = "hello world";
+const digits = /\d+/g;
+let count = 0;
+for (const _ of noMatch.matchAll(digits)) count++;
+console.log("noMatch count:", count);
+
+// Spread into Array — each entry is itself an array of (full, ...groups).
+const text3 = "aaa bbb ccc";
+const wordRe = /(\w)\w+/g;
+const arr = [...text3.matchAll(wordRe)];
+console.log("len:", arr.length);
+console.log(arr[0][0], arr[0][1]);
+console.log(arr[1][0], arr[1][1]);
+console.log(arr[2][0], arr[2][1]);
+
+// Single match.
+const text4 = "only one";
+const oneRe = /one/g;
+for (const m of text4.matchAll(oneRe)) {
+  console.log("single:", m[0]);
+}