Skip to content

Decompose codegen/api.py into a vera/runtime/ package (#421) - #789

Merged
aallan merged 16 commits into
mainfrom
feat/421-runtime-decomposition
Jun 24, 2026
Merged

Decompose codegen/api.py into a vera/runtime/ package (#421)#789
aallan merged 16 commits into
mainfrom
feat/421-runtime-decomposition

Conversation

@aallan

@aallan aallan commented Jun 24, 2026

Copy link
Copy Markdown
Owner

Summary

Decomposes the 4,358-line vera/codegen/api.py public-API module, extracting the wasmtime execution runtime into a new vera/runtime/ package. api.py: 4,358 → 1,165 lines (−73%), one layer per commit, each kept green against the #734 characterization harness.

Closes #421.

What moved to vera/runtime/

Infrastructure

  • ADT memory-layout utilities (ConstructorLayout, alignment helpers) → vera/codegen/memory.py
  • Trap classification + source backtraces (WasmTrapError, _classify_trap, _resolve_trap_frames) → runtime/traps.py
  • WASM heap marshalling — memory read/write, GC shadow-rooting, the ADT / Option / Array / Map-Set-bucket codecs — → runtime/heap.py
  • The Map/Set value-type dispatch table → runtime/collections.py

The twelve optional effect families → one register_<family>(linker, …) module each:
random, math, md, json, regex, html, map, set, decimal, http, inference, state.

execute() now calls these in sequence instead of inlining ~3,000 lines of branches. Shared state a family needs is passed explicitly (e.g. register_decimal(linker, ops, decimal_store, host_store_refs)).

What stayed inline — and why

The always-on IO family stays in execute() by design. Unlike the twelve pluggable adapters (registered conditionally, owning their own state, feeding nothing back into the result), IO is execute()'s observation channel: its host callbacks write into state that becomes the return valueoutput_buf/stderr_bufExecuteResult.stdout/stderr, last_violation → the trap diagnostic via _classify_trap, tee_stdout → the live-streaming decision — and it shares the _VeraExit Ctrl-C exception with execute()'s exit handling. Extracting the twelve adapters reduced coupling; extracting IO would only relocate a naturally-cohesive unit across a file boundary behind a 7-field context object. The decision and its reasoning are documented in vera/README.md → "Host-binding families (vera/runtime/)", a heading comment on api.py's IO section, and the runtime/__init__ docstring.

Public surface — unchanged

from vera.codegen.api import compile, execute, CompileResult, ExecuteResult, WasmTrapError and from vera.codegen import ConstructorLayout all still resolve. The compiled .wasm import interface is byte-for-byte identical. Internal refactor; no behaviour change.

Verification

Bonus

Lifting the ~3,000-line runtime out of the public-API module unblocks the deferred whole-vera/ mutation sweep (#387) — api.py's oversized execute() was inflating a 774 MB mutant file mutmut couldn't index.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Expanded runtime-backed effects: HTML, JSON, regex, random, math, state, map, set, HTTP, markdown, inference, decimal, plus improved trap reporting.
  • Bug Fixes
    • Reduce leftover temporary .vera files when checks/tests are interrupted.
    • Improved UTF-8 decode handling so errors are reported more cleanly.
  • Documentation
    • Updated changelog, testing/coverage totals, known issues, and runtime documentation to reflect the latest capabilities.

aallan and others added 12 commits June 24, 2026 09:16
… repo root

scripts/check_html_examples.py and tests/test_html.py validated each HTML doc snippet by writing it to a NamedTemporaryFile with dir=ROOT (the repo root) and running vera check/verify on the absolute path. Since the subprocess takes the abspath, the file never needed to live in the repo -- but an interrupted run (a killed pre-commit, Ctrl-C mid-pytest) stranded a tmp*.vera in the working tree, showing up untracked in git status. Dropped dir=ROOT so the temp files go to the system tempdir; added a /tmp*.vera .gitignore backstop. Both checkers verified green, no repo-root leak. Doc count bumped (test_html.py 189->187).

Co-Authored-By: Claude <noreply@anthropic.invalid>
Step 1 of the codegen/api.py decomposition (#421): the pure ADT memory-layout utilities (ConstructorLayout, _validate_wrap_handle, _wasm_type_size/_wasm_type_align/_align_up) are lifted out of the 4,358-line api.py into a focused vera/codegen/memory.py with no wasmtime/runtime dependency. Consumers (codegen/__init__, core, registration, closures, and the _validate_wrap_handle unit tests) import them from memory.py directly; api.py keeps only _validate_wrap_handle, which _wrap_handle uses internally inside execute().

Public surface unchanged: 'from vera.codegen import ConstructorLayout' and 'from vera.codegen.api import compile/execute/CompileResult/ExecuteResult' all resolve. api.py 4,358->4,247 lines. mypy + ruff clean; the #734 characterization harness (22 tests) and the moved-fn tests (6) stay green.

Co-Authored-By: Claude <noreply@anthropic.invalid>
Step 2 of the codegen/api.py decomposition (#421): the runtime-trap classification + source-backtrace machinery (WasmTrapError, TrapFrame, _find_frames_in_exception_chain, _resolve_trap_frames, _TRAP_FIX_PARAGRAPHS, _classify_trap) moves out of api.py into a new vera/runtime/ package (traps.py) -- the first member of the execution-runtime package. The block is self-contained (no api.py-internal deps); wasmtime is annotation-only there so it sits under TYPE_CHECKING.

api.py re-exports WasmTrapError (part of execute()'s public contract, kept importable for cli.py + the #734 harness) and imports _classify_trap/_resolve_trap_frames internally; the Diagnostic annotation import shifts to api.py's TYPE_CHECKING block. test_runtime_traps.py imports the helpers from runtime.traps. api.py 4,247->3,790 lines. mypy + ruff clean; the trap suite + #734 gate (88) stay green.

Co-Authored-By: Claude <noreply@anthropic.invalid>
…heap.py

Step 3 of the codegen/api.py decomposition (#421): the ~820-line heap-marshalling helper layer -- memory read/write, GC shadow-stack rooting (_ShadowGuard), wrapper-ADT handle tagging (_wrap_handle), the Map/Set bucket codec, and Result/Option/Array allocation -- lifts out of execute() into vera/runtime/heap.py. The helpers were already parameterised by the wasmtime.Caller, so they become plain module-level functions; the one closed-over 'store' (_read_wasm_string) is fixed to use 'caller' like its siblings.

execute() and the still-nested host families call them via a module-level import (35 names; 17 stay heap-internal). The dead _validate_wrap_handle import is dropped from api.py. Repointed the source-location test test_read_wasm_string_uses_errors_replace at heap.py (the function moved). api.py 3,790->3,001 lines. mypy + ruff clean; the full suite (4,726), 92 conformance, 35 examples, and the #734 gate stay green.

Co-Authored-By: Claude <noreply@anthropic.invalid>
…/runtime/

Step 4 of the codegen/api.py decomposition (#421): the two stateless host-binding families -- Random (#465) and Math (#467) -- move out of execute() into vera/runtime/random.py and vera/runtime/math.py, each exposing a register_<family>(linker, ops_used) that defines and registers its host callbacks. execute() now calls register_random/register_math instead of inlining the blocks. This templates the per-family pattern for the stateful families to follow. api.py 3,001->2,884 lines. mypy + ruff clean; the random/math codegen tests (27) + #734 gate stay green.

Co-Authored-By: Claude <noreply@anthropic.invalid>
…hared collection helpers

Step 5 of the codegen/api.py decomposition (#421).  The Markdown (#9.7.3) and JSON families move out of execute() into vera/runtime/md.py (register_md(linker)) and vera/runtime/json.py (register_json(linker, ops_used) -- it has per-op guards).  Extracting json surfaced a gap the heap layer left behind: a block of pure marshalling helpers (_write_i64, _write_f64, _read_i32, _read_f64, _alloc_option_some_{i64,i32,f64}, _alloc_array_of_{i64,i32,f64}) was defined under a conditional deep in execute() and shared by all five collection families.  These are caller-parameterised, so they relocate unchanged to vera/runtime/heap.py; the still-inline Map/Set/Decimal/HTML blocks and the new json.py import them.  _VAL_WASM_TYPES and the _host_store_refs-bound host_decref_handle/host_attach_bucket stay inline (they go with the collections layer).  api.py 2,884->2,592 lines.  mypy + ruff clean; 1,233 codegen/characterization + 63 md/html tests green.

Co-Authored-By: Claude <noreply@anthropic.invalid>
Step 6 of the codegen/api.py decomposition (#421).  The Regex (§9.6.15) and HTML (§9.7.4) families move out of execute() into vera/runtime/regex.py and vera/runtime/html.py via register_regex/register_html.  Both are stateless (the apparent 'store' uses flagged earlier were all in comments) -- the host callbacks rely only on the now-complete vera.runtime.heap helpers plus their own lazy imports (re, html.parser, vera.wasm.html_serde).  html.py needed an explicit 'from typing import Any' (the annotation lived on api.py's module imports before).  api.py 2,592->2,171 lines.  mypy + ruff clean; 40 regex/html codegen tests pass and ch09_regex/ch09_html run end-to-end.

Co-Authored-By: Claude <noreply@anthropic.invalid>
…ollections.py

Step 7 of the codegen/api.py decomposition (#421).  The Map<K,V> and Set<T> families move out of execute() into vera/runtime/map.py and vera/runtime/set.py via register_map/register_set(linker, ops_used).  Both dispatch per element/key/value type through _VAL_WASM_TYPES, which moves to a small shared vera/runtime/collections.py constant module that map.py and set.py import.  The families' nested _define_*/_make_option/_map_put helpers travel inside the register functions; everything else they need (bucket codec, Option/Array/String allocators, _WRAP_KIND_* tags) is already module-level in vera.runtime.heap.  The _host_store_refs-coupled host_decref_handle/host_attach_bucket shared runtime stays inline (it goes with the decimal layer).  api.py 2,171->1,735 lines (-60% from the original 4,358).  mypy + ruff clean; 91 map/set codegen tests pass and ch09_map/ch09_set run end-to-end.

Co-Authored-By: Claude <noreply@anthropic.invalid>
Step 8 of the codegen/api.py decomposition (#421).  Decimal (§9.6) is the one stateful family: it keeps a value-typed Python store (decimal_store: dict[int, PyDecimal]).  register_decimal(linker, ops_used, decimal_store, host_store_refs) takes both the store and the GC-introspection registry as parameters.  Because the shared host_decref_handle GC hook (kept inline in execute(), kind=3 path) must close over the same store, _decimal_store creation is lifted out of the Decimal branch to execute() top-level (an empty dict when Decimal is unused) -- the inline hook is otherwise untouched, minimising risk to the #706 destructor path.  api.py 1,735->1,480 lines (-66% from 4,358).  mypy + ruff clean; 68 decimal codegen + 72 gate/closure tests pass, and ch09_decimal/ch09_decimal_generics run end-to-end including under VERA_EAGER_GC=1 (decref-eviction stress).

Co-Authored-By: Claude <noreply@anthropic.invalid>
Step 9 of the codegen/api.py decomposition (#421).  HTTP and Inference move out of execute() into vera/runtime/http.py (register_http, stateless) and vera/runtime/inference.py (register_inference(linker, ops_used, env_vars)).  The LLM provider registry (_ProviderConfig, _PROVIDERS) and HTTP call helper (_call_inference_provider) -- module-level in api.py and used only by inference -- relocate into inference.py, eliminating a would-be circular import.  The request timeout, shared by both net families, is renamed _INFERENCE_TIMEOUT->_HTTP_TIMEOUT and homed in http.py (the lower-level layer); inference.py imports it.  Provider-unit unit tests in test_codegen.py repoint their imports/mock.patch targets, and the #591 UTF-8-hygiene source-location tests in test_runtime_traps.py repoint to http.py/inference.py.  api.py 1,480->1,233 lines (-72% from 4,358).  mypy + ruff clean; 27 http/inference codegen + the #591 hygiene tests + the 22-test gate pass; ch09_http/ch09_inference check clean.

Co-Authored-By: Claude <noreply@anthropic.invalid>
Step 10 of the codegen/api.py decomposition (#421).  The State<T> effect (§9.4) moves out of execute() into vera/runtime/state.py via register_state(linker, state_types, initial_state, state_store).  Like _decimal_store, the per-type value-stack store (state_store) is lifted to execute() because ExecuteResult reads each cell's top-of-stack for its .state field; register_state populates the passed dict (registering get/set/push/pop host functions per state type and applying initial-state test overrides).  api.py 1,233->1,165 lines (-73% from 4,358).  mypy + ruff clean; 32 State codegen tests pass and ch07_state_handler runs end-to-end.

Co-Authored-By: Claude <noreply@anthropic.invalid>
Completes the codegen/api.py decomposition (#421) by documenting why IO is the one effect family that stays inline in execute() rather than moving to vera/runtime/.  IO is execute()'s observation channel -- its host callbacks write into state that becomes the return value (output_buf/stderr_buf -> ExecuteResult.stdout/stderr, last_violation -> _classify_trap, tee_stdout -> live-streaming) and it shares the _VeraExit Ctrl-C exception with execute()'s exit handling.  Extracting the twelve optional adapters reduced coupling; extracting IO would only relocate a cohesive unit behind a 7-field context object.  Recorded in four places: a heading comment on api.py's IO section, the vera/runtime/__init__ docstring, a new 'Host-binding families (vera/runtime/)' subsection in vera/README.md (the architecture source of truth), and the #421 CHANGELOG bullet.  Also retires the now-done api.py row from the KNOWN_ISSUES 'Refactoring needed' table.  Docs only; no behaviour change.

Co-Authored-By: Claude <noreply@anthropic.invalid>
@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@aallan, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 41 minutes and 51 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: bc6c079d-ae39-4004-958d-7ec8ce0b4d04

📥 Commits

Reviewing files that changed from the base of the PR and between 902b751 and f539a72.

📒 Files selected for processing (5)
  • CHANGELOG.md
  • KNOWN_ISSUES.md
  • ROADMAP.md
  • TESTING.md
  • tests/test_codegen.py
📝 Walkthrough

Walkthrough

Decomposes vera/codegen/api.py by extracting memory helpers into vera/codegen/memory.py and host/runtime code into vera/runtime/ modules. Tests, docs, and temp-file handling are updated to reference the new module layout and runtime paths.

Changes

Runtime extraction and rewiring

Layer / File(s) Summary
Memory helpers and import rewiring
vera/codegen/memory.py, vera/codegen/__init__.py, vera/codegen/closures.py, vera/codegen/core.py, vera/codegen/registration.py
Adds ConstructorLayout plus WASM layout helpers in a new module, then updates codegen imports to source those symbols from vera.codegen.memory.
Runtime traps, heap, and collection primitives
vera/runtime/__init__.py, vera/runtime/traps.py, vera/runtime/heap.py, vera/runtime/collections.py
Adds runtime package documentation, trap-frame classification, heap marshalling and wrapper helpers, and the shared value-type dispatch table.
HTTP and inference host bindings
vera/runtime/http.py, vera/runtime/inference.py
Adds HTTP scheme validation and request bindings, plus inference provider selection, request execution, and UTF-8 decoding/error handling.
JSON, HTML, Markdown, and regex bindings
vera/runtime/json.py, vera/runtime/html.py, vera/runtime/md.py, vera/runtime/regex.py
Adds parse/render/query/extract bindings for structured text effects and regex matching/replacement bindings.
Map and set host bindings
vera/runtime/map.py, vera/runtime/set.py
Adds wrapper-based map and set host bindings, including insert/get/remove/query, size, and array conversion operations.
Math, random, state, and decimal bindings
vera/runtime/math.py, vera/runtime/random.py, vera/runtime/state.py, vera/runtime/decimal.py
Adds scalar effect bindings for maths, randomness, per-type state stacks, and decimal arithmetic/conversion/rounding.
Tests, temp files, and project docs
.gitignore, scripts/check_html_examples.py, tests/test_html.py, tests/test_codegen.py, tests/test_runtime_traps.py, CHANGELOG.md, KNOWN_ISSUES.md, TESTING.md, vera/README.md, ROADMAP.md
Updates temporary .vera file handling, rewires tests to the new runtime modules, adds runtime import-hygiene coverage, and refreshes project documentation and status tables.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related issues

Possibly related PRs

  • aallan/vera#357: The HTTP effect bindings in this PR are extracted into vera/runtime/http.py, matching the HTTP effect surface introduced there.
  • aallan/vera#533: This PR moves WasmTrapError and trap classification into vera/runtime/traps.py, which overlaps with the trap-categorisation surface in that PR.
  • aallan/vera#546: TrapFrame, _resolve_trap_frames, and trap-frame reporting are part of the same backtrace machinery used in that PR.

Suggested labels

compiler, tests, ci, docs

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The PR also includes behaviour hardening and hygiene changes, such as HTTP scheme validation and math/shadow-guard fixes, beyond the decomposition scope. Keep this PR focused on the api.py split; move the behavioural fixes and other follow-up hardening into separate PRs.
Docstring Coverage ⚠️ Warning Docstring coverage is 54.71% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly states the main change: decomposing codegen/api.py into runtime package pieces for #421.
Linked Issues check ✅ Passed The PR extracts the memory helpers and runtime host layer, keeps the public compile/execute surface stable, and adds supporting tests/docs.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/421-runtime-decomposition

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.15680% with 50 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.88%. Comparing base (5452728) to head (f539a72).

Files with missing lines Patch % Lines
vera/runtime/heap.py 96.63% 14 Missing ⚠️
vera/runtime/map.py 92.56% 11 Missing ⚠️
vera/runtime/html.py 96.29% 5 Missing ⚠️
vera/runtime/regex.py 92.98% 4 Missing ⚠️
vera/codegen/memory.py 89.65% 3 Missing ⚠️
vera/runtime/http.py 91.66% 3 Missing ⚠️
vera/runtime/decimal.py 97.84% 2 Missing ⚠️
vera/runtime/inference.py 96.82% 2 Missing ⚠️
vera/runtime/math.py 91.66% 2 Missing ⚠️
vera/runtime/md.py 95.23% 2 Missing ⚠️
... and 1 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #789      +/-   ##
==========================================
+ Coverage   91.85%   91.88%   +0.02%     
==========================================
  Files          71       87      +16     
  Lines       26398    26493      +95     
  Branches      321      321              
==========================================
+ Hits        24249    24343      +94     
- Misses       2141     2142       +1     
  Partials        8        8              
Flag Coverage Δ
javascript 65.33% <ø> (ø)
python 94.96% <96.15%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
scripts/check_html_examples.py (1)

103-113: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Fix Windows-incompatible tempfile handoff in subprocess checks.

Line 103 and Line 124 keep the temporary .vera file open with delete=True while passing its path to subprocess.run(...). That is not portable on Windows and can fail to reopen the file. Use delete=False, close before subprocess invocation, and unlink in finally; also pass encoding="utf-8" explicitly.

Suggested patch
 def try_check(content: str, root: Path) -> str | None:
     """Try to type-check content. Returns error message or None."""
-    with tempfile.NamedTemporaryFile(
-        mode="w", suffix=".vera", delete=True
-    ) as f:
-        f.write(content)
-        f.flush()
+    f = tempfile.NamedTemporaryFile(
+        mode="w", suffix=".vera", delete=False, encoding="utf-8"
+    )
+    try:
+        f.write(content)
+        f.close()
         result = subprocess.run(
             [sys.executable, "-m", "vera.cli", "check", f.name],
             capture_output=True,
             text=True,
             cwd=str(root),
             timeout=30,
         )
         if "OK:" in result.stdout:
             return None
         # Return first line of stderr or stdout for diagnostics
         err = result.stderr.strip() or result.stdout.strip()
         return err.split("\n")[0][:200]
+    finally:
+        Path(f.name).unlink(missing_ok=True)
@@
 def try_verify(content: str, root: Path) -> str | None:
     """Try to verify contracts. Returns error message or None."""
-    with tempfile.NamedTemporaryFile(
-        mode="w", suffix=".vera", delete=True
-    ) as f:
-        f.write(content)
-        f.flush()
+    f = tempfile.NamedTemporaryFile(
+        mode="w", suffix=".vera", delete=False, encoding="utf-8"
+    )
+    try:
+        f.write(content)
+        f.close()
         result = subprocess.run(
             [sys.executable, "-m", "vera.cli", "verify", f.name],
             capture_output=True,
             text=True,
             cwd=str(root),
             timeout=60,
         )
         if "OK:" in result.stdout:
             return None
         err = result.stderr.strip() or result.stdout.strip()
         return err.split("\n")[0][:200]
+    finally:
+        Path(f.name).unlink(missing_ok=True)

As per coding guidelines, “In Python code, explicitly pass encoding="utf-8" to open(), read_text(), and write_text() calls,” and as per path instructions (TESTING.md), tempfile paths passed to subprocesses must use delete=False with manual unlink for Windows portability.

Also applies to: 124-134

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/check_html_examples.py` around lines 103 - 113, The tempfile handoff
in the subprocess checks is Windows-incompatible because the `.vera` file stays
open with `delete=True` while `subprocess.run(...)` tries to reopen it. Update
the tempfile usage in the check helper around the `subprocess.run` call to
create it with `delete=False`, close it before invoking `vera.cli check`, and
remove it in a `finally` block; also make sure the file write uses
`encoding="utf-8"` explicitly.

Sources: Coding guidelines, Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@vera/runtime/decimal.py`:
- Around line 89-92: The Decimal wrapper returned from _wrap_handle is being
held across _alloc_option_some_i32, which can allocate and leave the WASM heap
pointer unrooted under GC pressure. Update the decimal_from_string and
decimal_div paths to use the existing _ShadowGuard pattern so the wrapped
Decimal is rooted on the shadow stack before calling _alloc_option_some_i32,
then unroot it after the Option.Some allocation completes.

In `@vera/runtime/heap.py`:
- Around line 174-204: The heap shadow-stack push path in push() only guards sp
against limit, so it can still write a partial 4-byte slot when fewer than 4
bytes remain. Update the overflow check in Heap.push to reject any case where sp
+ 4 would exceed the shadow-stack limit before calling memory.data_ptr and
writing the packed pointer, preserving the existing RuntimeError diagnostic
shape for overflow.

In `@vera/runtime/http.py`:
- Around line 26-29: Validate the parsed URL scheme in the Http handling path
before calling urllib.request.urlopen so only http and https are allowed; reject
any other scheme (such as file) for both the GET and POST branches. Update the
logic around _read_wasm_string and the urlopen call site to parse/check the
scheme first, then proceed only when the scheme is permitted, keeping the
existing _HTTP_TIMEOUT behavior unchanged.

In `@vera/runtime/math.py`:
- Around line 21-43: The blanket ValueError-to-NaN handling in _math_unary_host
is too broad for logarithms. Update the host wrapper so math.log, math.log2, and
math.log10 preserve zero inputs as -Infinity while still translating
negative-domain ValueError cases to NaN. Use the existing _math_unary_host and
its inner host callback to special-case x == 0.0 for the log functions, keeping
parity with browser/IEEE semantics without changing other math.* behavior.

---

Outside diff comments:
In `@scripts/check_html_examples.py`:
- Around line 103-113: The tempfile handoff in the subprocess checks is
Windows-incompatible because the `.vera` file stays open with `delete=True`
while `subprocess.run(...)` tries to reopen it. Update the tempfile usage in the
check helper around the `subprocess.run` call to create it with `delete=False`,
close it before invoking `vera.cli check`, and remove it in a `finally` block;
also make sure the file write uses `encoding="utf-8"` explicitly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 2fa83f7b-4761-48ff-b0a9-16ce67428fe5

📥 Commits

Reviewing files that changed from the base of the PR and between 5452728 and e635061.

📒 Files selected for processing (31)
  • .gitignore
  • CHANGELOG.md
  • KNOWN_ISSUES.md
  • TESTING.md
  • scripts/check_html_examples.py
  • tests/test_codegen.py
  • tests/test_html.py
  • tests/test_runtime_traps.py
  • vera/README.md
  • vera/codegen/__init__.py
  • vera/codegen/api.py
  • vera/codegen/closures.py
  • vera/codegen/core.py
  • vera/codegen/memory.py
  • vera/codegen/registration.py
  • vera/runtime/__init__.py
  • vera/runtime/collections.py
  • vera/runtime/decimal.py
  • vera/runtime/heap.py
  • vera/runtime/html.py
  • vera/runtime/http.py
  • vera/runtime/inference.py
  • vera/runtime/json.py
  • vera/runtime/map.py
  • vera/runtime/math.py
  • vera/runtime/md.py
  • vera/runtime/random.py
  • vera/runtime/regex.py
  • vera/runtime/set.py
  • vera/runtime/state.py
  • vera/runtime/traps.py
💤 Files with no reviewable changes (2)
  • KNOWN_ISSUES.md
  • tests/test_html.py

Comment thread vera/runtime/decimal.py
Comment thread vera/runtime/heap.py
Comment thread vera/runtime/http.py
Comment thread vera/runtime/math.py
aallan and others added 2 commits June 24, 2026 13:03
…ounts

Addresses the PR #789 review (#421 decomposition).

Import cycle (pr-test-analyzer): _validate_wrap_handle was parked in the compile-time vera/codegen/memory.py, but it is a runtime heap concern (called only by heap.py's _wrap_handle).  That created vera/runtime/heap.py -> vera.codegen.memory -> codegen/__init__ -> api.py -> vera.runtime.decimal -> back to a partially-initialised heap for _WRAP_KIND_DECIMAL, so a cold 'import vera.runtime.<x>' (no prior vera.codegen import) raised ImportError -- undercutting the very modularity the decomposition adds.  Move _validate_wrap_handle to heap.py (its real home; no other runtime->codegen edge remains), repoint its 6 unit tests, and add TestRuntimePackageImportHygiene421 -- a fresh-interpreter cold-import of all 15 runtime submodules (RED before the move, green after).

State docs accuracy (comment-analyzer): State is the one family registered UNCONDITIONALLY (it loops over result.state_types) and it threads state_store back into ExecuteResult.state -- so the 'registered conditionally, owns its own state, feeds nothing back' generalisation was wrong.  Corrected the api.py IO-section comment, the vera/README.md 'family module' paragraph, and the state.py docstring (which had claimed state_store is 'created here and internal') to say: most families are stateless/conditional; the two stateful ones (Decimal, State) thread a single store passed as one parameter that execute() reads back; IO differs in DEGREE (seven interwoven pieces).

Line counts (comment-analyzer): the IO comment grew api.py to 1,182 (not 1,165); the move makes memory.py 67 and heap.py 1,021; runtime/ is 3,540.  Fixed in README + CHANGELOG.  TESTING.md/ROADMAP.md test totals bumped for the new test (4759->4760, test_runtime_traps 66->67).

Co-Authored-By: Claude <noreply@anthropic.invalid>
…789 CR)

CodeRabbit flagged try_check/try_verify handing a still-open delete=True NamedTemporaryFile to subprocess.run -- Windows can't reopen a held file.  Switch both to the delete=False + manual close()/unlink()-in-finally + encoding='utf-8' pattern already used in tests/test_html.py and documented in TESTING.md's Test Fixture Conventions.  Script-only; no behaviour change on POSIX.

Skip-changelog: CI-script Windows-portability fix, no vera/ or spec/ change

Co-Authored-By: Claude <noreply@anthropic.invalid>
@aallan

aallan commented Jun 24, 2026

Copy link
Copy Markdown
Owner Author

Review round — outcome

Internal (pr-review-toolkit). code-reviewer approved — a faithful, behavior-preserving extraction (the #734 characterization gate plus 1,277 codegen/runtime tests green; the byte-for-byte move audit came back clean). Two findings became fixes in 3d1379d:

  • Import cycle (pr-test-analyzer). _validate_wrap_handle had been parked in the compile-time codegen/memory.py, but it is a runtime heap concern (only heap.py's _wrap_handle calls it). That created heap -> codegen.memory -> codegen/__init__ -> api -> runtime.decimal -> heap, so a cold import vera.runtime.<x> (no prior vera.codegen import) raised ImportError — undercutting the modularity the decomposition adds. Moved it to heap.py (no other runtime -> codegen edge remains) and added TestRuntimePackageImportHygiene421, which cold-imports all 15 runtime submodules in fresh interpreters (RED before the move, green after).
  • State docs (comment-analyzer). Corrected the "registered conditionally / owns its own state / feeds nothing back" generalization — State registers unconditionally and threads state_store into ExecuteResult.state. Fixed the line counts (api.py is 1,182 after the IO comment).

CodeRabbit. The scripts/check_html_examples.py tempfile-portability finding is fixed in 03c83da (delete=False + manual close/unlink + encoding="utf-8", matching tests/test_html.py and TESTING.md). The four inline findings are replied to individually: decimal.py:92 is a false positive (_alloc_option_some_i32 already shadow-roots its payload); heap.py:204, http.py:29, and math.py:43 are pre-existing (verbatim in codegen/api.py on main) and out of scope for this behavior-preserving move — recommended as follow-ups:

  • Http accepts non-http(s) schemes (# noqa: S310 on main) — security.
  • Math.log(0) returns NaN vs JS/IEEE −∞ — parity.
  • shadow-stack push could use a stricter sp + 4 > limit — defensive.

@coderabbitai — thanks; the three pre-existing items are noted for a follow-up.

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

@aallan — clean resolution across the board. A few specific acknowledgements:

  • Import cycle — the diagnosis is tight: heap → codegen.memory → codegen/__init__ → api → runtime.decimal → heap is a genuine circular import that would surface only on cold import vera.runtime.* (no prior vera.codegen import), exactly the scenario the decomposition is meant to enable. Moving _validate_wrap_handle to heap.py is the correct fix — it is a runtime heap invariant, not a compile-time layout concern. The TestRuntimePackageImportHygiene421 gate in fresh interpreters is the right regression guard.

  • State unconditional registration — noted; the earlier generalisation ("registered conditionally") was incorrect. Threading state_store back through ExecuteResult.state is the correct characterisation.

  • decimal.py:92 false positive — withdrawn; _alloc_option_some_i32 does shadow-root its payload before the outer allocation, so there is no GC hazard at that call site.

For the three pre-existing follow-ups, would you like me to open tracking issues now? The three items would be:

  1. Securityvera/runtime/http.py: urlopen accepts non-http(s) schemes (mirrors # noqa: S310 suppression on main).
  2. Parityvera/runtime/math.py: math.log(0) returns NaN rather than the IEEE 754 / JS-spec −∞.
  3. Defensivevera/runtime/heap.py: shadow-stack push guard should use sp + 4 > limit rather than the current condition.

Happy to file them against the repo and link back to this PR if that would be useful.

…oadmap

Addresses both PR #789 follow-up requests.

Security (CodeRabbit, fixed now): Http.get/post validate the URL scheme and return Result.Err for file://, ftp://, data:, etc., instead of handing them to urllib.request.urlopen (which would read local files or speak arbitrary protocols).  New _is_allowed_http_url predicate gates both host callbacks; 2 tests (predicate unit + end-to-end file:// reject).  CHANGELOG gets a ### Security entry.  The #591 source-location guards in test_runtime_traps.py get a wider span=2500 window, since the scheme check grew host_http_post past the old 1500-char default and pushed its decode() call out of the grep window.

CodeRabbit follow-ups filed as issues + tracked in KNOWN_ISSUES Bugs: #790 (Math.log(0)/log2(0)/log10(0) return NaN vs IEEE-754/JS -inf -- pre-existing math wrapper) and #791 (_ShadowGuard.push partial-slot guard -- pre-existing, defensive).  decimal.py:92 was a verified false positive (no change).

Roadmap reprioritisation (per request): Tier 1 now leads with #387 finish the mutation sweep (whole-vera sweep unblocked by the #421 decomposition), and the two test-file splits are promoted into Tier 1 in order -- #419 (test_codegen, 21,084 lines) from Tier 3 and #420 (test_checker, 6,347) from Not-doing-now.  #421 removed from Tier 1 (closed by this PR); KNOWN_ISSUES refactoring-table line counts refreshed.

Co-Authored-By: Claude <noreply@anthropic.invalid>
@aallan

aallan commented Jun 24, 2026

Copy link
Copy Markdown
Owner Author

Follow-up round — Http security fix, tracked findings, roadmap

Landed in 902b751:

Http scheme validation (security, fixed now). Http.get/Http.post validate the URL scheme and return Result.Err for file://, ftp://, data:, and any other non-http(s) scheme, instead of handing them to urllib.request.urlopen. New _is_allowed_http_url predicate gates both callbacks; a predicate unit test + an end-to-end file://-reject test; CHANGELOG ### Security entry. (The #591 source-location guards in test_runtime_traps.py got a wider span=2500 window, since the scheme check pushed host_http_post's decode(...) past the old 1500-char default.)

Pre-existing findings filed + tracked. #790 (Math.log(0) / log2(0) / log10(0) return NaN where IEEE-754/JS give −∞) and #791 (shadow-stack push partial-slot guard) — both in KNOWN_ISSUES.md Bugs, labeled bug. decimal.py:92 stays a verified false positive (_alloc_option_some_i32 already shadow-roots its payload).

Roadmap. Tier 1 reprioritised: #387 finish the mutation sweep pulled to the top (whole-vera/ sweep now unblocked by this decomposition), then the two test-file splits #419 / #420 promoted into Tier 1 in order. #421 removed from Tier 1 (closed here).

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Line 17: Update the `#421` changelog entry to match the actual placement of
`_validate_wrap_handle`: the note currently says it moved to
`vera/codegen/memory.py`, but the final code/tests reference
`vera.runtime.heap`. Adjust the wording around the `codegen/api.py`
decomposition summary so the symbol location is accurate and consistent with the
rest of the runtime split described in the entry.

In `@TESTING.md`:
- Line 9: The test summary in the overview is mathematically inconsistent, so
update the aggregate count or one of the component counts to make the totals
match. Fix the breakdown in the TESTING.md overview row by verifying the numbers
for passed, stress, and skipped tests, and ensure the “Tests” total aligns with
the sum shown.

In `@tests/test_codegen.py`:
- Around line 12931-12947: The test in
test_http_get_rejects_file_scheme_end_to_end only asserts the Err result and
does not verify that Http.get short-circuits before urlopen is reached. Harden
this test by adding an assertion around the urlopen path in the existing test
body (or its surrounding harness) so the failure of the underlying network call
is explicitly proven, using the test_http_get_rejects_file_scheme_end_to_end and
Http.get symbols to locate the check. Keep the current Err-branch assertion, but
make the test fail if urlopen is invoked so regressions in the scheme guard are
caught reliably.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: a1be4198-8cf8-483e-bf16-b57c2d5c3572

📥 Commits

Reviewing files that changed from the base of the PR and between e635061 and 902b751.

📒 Files selected for processing (13)
  • CHANGELOG.md
  • KNOWN_ISSUES.md
  • ROADMAP.md
  • TESTING.md
  • scripts/check_html_examples.py
  • tests/test_codegen.py
  • tests/test_runtime_traps.py
  • vera/README.md
  • vera/codegen/api.py
  • vera/codegen/memory.py
  • vera/runtime/heap.py
  • vera/runtime/http.py
  • vera/runtime/state.py

Comment thread CHANGELOG.md Outdated
Comment thread TESTING.md Outdated
Comment thread tests/test_codegen.py
…ttp test

Three CodeRabbit inline findings on PR #789.

CHANGELOG: _validate_wrap_handle was listed under codegen/memory.py, but the import-cycle fix relocated it to runtime/heap.py.  Removed it from the memory.py group and named it in the heap.py marshalling group so the entry matches the final code.

TESTING.md: the overview breakdown was off by one (4,729 passed + 16 stress + 16 skipped = 4,761, not the 4,762 total) -- the #421 cold-import test bumped the total but not the passed count.  Fixed to 4,730 passed (4,730 + 16 + 16 = 4,762).

test_codegen.py: test_http_get_rejects_file_scheme_end_to_end now patches urllib.request.urlopen and asserts it is never called, so a removed scheme guard is caught even where file:///etc/passwd doesn't exist (Windows), where a failing urlopen would otherwise also yield Err and mask the regression.  The Err-branch assertion is kept.

Co-Authored-By: Claude <noreply@anthropic.invalid>
@aallan
aallan merged commit 5a14b5c into main Jun 24, 2026
27 checks passed
@aallan
aallan deleted the feat/421-runtime-decomposition branch June 24, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Decompose vera/codegen/api.py (2,228 lines): extract memory layout and execution runtime

1 participant