CI workflow, native-recompile back end (--emit-c), runtime library + SDL2

.github/workflows/ci.yml

@@ -0,0 +1,56 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - run: uv sync
+      - run: uv run ruff check
+      - run: uv run ty check
+      - run: uv run pytest
+
+  # the c0 native-recompile back end across platforms and compilers
+  # (graduation plan phase 4); the decoder suite is platform-independent
+  # and already covered by `test` above
+  c0:
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - os: ubuntu-latest
+            cc: gcc
+          - os: ubuntu-latest
+            cc: clang
+          - os: macos-latest
+            cc: clang
+          - os: windows-latest
+            cc: gcc
+    runs-on: ${{ matrix.os }}
+    # the Windows lane (MinGW-w64 gcc) is experimental until proven green
+    continue-on-error: ${{ matrix.os == 'windows-latest' }}
+    env:
+      CC: ${{ matrix.cc }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          python-version: "3.12"
+      - if: runner.os == 'Linux'
+        # SDL2 so the headless SDL backend test runs instead of skipping
+        run: sudo apt-get update && sudo apt-get install -y libsdl2-dev
+      - if: runner.os == 'macOS'
+        run: brew install sdl2
+      - run: uv sync
+      - run: uv run pytest tests/tbx/test_c0.py -q

CLAUDE.md

@@ -26,6 +26,17 @@ python tbx/tools/dump_user_code.py                    # tests/fixtures/usercode/
 uv run python tests/tbx/test_ir_snapshot.py --write   # tests/fixtures/ir_snapshot.txt
 ```
 
+With the external toolchain oracle available (`TBX_ORACLE`, see
+`tbx/tools/oracle.py` — a headless real-TB-compiler automation from a sister
+project; needs node + mtools):
+
+```sh
+uv run python -m tbx.tools.verify_fixture STEM        # byte-exact round trip
+uv run python -m tbx.tools.dump_dos_output --missing  # tests/fixtures/dosout/
+```
+
+Requires Python 3.11+. CI (`.github/workflows/ci.yml`) runs ruff, ty, and pytest on Python 3.11–3.13 for pushes to main and pull requests, plus a `c0` platform matrix (Linux gcc/clang with SDL2, macOS clang, experimental Windows MinGW) running `test_c0.py`; all must pass.
+
 The core package (`tbx.decode0`, `tbx.ir`, `tbx.emit0`, `tbx.cli`) has **zero runtime dependencies**; keep it that way. Only `tbx/tools/` may use iced-x86 (the `debug` extra), and `tests/tbx/test_cfg.py` guards it with `pytest.importorskip`.
 
 ## Architecture
@@ -37,25 +48,31 @@ The pipeline is EXE bytes → op stream → typed IR → canonical source:
 3. **Lift** (`decode0/core.py` + `decode0/handlers/`, `decode0/lift.py`, `decode0/select_case.py`) — `decode_user_code` runs a dispatch loop over the op stream. All loop state lives in `DecodeState` (`core.py`); handlers in `decode0/handlers/` (arith, control, dos_io, fileio, graphics) each consume ops and return `True`, and `select_case.py` is a state machine consulted at the top of each iteration. `lift.py` folds structured control flow (FOR/WHILE/DO, block IF, SUB/DEF FN) from its compiled shapes; `rename.py` renames variables to A, B, C… in first-store order.
 4. **Emit** (`emit0.py`) — typed IR → canonical source: one statement per line, numbered 10, 20, …. Line numbers are renumbered freely EXCEPT where byte-significant (error-trap line tables, TRON trace hooks), in which case the originals are recovered and preserved exactly.
 
+An alternative back end, `c0.py` (`tbx --emit-c`), lowers the same IR to a self-contained C translation unit for modern platforms (behavioral fidelity, not byte-exact; needs gcc/clang for labels-as-values GOSUB — POSIX or MinGW-w64 on Windows, the runtime carries `#ifdef _WIN32` paths). The C runtime lives in `tbx/c0_runtime/` as standalone units — each `.c` includes `tb_runtime.h` and compiles on its own; `make` there builds it as an ordinary library (`libtbrt.a`, `SDL=1` for the SDL backend) and `tbx --emit-c --no-runtime` emits programs that link against it instead of embedding it. By default `c0.py` amalgamates header + fragments in manifest order, stripping the include lines, into the self-contained output. A symbol goes in `tb_runtime.h` (non-`static`) only if another fragment or c0-generated code references it; fragment-internal helpers stay `static`. Machine access (PEEK/POKE/OUT/INP/WAIT/REG/BLOAD/BSAVE/DEF SEG/CHAIN) runs against the emulated real-mode machine in `machine.c` (documented surrogates, not DOS); `tbx --emit-c --sdl` swaps the PPM-at-exit graphics surrogate for a real SDL2 window (`sdl.c`, gated by `-DTB_SDL`, with the window keyboard feeding INKEY$/INSTAT). c0 is fail-loud like the decoder — unsupported constructs raise, never mistranslate — and `test_c0.py` pins recompiled-program stdout plus a corpus-coverage floor (currently 564/564 transpile). The byte-exact rules below do NOT govern c0; its standard is handbook semantics.
+
 `tbx/ir/` is the shared IR: immutable dataclasses, pure data, pattern-matched by analyses. `unparse(parse_expr(s)) == s` is a checked invariant (`test_ir.py`). Both `tbx.ir` and `tbx.decode0` re-export their submodules' surface through `__init__.py`, so callers use `ir.Foo` / `decode0.bar`.
 
+**The decoder→emitter contract**: `decode_user_code` returns `Program` (`decode0/meta.py`), a `list` subclass of IR statements carrying side-channel attributes that `emit0.emit` reads via `getattr` (so plain statement lists also emit, as tests exploit): `lines` (original line numbers when the error-trap line table makes them byte-significant; `None` = renumber freely), `metas` (`(stmt_index, text)` pairs for $STACK/$SOUND/$EVENT, emitted unnumbered before the indexed statement), `toggles` (IDE Options letters, reported out-of-band by the CLI), and the TRON trio `hook_seq`/`traced`/`trace_partial` (per-physical-line trace-hook numbering). Changes to any of these must keep both sides in sync.
+
 ## The calibration rule (most important convention)
 
-The decoder is **fail-loud**: any byte pattern outside the calibrated vocabulary raises `ValueError` (with offending byte and file offset) rather than guessing. A byte pattern joins the vocabulary only after a fixture program in `tests/fixtures/corpus/` witnesses it and its decompile-recompile round trip was verified byte-exact against the real Turbo Basic compilers. Do not add speculative decodings. Verifying *new* fixtures end-to-end requires the original DOS toolchain (under an emulator), which this repo does not include or automate — existing goldens encode past verifications.
+The decoder is **fail-loud**: any byte pattern outside the calibrated vocabulary raises `ValueError` (with offending byte and file offset) rather than guessing. A byte pattern joins the vocabulary only after a fixture program in `tests/fixtures/corpus/` witnesses it and its decompile-recompile round trip was verified byte-exact against the real Turbo Basic compilers. Do not add speculative decodings. Verifying *new* fixtures end-to-end requires the original DOS toolchain (under an emulator), which this repo does not include — on machines with the external oracle (`TBX_ORACLE`), `python -m tbx.tools.verify_fixture STEM` automates the check; existing goldens encode past verifications.
 
 Where the compiler is genuinely lossy, aliases are normalized to one canonical form that recompiles byte-identically (STOP/SYSTEM ≡ END, INCR x ≡ x = x + 1, DATA regrouped as one statement, pre-test WHILE ≡ DO WHILE…LOOP) — normalization is fine, guessing is not.
 
 IDE compiler toggles (Keyboard break, Bounds, Overflow, Stack test, 8087) have no source spelling and are deliberately **not** emitted (even as comments — comment text would perturb a runtime table under K/O). They ride on `Program.toggles`; the CLI reports them on stderr.
 
 ## Tests and fixtures
 
-Three regression layers, coarsest to finest:
+Regression layers, all swept on every pytest run:
 
-- `tests/fixtures/usercode/*.bas` — golden emitted source per corpus EXE (checked via CLI tests and used to select the snapshot sweep).
-- `tests/fixtures/ir_snapshot.txt` — one `repr()` per IR statement for every corpus EXE that has a usercode golden; drift fails here first with the exact program and line (`test_ir_snapshot.py`).
-- `tests/fixtures/ops/*.txt` — canonical op-stream dumps.
+- `tests/fixtures/ops/*.txt` — canonical op-stream dump per corpus EXE, gated by `test_goldens.py` (which reuses `dump_ops.canon`, so tool and test can't drift apart). Scan-level drift fails here.
+- `tests/fixtures/ir_snapshot.txt` — one `repr()` per IR statement for every corpus EXE that has a usercode golden; decoder drift fails with the exact program and statement line (`test_ir_snapshot.py`).
+- `tests/fixtures/usercode/*.bas` — golden emitted source, swept by `test_goldens.py`. Emit-level drift fails here.
+- `tests/fixtures/dosout/*.txt` (+ `<stem>.file.<NAME>` for files the program wrote) — what the ORIGINAL corpus EXE visibly did running on the real (emulated) machine, captured once by `dump_dos_output.py`. `test_c0.py::test_dos_golden` holds the recompiled native binaries to them — the c0 analog of the byte-exact rule. Waivers in `test_c0.py` name the surrogate that justifies each divergence.
+- Hand-written per-feature tests in `tests/tbx/` that pin exact IR for specific fixtures — the strongest guard, since goldens can be regenerated but pinned IR must be edited deliberately.
 
-Plus hand-written per-feature tests in `tests/tbx/` that pin exact IR for specific fixtures.
+Golden regeneration is only for **intended** changes; review the git diff it produces as carefully as code. `dump_user_code.py` deliberately skips flag fixtures (non-empty `Program.toggles`) — they carry no `.bas` golden because their source is identical to the unflagged program's, and `test_goldens.py` enforces that absence.
 
 Corpus naming (`tests/fixtures/corpus/`): `.exe` files are compiled fixtures, `.bas` alongside them are the authored originals. Prefixes: plain `t1_`/`tier*`/`zz_` = TB 1.1; `v10_` = the same program compiled with TB 1.0 (dialect tests assert identical IR across both); `f<code>_` = compiled with one IDE Options toggle ON (fkb=Keyboard break, fbd=Bounds, fov=Overflow, fst=Stack test, f87=8087) — these carry no `.bas` golden so the sweep skips them, and `test_flags.py` pins them directly.
 

README.md

@@ -1,5 +1,7 @@
 # tbx — a byte-exact decompiler for Borland Turbo Basic 1.0/1.1
 
+[![CI](https://github.com/leag/tbx/actions/workflows/ci.yml/badge.svg)](https://github.com/leag/tbx/actions/workflows/ci.yml)
+
 `tbx` recovers Turbo Basic source from 16-bit MS-DOS EXEs compiled by Borland
 Turbo Basic 1.0 or 1.1. Its correctness standard is unusual for a decompiler:
 a recovery counts only if **recompiling the emitted source in the original
@@ -104,6 +106,85 @@ the real Turbo Basic 1.0/1.1 compilers when it was added; validating new
 recoveries end-to-end requires access to the original DOS toolchain (e.g.
 under an emulator), which this repository does not include or automate.
 
+## Native recompilation
+
+Beyond source recovery, `tbx` can recompile a decoded program for modern
+platforms by emitting a self-contained C translation unit:
+
+```
+tbx PROGRAM.EXE --emit-c -o program.c
+cc program.c -lm -o program        # gcc or clang (labels-as-values is used)
+```
+
+The output compiles unchanged on Windows with MinGW-w64 gcc or clang (not
+MSVC) — the runtime carries `#ifdef _WIN32` paths for its keyboard, console,
+directory, and timing pieces (exercised under Wine).
+
+The generated file embeds a runtime that follows Turbo Basic semantics:
+GW-BASIC-style PRINT and PRINT USING layout with per-channel TAB/SPC columns,
+CINT banker's rounding, 16-bit integer operators, single-precision default
+variables, DATA/READ/RESTORE, sequential and random-access file I/O
+(FIELD/LSET/RSET/GET/PUT, MKx$/CVx), error trapping (ON ERROR GOTO / ERR /
+ERL / RESUME; untrapped errors abort with TB's code and line), SUB/CALL with
+by-reference parameters, multi-line DEF FN, ON TIMER traps polled at statement
+boundaries, and graphics (PSET/LINE/CIRCLE/PAINT/GET/PUT/VIEW/WINDOW/DRAW/
+POINT/PMAP/PALETTE) rendered into an in-memory CGA/EGA framebuffer, and PLAY
+decoded to audio. Devices with no modern counterpart are rendered to files
+instead of replicated: `TB_SCREEN_PPM=out.ppm` dumps the final framebuffer
+image and `TB_PLAY_WAV=out.wav` dumps the PLAY audio (mono 16-bit PCM), each
+at exit. These file surrogates are gated at compile time — build with
+`-DTB_FILE_DEVICES=0` to omit them and leave the devices absent (silent PLAY,
+no screen dump). Terminal statements map to ANSI escapes; the remaining
+device statements (KEY LIST, SOUND, WIDTH) are no-ops, and device functions
+read as absent (STICK/STRIG/PEN = 0).
+
+Machine access runs against an emulated real-mode machine rather than being
+rejected: PEEK/POKE/DEF SEG/BLOAD/BSAVE address a private zero-filled 1 MiB
+memory image (POKEd values PEEK back; BSAVE/BLOAD round-trip through files
+with the real 7-byte header), OUT/INP are 64 K one-byte port latches, WAIT
+returns at once (the absent device is treated as ready), REG is real storage
+behind a no-op CALL INTERRUPT, CHAIN execs the named file from the working
+directory (so a chained program recompiled alongside works, TB error 53 if
+absent), and CALL ABSOLUTE compiles but aborts if reached — there is no
+machine code to run. These are documented surrogates, not DOS: a program
+that PEEKs a BIOS structure at a magic address reads 0.
+
+Fidelity is behavioral, not byte-exact, and the back end stays fail-loud:
+a construct outside the implemented vocabulary raises instead of
+mistranslating. All 564 fixture-corpus programs recompile and run natively
+today.
+
+### Reusing the runtime
+
+The C runtime lives in `tbx/c0_runtime/` as standalone units — each `.c`
+compiles on its own against `tb_runtime.h`, so it also builds as an ordinary
+static library. `--no-runtime` then emits programs that `#include` the
+header instead of embedding the runtime, letting many recompiled programs
+share one build of it:
+
+```
+make -C tbx/c0_runtime                              # libtbrt.a
+tbx PROGRAM.EXE --emit-c --no-runtime -o program.c
+cc program.c -Itbx/c0_runtime -Ltbx/c0_runtime -ltbrt -lm -o program
+```
+
+### SDL2 graphics
+
+With SDL2 installed, `--sdl` presents graphics in a real window instead of
+the in-memory framebuffer's PPM-at-exit surrogate:
+
+```
+tbx PROGRAM.EXE --emit-c --sdl -o program.c
+cc program.c -lm $(sdl2-config --cflags --libs) -o program
+```
+
+The window is 4:3, like the CRTs these modes were designed for, so 640×200
+keeps its authentic non-square pixels. Keys typed into the window feed
+INKEY$/INSTAT, making interactive graphics programs playable; when the
+program ends, the final image stays up until a key or the close button
+(set `TB_SDL_HOLD=0` to skip). For the library build, use
+`make -C tbx/c0_runtime SDL=1` and compile programs with `-DTB_SDL=1`.
+
 ## Debugging tools
 
 The decoder fails loudly by design: an unrecognized compiler template raises