docs: ESP32-S3 micro-LLM deep research + ADR-324/325

docs/adr/ADR-324-esp32-micro-llm-research-spike.md

@@ -0,0 +1,111 @@
+# ADR-324: ESP32-S3 micro-LLM inference — bounded research spike, production non-goal
+
+- **Status**: Proposed
+- **Date**: 2026-08-16
+- **Deciders**: ruv
+- **Owners**: RuView firmware and edge runtime maintainers
+- **Tags**: esp32, llm, edge, research, evidence-labeling, firmware
+- **Numbering note**: ADR-324 is the next free number in the authoring
+  checkout (highest existing: ADR-323; ADR-322 is an observed gap). Re-run the
+  ADR index/collision check immediately before merge and rename if needed.
+- **Extends**: ADR-028, ADR-039, ADR-040, ADR-045, ADR-110, ADR-175, ADR-304
+- **Supersedes**: None
+- **Companion research**: `docs/research/esp32-micro-llm-inference.md`
+- **Companion ADR**: ADR-325 (flash-resident weight streaming)
+
+## Executive decision
+
+RuView will treat on-device micro-LLM inference on ESP32-S3 as a **bounded,
+optional research spike on dedicated companion hardware**, and explicitly a
+**non-goal for production CSI node firmware**. No LLM code, model weights, or
+partition changes enter `firmware/esp32-csi-node/` under this ADR. Every
+externally sourced performance number is `CLAIMED` until reproduced on RuView
+silicon with a captured witness log.
+
+## Context
+
+Public demonstrations in 2026 (slvDev's `esp32-ai`, building on
+`karpathy/llama2.c` and `DaveBben/esp32-llm`) show a 28.9 M-parameter
+TinyStories transformer running at 9.5–9.88 tok/s (`CLAIMED`) on an ESP32-S3
+with 512 KB SRAM, 8 MB PSRAM, and 16 MB flash, using 4-bit quantization and
+Gemma-3n-style Per-Layer Embeddings memory-mapped from flash. The companion
+research document analyzes the techniques and coverage in depth.
+
+The question for RuView is whether any of this belongs on our sensing nodes.
+The material facts are:
+
+1. **It does not fit our fleet.** Production nodes use the ADR-045 8 MB
+   partition map (2× 2 MB OTA + 1.875 MB SPIFFS) or the 4 MB variant. The
+   14.9 MB model artifact cannot be stored on either (`SYNTHETIC`, partition
+   arithmetic). Replication requires a 16 MB-flash S3 SKU we do not deploy.
+2. **It contends with the product.** The demos own both cores and run no
+   radio traffic during inference. RuView nodes dedicate core 0 to
+   WiFi + 20 Hz CSI capture and core 1 to the Tier 1–2 DSP pipeline and WASM
+   modules (ADR-039). LLM inference would degrade the primary sensing path.
+3. **The model class has no task capability.** TinyStories-scale models
+   generate short fiction only — no instruction following, no factual recall.
+   Every on-node use case we examined is served better by deterministic
+   templates or is blocked on datasets that do not exist.
+4. **The memory techniques are genuinely valuable** — for RuView's own edge
+   models, which is split out as ADR-325 so its fate is independent of the
+   LLM demo's.
+
+The rejected premise is: "an LLM now runs on our chip, therefore our product
+should run an LLM." Feasibility of a demo is not fitness for a sensing
+appliance.
+
+## Decision
+
+1. **Production non-goal.** `firmware/esp32-csi-node/` gains no LLM inference
+   path, no model partitions, and no LLM-derived output fields. Any future
+   reversal requires a new ADR with `MEASURED` evidence of zero CSI-path
+   regression.
+2. **Bounded research spike (optional, unscheduled).** If funded, the spike
+   runs on a dedicated ESP32-S3 N16R8-class devkit that performs no CSI
+   capture, following the validation plan in the research document:
+   unmodified `esp32-ai` reproduction with witness log; WiFi-coexistence
+   penalty measurement; a ≤3.5 M-parameter fit-our-flash retrain solely to
+   characterize the quality cliff.
+3. **Evidence labeling.** All numbers from external sources remain `CLAIMED`
+   in every RuView document, README, and communication until step 2 produces
+   `MEASURED` rows backed by a committed witness log (WITNESS-LOG discipline
+   per ADR-028). Builds and simulators are not hardware evidence.
+4. **Trust boundaries.** Third-party inference code and weights undergo
+   license review at intake; model weights are never committed to this
+   repository; fetched artifacts are SHA-256 pinned. Micro-LLM output is
+   never admitted into the evidence engine (ADR-304), never labeled as
+   perception, and never enters a safety-relevant path (fall/presence
+   alerts).
+5. **ESP32-C6 exclusion.** The spike targets S3 only; the single-core RISC-V
+   C6 without PIE SIMD is out of scope (ADR-110 unaffected).
+
+## Options considered
+
+- **Do nothing.** Cheapest, but discards transferable memory techniques and
+  leaves recurring "can we run an LLM on the nodes?" questions undocumented.
+  Rejected in favor of a written boundary.
+- **Integrate into production firmware as a Tier 3.** Rejected: flash
+  impossibility on deployed SKUs, core/PSRAM contention with sensing,
+  no task capability at this model scale, safety-path hallucination risk.
+- **Companion-hardware research spike (chosen).** Isolates risk, produces
+  `MEASURED` data, feeds ADR-325.
+- **Server-side LLM only.** Remains the correct place for any real language
+  capability; this ADR does not change server architecture.
+
+## Consequences
+
+- Contributors get a citable "no" for production LLM integration and a
+  citable "yes, like this" for research.
+- The spike, if run, costs one devkit and bounded engineering time, and
+  produces the repository's first `MEASURED` micro-LLM data.
+- Risk accepted: the field moves fast; this ADR may need revisiting if
+  instruction-capable models reach ~10 M parameters. The evidence-labeling
+  and non-contention requirements would still hold.
+
+## References
+
+- `docs/research/esp32-micro-llm-inference.md` (sources, techniques, fit
+  analysis)
+- ADR-028 (capability audit / witness discipline), ADR-039 (edge tiers),
+  ADR-045 (partition map), ADR-175 (measured quantization precedent),
+  ADR-304 (evidence engine), ADR-325 (companion decision)

docs/adr/ADR-325-flash-resident-edge-model-weight-streaming.md

@@ -0,0 +1,111 @@
+# ADR-325: Flash-resident weight streaming for RuView edge models
+
+- **Status**: Proposed
+- **Date**: 2026-08-16
+- **Deciders**: ruv
+- **Owners**: RuView firmware and edge runtime maintainers
+- **Tags**: esp32, memory, quantization, edge, inference, firmware, wasm
+- **Numbering note**: ADR-325 follows ADR-324 in the authoring checkout.
+  Re-run the ADR index/collision check immediately before merge and rename if
+  needed.
+- **Extends**: ADR-028, ADR-039, ADR-040, ADR-045, ADR-102, ADR-163, ADR-175
+- **Supersedes**: None
+- **Companion research**: `docs/research/esp32-micro-llm-inference.md`
+- **Companion ADR**: ADR-324 (micro-LLM research spike)
+
+## Executive decision
+
+RuView adopts a **design investigation** into flash-resident, memory-mapped,
+quantized weight storage for its **own** edge inference models on ESP32-S3 —
+the durable technique behind the 2026 micro-LLM demos — decoupled from any
+language-model ambition. The target outcome is that future neural
+presence/pose/vitals edge models are bounded by *flash* capacity rather than
+by the ~7.36 MB of free PSRAM or ~280 KB of free SRAM, on the existing 8 MB
+fleet SKU. This ADR authorizes design and benchmarking only; any partition
+map or wire change requires its own follow-up ADR with `MEASURED` evidence.
+
+## Context
+
+Today RuView's on-node intelligence is DSP (ADR-039 Tiers 1–2) plus WASM3
+modules in 640 KB of PSRAM arenas (ADR-040/041), with neural inference done
+server-side. If RuView ever ships a neural edge model (e.g. an on-node
+presence classifier to cut the Tier 2 heuristic's false-positive rate), the
+naive approach loads all weights into PSRAM/SRAM, capping model size and
+competing with WASM arenas and CSI buffers.
+
+The `esp32-ai` demonstration (`CLAIMED`, see companion research) validated a
+different memory contract on identical silicon: keep only activations and hot
+weights in SRAM, the compute-dense core in PSRAM, and the bulk of parameters
+4-bit-quantized in flash, accessed via the ESP32-S3 MMU's memory-mapped
+read path with sparse, cache-friendly gathers (~450 bytes/token in the demo).
+The same contract applies to non-LLM models: embedding/lookup layers,
+frozen feature banks, and per-domain calibration tables are all
+sparse-access, read-only structures.
+
+RuView precedents: ADR-175 established measured INT8 quantization discipline;
+ADR-163 established edge latency measurement; ADR-028 established witness
+evidence. This ADR composes them.
+
+## Decision
+
+1. **Adopt the memory contract as a design target** for future RuView edge
+   models on ESP32-S3: read-only quantized weight banks live in a dedicated
+   flash region and are memory-mapped, not copied; PSRAM holds compute-dense
+   weights and scratch; SRAM holds activations. INT8 remains the default
+   quantization (ADR-175 precedent); sub-byte formats require their own
+   measured accuracy gate.
+2. **Fit the fleet, not the demo.** All designs must fit the ADR-045 8 MB
+   partition map's current free space (1.875 MB SPIFFS region or a future
+   dedicated `model` data partition of comparable size). Designs requiring a
+   16 MB SKU are research-only under ADR-324.
+3. **Benchmark before build.** The first deliverable is a microbenchmark
+   suite on real S3 silicon measuring: memory-mapped flash gather latency and
+   bandwidth (sequential vs. strided), ESP-DSP SIMD matmul throughput
+   against PSRAM- vs. flash-resident operands, and CSI-pipeline interference
+   (CSI packets/s at 20 Hz with the benchmark running on core 1). Results
+   are committed as `MEASURED` with witness logs; until then, all sizing in
+   this ADR is `SYNTHETIC`/`CLAIMED`.
+4. **Integrity and provenance.** Any model bank flashed to a node carries a
+   SHA-256 recorded in NVS and reported in the node hello; unsigned or
+   mismatched banks are rejected at mount time. Model artifacts are never
+   committed to this repository; distribution follows the edge module
+   registry path (ADR-102) with the same review gates as WASM modules.
+5. **Non-goals.** No production partition change, no wire-format change, no
+   model training, and no LLM integration under this ADR. WASM-visible
+   weight access (a `model_read` host API) is deferred until the benchmark
+   shows the flash path sustains it alongside CSI capture.
+
+## Options considered
+
+- **Status quo (all weights in RAM).** Simple, but caps any future edge model
+  at well under the free PSRAM after WASM arenas, and couples model growth to
+  RAM contention with the sensing pipeline. Rejected as the default.
+- **Flash-resident streaming (chosen for investigation).** Raises the
+  capacity ceiling ~an order of magnitude on existing SKUs; costs MMU window
+  management and benchmark-verified bandwidth budgeting.
+- **16 MB SKU migration.** Hardware cost and fleet churn for an unproven
+  need; remains available later and is orthogonal to this technique.
+- **Keep all neural inference server-side.** Remains the default for heavy
+  models; this ADR only lowers the barrier for small on-node models where
+  latency/privacy justify them.
+
+## Consequences
+
+- Positive: future edge models are flash-bounded (~1.8 MB usable today,
+  `SYNTHETIC`) instead of RAM-bounded; techniques arrive pre-validated by a
+  measured benchmark rather than by external claims; provenance gating
+  extends the existing module-registry trust model to weights.
+- Negative/risks: flash read contention with OTA and SPIFFS logging must be
+  measured, not assumed; MMU mapping bugs are a new failure class at the
+  hardware boundary (input validation at the mount path is mandatory);
+  benchmark effort is spent even if no neural edge model ever ships.
+- The ADR-324 spike and this benchmark share kernels; either can proceed
+  without the other.
+
+## References
+
+- `docs/research/esp32-micro-llm-inference.md` (technique analysis, sources)
+- ADR-028 (witness evidence), ADR-039 (edge tiers and core budget), ADR-040/
+  ADR-041 (WASM sensing), ADR-045 (partition map), ADR-102 (edge module
+  registry), ADR-163 (edge latency measurement), ADR-175 (measured INT8
+  quantization)