Runtime metrics for generated executors (design doc 18)

[moznion]

Implements design doc 18 — runtime metrics for generated executors (included as the first commit, D1–D8 settled with all recommendations adopted): operators can now answer, from a metrics endpoint, how the composed-SQL cache performs, which shapes traffic actually uses vs. what was verified, how long calls take, and what gets refused before any SQL is sent.

Layers

1. runtime/ — neutral observation surface (no new dependencies)

   • Observer interface: ObserveCompose (from the cache, hit/miss + the retained ShapeKey), ObserveExec (ctx, canonical key encoding, duration, rows, error), ObserveReject (the sentinel refusals).
   • ComposedCache.Stats() (hits/misses/inserts/evictions/entries/capacity/SQLBytes) and TopShapes(n) — scrape-time snapshots under the mutex, deterministic ordering.
   • ShapeSpaceInfo — the shared struct behind the generated registry.

2. codegen — generated packages wire it up

   • Queries.SetObserver (travels through WithTx, shared with the cache) + Cache() accessor.
   • Every method reports rejects on its validation branches and one exec event per DB call (guarded clock; per-annotation row counts; @filter-tree routes through observeExecTree so the call site never encodes the tree twice).
   • db.gen.go carries a ShapeSpace registry computed from shape.Count at generate time: enumerable count (uint64-saturating, Exact flag) + Unbounded flag (@filter-tree everywhere; @in arity on expanding dialects). Examples regenerated for all three dialects.

3. contrib/otel — OpenTelemetry adapter, separate go.mod

   • Package otelsqletch: the doc-18 §6 instrument table, per-event series labeled by query/instance only, bounded exact used-shape tracking with a loud saturated flip (D5), scrape callbacks over Stats/TopShapes/ShapeSpace, and a TraceObserver decorator that puts full-cardinality shape keys where they belong — on spans, never on metric labels (§7, asserted negatively in tests).
   • OpenMetrics/Prometheus exposition comes from the standard OTel Prometheus exporter; the core module never sees the OTel graph.

Performance findings (measured, pinned in doc 18 §4/D4 and by benchmarks)

• The ungated per-entry hit counter cost the parallel hit path ~2.5× (12.8→34 ns/op) — the D4 escalation clause fired: counting now gates behind a sticky flag set at first SetObserver/Stats/TopShapes. Unobserved traffic measures at baseline (12.0–12.3 ns/op, 1 alloc); BenchmarkGeneratedCallParallelObserved pins the metrics-on worst case (~130 ns/op).
• Passing the caller's ShapeKey through the observer interface heap-allocated every call site's key slices (+1 alloc/op, observed or not). Fix: the cache passes its retained key (proven identical by keysEqual), and ObserveExec takes the canonical encoding, built inside the observer guard.

Testing

• runtime/: scripted CLOCK-eviction stats pins, hits-survive-eviction fold, all three serve paths' hit bits, concurrent same-shape coherence, allocation-free hit path with and without an observer (AllocsPerRun), stats-under-churn cross-checks, race-detector scrape-during-churn.
• codegen: observer-site and ShapeSpace golden assertions incl. uint64 saturation via a 17-key @order-by; TestComposeConformance passes unmodified.
• contrib/otel: SDK manual-reader tests for every instrument, the no-shape-key-label rule, saturation flip, TraceObserver span attributes/events/forwarding.
• E2E (devdb, all three dialects, run locally — full suite green): the generated-module fixtures install a recording observer and pin exec/compose 1:1, deliberate-misuse reject counts, cache-stats coherence, and the ShapeSpace flags.
• golangci-lint --build-tags devdb, standalone staticcheck (main + contrib), goimports: clean. Fuzz smoke on both targets: pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01U8QzqDQ6XJpN3hbD7y8nzi