[WAF-08] Implement collection variables, selectors, origins, and macros

[chrisbbreuer]

Objective

Implement ARGS families, headers, cookies, files, multipart, XML, GEO, RULE, ENV, TX, persistent-collection namespace interfaces, key and regex selectors, exclusions, counts, origins, and runtime macro expansion. Cross-request persistence, expiry, and synchronization for IP/SESSION/USER/GLOBAL/RESOURCE are implemented by WAF-09 against these interfaces.

Deliverables

• Implement bounded owned collection storage with repeated and map mutation semantics.
• Preserve source origins and offsets and enforce minimum phase availability.
• Implement exact and production zig-regex key selectors, exclusions, and count targets.
• Compile and expand bounded runtime macros with explicit baseline policy.
• Publish atomic derived ARGS, cookie, header, match, and file metadata families for downstream parsers.
• Keep request-path work independent from UI, PostgreSQL, and blocking audit I/O.
• Use Zig 0.17-dev and dependencies pinned through Pantry; do not add Git submodules.

Acceptance criteria

• The scoped implementation and tests are merged on main in small conventional commits.
• Zig formatting, focused tests, and the relevant CI suite pass.
• Compatibility claims have executable evidence and no unexplained skips or silent fallbacks.
• Untrusted inputs have deterministic entry, key, value, token, output, and physical-allocation bounds.
• Selector exhaustion is an error, never a false non-match.
• Documentation, dedicated benchmark, and machine-readable compatibility matrix are updated.

Relationships

• Parent: WAF-00

Blocked by

• WAF-06
• WAF-07

Blocks

• WAF-09 — persistent backends, expiry, synchronization
• WAF-23 — request target/header/cookie parsing
• WAF-25 — multipart and file producers
• WAF-26 — URL-encoded/JSON body producers
• WAF-27 — XML producers

Baselines

• ModSecurity 3.0.16
• Coraza 3.7.0
• OWASP CRS 4.28.0
• Zig 0.17-dev resolved and locked through Pantry

[chrisbbreuer]

Unblocked: WAF-07 is complete and closed in #8. The executable scalar registry, origin metadata, phase availability, bounded storage, and collection-producer boundary are now available on main at b59955e.

[chrisbbreuer]

Progress: pushed b0ac68c (feat(engine): add bounded collection storage) directly to main.

This establishes the 37-namespace collection registry, per-namespace key policy, repeated-value insertion order, owned keys/values, exact source offsets/origins, deterministic entry/key/value/aggregate limits, allocation-free exact/all selection, and a matcher callback contract for compiled regex selectors. Focused tests cover registry uniqueness, repeated headers, case semantics, source preservation, matcher scans, and pre-ownership limit failures.

WAF-09 remains the owner of persistence and expiry behavior for IP/SESSION/USER/GLOBAL/RESOURCE; WAF-08 defines those namespaces and the transaction-facing contract they plug into.

[chrisbbreuer]

Progress: pushed HEAD (feat(engine): compile bounded runtime macros) directly to main. Runtime strings now compile once into immutable literal/scalar/collection tokens, validate variable names and hostile syntax, expand with explicit input/token/output bounds, and expose the ModSecurity-vs-Coraza missing-value difference as a configured policy rather than a silent behavior change. Tests cover scalar, keyed collection, first-value, malformed input, unknown names, and output exhaustion.

[chrisbbreuer]

Progress correction and integration:

• The runtime macro commit referenced as HEAD in my prior comment is 29e544d.
• Pushed 2a0a66e (feat(engine): wire transaction collections) to main.

Transactions now publish repeated request/response headers and name collections with exact offsets, retain match history, expose phase-gated collection iterators, allow bounded subsystem publication, and resolve compiled macros against live scalar/collection state. Paired collection entries are limit-atomic.

The initial per-value allocation design measured ~56 us/lifecycle and was rejected before commit. Transaction-local arena ownership reduced the same populated ReleaseFast benchmark to ~7.56 us (7,632-byte transaction) versus ~5.18 us before collection publication. This is current tracking evidence; WAF-39 still owns comparative release gates.