[WAF-10] Implement the SecLang lexer, parser, diagnostics, and includes

[chrisbbreuer]

Objective

Implement a bounded, source-preserving SecLang frontend compatible with the stable syntax accepted by ModSecurity 3.0.16 and Coraza 3.7.0. The frontend must lex logical lines, parse directives and rule structure, resolve local include trees safely, reject trailing or malformed input, and emit actionable diagnostics without performing semantic directive compilation owned by WAF-11/WAF-12.

Compatibility contracts

• Accept UTF-8 configuration bytes while treating SecLang structural punctuation and whitespace according to the pinned baselines.
• Normalize LF and CRLF line endings; preserve byte offsets and physical line/column locations.
• Join backslash continuations only where baseline syntax permits and retain the complete logical-to-physical source map.
• Recognize comments only outside quoted/escaped content.
• Support single-quoted, double-quoted, and unquoted directive arguments with compatible escaping and empty-value behavior.
• Parse directive names case-insensitively while preserving original spelling and source slices for formatting/diagnostics.
• Parse SecRule targets, target negation/count markers, selectors, operators, operator negation, quoted parameters, and the complete comma-delimited action field without losing escapes or macro text.
• Parse SecAction, SecDefaultAction, marker, update, removal, engine, body, audit, debug, collection, and other directives into a generic lossless directive AST; WAF-12 owns semantic support validation.
• Reject non-whitespace trailing bytes, unterminated quotes, dangling escapes/continuations, missing rule operands, empty directive names, malformed action separators, and invalid include forms.
• Expand local Include and IncludeOptional trees deterministically. Authenticated remote rules remain WAF-13.

Deliverables

• src/seclang/source.zig: owned source registry, stable source IDs, byte/line lookup, and include ancestry.
• src/seclang/lexer.zig: bounded logical-line and token lexer with quote/escape/comment state.
• src/seclang/parser.zig: lossless directive and rule AST plus strict full-input parser.
• src/seclang/include.zig: root-confined local resolver with deterministic glob ordering and cycle detection.
• Structured diagnostics with code, severity, primary span, optional secondary/include spans, and bounded human rendering.
• Public parse APIs for strings, files, and include trees with explicit allocator ownership.
• Focused baseline fixtures, malformed-input tables, include integration tests, fuzz entry points, and parser throughput/allocation benchmarks.
• Documentation and machine-readable compatibility evidence.

Interfaces

Parser.parseSource consumes one owned source and returns an owned Document containing directives and diagnostics. IncludeResolver.parseTree receives an explicit configuration root and entry path, registers each canonical source once, and returns documents in deterministic expansion order. AST nodes reference stable SourceId plus half-open byte spans; no node borrows caller buffers.

The parser produces syntax, not a partially validated WAF. Semantic compilation must consume a completely parsed document set and may publish only if every required directive and feature validates.

Resource and security limits

• Maximum bytes per source, logical line, token, quoted value, and diagnostic.
• Maximum tokens/directives/diagnostics per source.
• Maximum include depth, source count, aggregate bytes, and glob matches.
• Canonical root confinement; reject absolute paths when policy forbids them, .. escapes, symlink escapes, non-regular files, cycles, and duplicate canonical sources.
• Deterministic lexical glob ordering independent of filesystem enumeration.
• No recursive parser call proportional to attacker-controlled rule/action count.
• Failure occurs before unbounded ownership; resource-limit errors remain distinct from syntax and I/O errors.

Acceptance criteria

• Representative pinned ModSecurity and Coraza configuration fixtures produce the expected lossless AST and spans.
• Every CRS 4.28 configuration file is lexed and parsed; unsupported semantics may be reported only by later compilation, not misparsed here.
• Line continuation, quoting, escaping, comments, macros, regex delimiters, action commas, and target selectors have differential fixtures.
• Include ordering, optional-missing behavior, cycle detection, canonical deduplication, and root/symlink escape rejection are executable.
• Malformed and trailing input always produces a bounded diagnostic or typed limit error; no silent truncation or partial-success publication.
• Parser fuzzing covers arbitrary bytes, deep quoting/escaping, long logical lines, and include graphs.
• ReleaseFast benchmark reports bytes/second, directives/second, allocations, and peak owned bytes for small configs and CRS.
• Exact Pantry-pinned Zig formatting, tests, compile checks, and parser benchmark pass locally and in hosted CI.
• Implementation lands on main in small conventional commits with linked evidence.

Test matrix

• LF/CRLF/final-line-without-newline and continuation boundaries.
• Empty, single, double, and mixed quoting; escaped quotes, backslashes, commas, hashes, and whitespace.
• Comments at line start/after tokens versus hashes inside operators, regexes, macro text, and quotes.
• SecRule with one/many/negated/count/selector targets, negated operators, empty/nonempty parameters, and zero/many actions.
• Directives with zero/one/many arguments and original-case preservation.
• Syntax diagnostics at first/middle/final byte with exact line/column and include ancestry.
• Include file, sorted glob, optional missing, duplicate, cycle, depth/source/byte/glob limit, absolute path policy, traversal, and symlink escape.
• Allocation failure injection and repeated parse/deinit leak checks.

Relationships

• Parent: WAF-00

Blocked by

• WAF-03 — complete

Blocks

• WAF-11 — immutable semantic execution plans
• WAF-12 — stable directive union validation
• WAF-13 — updates and authenticated remote rules
• WAF-33 — validate/format/test/explain CLI
• WAF-34 — SecRules Test Set
• WAF-37 — CRS FTW compatibility
• WAF-38 — parser and include fuzzing

Baselines and upstream references

• ModSecurity 3.0.16
• Coraza 3.7.0
• OWASP CRS 4.28.0
• Zig 0.17.0-dev.1441+d5181a9c9, resolved and locked through Pantry

[chrisbbreuer]

WAF-03 (#4) is complete, so this issue is unblocked. The implementation contract has been expanded from a placeholder into concrete source ownership, lossless syntax, include-security, diagnostics, limits, differential fixtures, fuzzing, and benchmark acceptance criteria. Work starts only against this tracked scope.

[chrisbbreuer]

The first WAF-10 slice landed as feat(parser): add owned SecLang sources.

It adds stable 32-bit source IDs, copied UTF-8 path/config bytes, half-open spans, physical line starts, byte-accurate 1-based locations, include ancestry, and deterministic per-source/aggregate/path/line/source-count limits. Invalid UTF-8, ancestry, IDs, and spans remain distinct typed errors, and rejected input is validated before ownership. Focused ownership, CRLF location, ancestry, invalid UTF-8, limit, and span tests pass with the exact pinned compiler.

[chrisbbreuer]

Logical-line normalization landed as feat(parser): normalize SecLang logical lines.

The iterator handles LF/CRLF physical lines, applies the pinned scanner continuation rule (strip horizontal whitespace before backslash and indentation after newline), retains logical-to-physical segments and whole physical spans, and reports bounded line/segment and dangling-continuation errors. Returned lines own both text and source maps with allocation-failure-safe cleanup. Focused mapping, CRLF, multi-continuation, limit, and EOF tests plus full compile checks passed.

[chrisbbreuer]

Quoted argument tokenization landed as feat(parser): tokenize quoted SecLang arguments.

Tokens preserve raw logical slices, quote kind, logical offsets, and mapped physical spans. The state machine handles single/double/unquoted arguments, escaped bytes, empty quoted values, inline hashes inside active tokens, and comments beginning between tokens. Unterminated quotes, dangling escapes, token size, and token count are explicit bounded errors. Focused quote/escape/hash/comment and hostile-limit tests plus full compile checks passed.