feat(kb): add span-level retrieval metrics (recall/precision/precision-ceiling/IoU) — doc-ID precision@k cannot compare chunkers #13244

Problem

Retrieval quality is currently scored only as precision@k over document IDs. That metric is not
comparable across chunking strategies: a strategy that emits larger chunks trivially scores higher,
because each retrieved ID covers more of the document. Any future comparison of chunkers or chunk
sizes using precision@k will therefore pick the strategy with the biggest chunks, regardless of
whether it actually retrieves the relevant text.

The fix is span-level scoring: measure the character overlap between the retrieved text and the
ground-truth excerpt, which is independent of how the corpus was split.

Evidence

• autobot-backend/knowledge/rag_benchmarks.py:1156-1160 — _precision_at_k() is a set-membership
  fraction over doc IDs; there is no span or character-offset arithmetic anywhere in the module.
• autobot-backend/knowledge/rag_benchmarks.py:1251-1290 — BenchmarkResult carries only
  retrieved_ids, ranked_ids, precision_at_k; no retrieved text or offsets are retained.
• _GROUND_TRUTH maps a query to a set of doc IDs, not to text excerpts, so there is nothing to
  compute an overlap against today.

Proposed approach

Add a span-scoring module with four metrics, all computed as character-interval set operations
against the source document:

Implementation notes:

• Resolve each retrieved chunk back to its offsets in the source document: exact substring first,
  then a whitespace-insensitive match, then fail loudly. Do not silently fuzzy-match — a chunker
  that rewrites text must be reported as unscoreable, not scored wrongly.
• A document may contain the same chunk text more than once (repeated boilerplate). Resolve
  ambiguous matches by nearest-unclaimed occurrence, not by first occurrence, or the score lands on
  the wrong span.
• Union overlapping retrieved spans before computing the precision denominator. Skipping this
  penalises any overlapping-window chunker for its own overlap and would steer us away from a
  legitimate strategy.
• The "precision ceiling" metric is the useful diagnostic: it separates "the chunker cannot win"
  from "retrieval did not find it".

Ground truth must gain an excerpt field (query → verbatim excerpt) alongside the existing doc-ID
set; keep both so precision@k continues to work.

Risks / constraints

• Requires ground-truth excerpts, which today's fixture does not have — coordinate with the
  eval-set generator issue in this umbrella, or hand-write excerpts for the existing 20-doc fixture
  as an interim.
• Span resolution is O(chunks × references) per query; keep the reference corpus bounded.

Acceptance criteria

• Span-scoring module with the four metrics above, unit-tested against hand-computed values.
• Overlapping retrieved spans are unioned before the precision denominator, with a test that a
  50%-overlap window chunker is not penalised relative to a zero-overlap one at equal coverage.
• Duplicate chunk text resolves to distinct spans, with a test using a document containing the
  same sentence twice.
• A chunker that mutates chunk text raises a clear error rather than producing a silent score.
• BenchmarkResult carries the span metrics; existing precision@k assertions still pass.