Answer engine: grounded, cited synthesis with verify-then-cite - #9
Conversation
The answer engine's whole contract is that every claim ties to a span of a specific source and can be checked, so the grounding model is the first thing to land. It follows the Gemini metadata shape: chunks are sources, supports map a byte span of the answer to the chunks that back it. Two things are easy to get wrong and are fixed here once. Offsets are byte offsets into the UTF-8 answer, not rune offsets, so a marker never splits a multi-byte character. Markers are inserted in reverse order by end offset, so inserting one never shifts the offsets of a span not processed yet. Validate rejects an out-of-range span, a span whose text does not match its bytes, a citation of a missing chunk, and a span with no evidence behind it.
This is the piece that makes per-query LLM cost survivable. The router decides, before any model runs, whether a query takes the classic search path with no model, a single retrieve-rerank-synthesize pass, or the agentic deep-research loop. Production swaps in a small trained classifier behind the Classifier seam; the rule classifier here is the explainable reference and the behavioral target for that model. The policy biases toward escalation on purpose. A comparison wins the agentic route over a plain question, a long informational query escalates, and an ambiguous query goes to a synthesized pass rather than dropping to bare search, because a confidently wrong cheap answer is the failure mode to avoid. EscalationRate is here so the health check can watch for the cheap tier getting too weak.
The synthesis model is the expensive external piece, so it sits behind the Synthesizer seam and the work that is actually ours to get right lives here: how the reranked passages are ordered and trimmed before the model reads them. Each of the three context decisions from doc 09.3 is a plain function, because each is a place naive RAG quietly loses quality. Order deals the strongest passages to the two edges of the context and the weakest to the middle, where a model attends least. Budget fills to a fraction of the window rather than cramming it. Consolidate caps how many passages one source contributes, so a site cannot outvote the corpus by repeating itself. Freshen blends a half-life recency decay into the score for time-sensitive intents, and leaves a passage with no timestamp alone rather than guessing.
This is where the only-answer-what-you-can-prove promise is enforced. Decompose splits the answer into atomic claims at sentence boundaries, recording each claim's byte span so a verified claim becomes a grounding support without re-finding its offsets. Check runs the entailment model over a claim against every passage it cites and passes the claim if any one source entails it. Correct applies the policy: keep an entailed claim above the confidence floor, drop the rest, and flip the answer to unverified when anything was dropped. The NLI model is external (MiniCheck-class over gRPC), so it sits behind the Verifier seam. The OverlapVerifier reference scores by content-word overlap; it is a stand-in for the model, not a replacement, and it documents the contract the real model has to meet. Decompose does not split a decimal point, since a period without trailing space is not a sentence boundary.
Correctness and attribution are different properties, so the answer engine needs its own harness on top of the relevance metrics. The RAGAS family diagnoses retrieval and generation separately: faithfulness is the grounding metric, context precision rewards ranking the passages that matter first, and context recall catches a needed passage that retrieval never fetched. The ALCE family scores the citations: recall is whether the cited passages entail each sentence (the AIS score), precision is whether each citation is pulling weight. Like rank/eval, these take judgments rather than calling a model, so the arithmetic is a pure function and the NLI model is not a hidden dependency. A test pins the harness against the published ASQA best-system numbers from doc 09.6 so the metrics stay comparable to the literature.
The engine threads a query through the whole pipeline over the seams the other packages define, so the production engine is this same control flow with the gRPC clients dropped in. It routes first and returns ErrSearchRoute for a navigational query so the caller never touches the model, retrieves a wide candidate set, reranks to a small context, then constructs the context with consolidate, order, and budget before the model reads it. The citation flow is the load-bearing part. The model emits inline markers, so the engine parses them off the text into per-sentence claims with byte spans, verifies each claim against the passages it cited, and re-inserts markers only for the survivors. An answer with an unsupported claim comes back with that claim dropped and Verified false, never with an unsupported citation shown. Lives in its own package because synth and verify both import the root answer package, so the composition that imports all of them cannot sit there too.
| ordered := make([]Support, len(supports)) | ||
| copy(ordered, supports) | ||
| sort.SliceStable(ordered, func(i, j int) bool { | ||
| return ordered[i].Segment.End > ordered[j].Segment.End |
There was a problem hiding this comment.
This descending sort by end offset is the one line that makes byte-offset citation insertion safe, so it is worth being explicit about. If we inserted markers left to right, the first marker would push every later span to the right by its own length, and the recorded offsets would all be stale by the time we reached them. Going right to left, every insertion happens past the offsets we still have to use, so they stay valid as the string grows. The copy above is so a caller's support slice is not reordered under it, since Insert is a read of that slice, not an owner of it.
| q := strings.ToLower(strings.TrimSpace(query)) | ||
| words := strings.Fields(q) | ||
|
|
||
| for _, cue := range multiHopCues { |
There was a problem hiding this comment.
Order matters here and it is deliberate: the multi-hop check runs before the plain-question check, so a query like 'how does a skip list compare to a btree' takes the agentic route even though 'how' alone would have sent it single-pass. The reasoning is the escalation bias from doc 09.5. A comparison that we answer in a single retrieval pass is exactly where a bridge fact gets missed and the answer is confidently wrong, which is the failure the router exists to prevent, so when the signals are mixed we pay for the more expensive route rather than risk the cheap miss. EscalationRate is the counterweight: if this biasing pushes the escalation rate past about 0.3 it means the cheap tier is too weak, not that the bias is wrong.
| for _, c := range claims { | ||
| verdict := Check(v, c, passages) | ||
| if !verdict.Entailed || verdict.Score < minConf { | ||
| res.Verified = false |
There was a problem hiding this comment.
This is the whole only-answer-what-you-can-prove contract in one branch. A claim that fails entailment is not patched or re-cited here, it is simply left out of the supports and the answer is flagged unverified, so the engine can hedge the wording rather than show a citation that does not hold. The alternative, keeping a weak citation because the model was confident, is the exact fabrication mode the verifier exists to catch. The confidence floor above this matters too: a model can nominally entail a claim at its own low threshold while the evidence is thin, so MinConfidence drops a borderline entailment rather than dressing it as a solid citation.
|
On the citation flow, since it is the part that ties the packages together. The model emits inline markers, so the engine parses them off the raw text in answer/engine/citations.go before anything else: it produces the clean answer (markers removed, one separating space kept between sentences so it still reads right) plus one claim per sentence carrying the byte span and the 0-based chunk indices the markers named. That clean-plus-spans shape is exactly what verify.Correct consumes and what ground.Insert re-annotates, so the round trip is parse, verify, re-insert, and only the verified supports get markers back. The reason to parse rather than trust the model's own placement is that an unverified marker is worthless under the contract, so the marker you see on the page is always one that passed entailment, never one the model merely wrote. |
|
Scope note for reviewers, the deliberate narrowing for this tier. Everything here is in-process and tested against seams, no served model and no NLI model yet. The OverlapVerifier is a content-word-overlap stand-in for the MiniCheck-class model, good enough to exercise the decompose-check-correct flow and to document the contract the real model has to meet, not a real fact-checker. The rule classifier is the same idea for the router: explainable, dependency-free, and the behavioral target for the small trained model that replaces it. The agentic multi-hop loop (doc 09.4) and the conflict evaluator's CRAG path (doc 09.3) are not in this PR; the single-pass funnel is, and the agentic mode is the router-gated extension that builds on it. Numbers like the candidate pool, the context size, the utilization fraction, and the confidence floor are config, not constants, so they tune per deployment. |
1 participant
What this is
The AI answer engine: the retrieve-rerank-synthesize-cite pipeline that turns a
query into a grounded, cited answer over OpenIndex's own auditable corpus. It
realizes architecture doc 09 and implementation doc 09. The governing principle
carries through every package here: only answer what you can prove.
The whole subsystem is the
answer/package tree, built against seams andtested in process, so the production engine is this same control flow with the
gRPC clients and the served models dropped in.
The pipeline
A four-stage funnel layered on the retrieval stack (docs 05 to 08), because
synthesis quality is bounded by retrieval quality:
What landed
answerPassage,Citation,Answer.answer/groundValidate.answer/routerEscalationRate.answer/synthOrder,Budget,Consolidate,Freshen) and theSynthesizerseam.answer/verifyVerifier(NLI) seam, the per-claimCheck, and the verify-then-citeCorrectpolicy.answer/evalanswer/engineDeliberate narrowing
Built against seams, references in process. Same pattern as the earlier
PRs. The expensive external pieces are behind interfaces:
RetrieverandRerankerfor the serving tier,synth.Synthesizerfor the served model(an Apache-2.0 base fine-tuned to emit the citation format, vLLM/SGLang,
doc 09.5), and
verify.Verifierfor the MiniCheck-class NLI guardrail(doc 09.2). Each has a correct, tested reference (
OverlapVerifier, the ruleclassifier, the stub seams in tests) so the pipeline runs end to end with no
model and no network. The seams are the deliverable; the bindings are a later
mechanical swap.
Grounding is byte offsets, and citation insertion is reverse order. Two
details that corrupt a citation if you get them wrong are fixed once in
ground: offsets are byte offsets into the UTF-8 answer so a marker neversplits a rune, and markers are inserted in descending end-offset order so
inserting one never shifts a span not yet processed.
Validaterejects a spanout of range, a span whose text does not match its bytes, a citation of a
missing chunk, or a span with no evidence.
The router carries the economics. Running a model on every query is not
survivable, so the router decides the path before any model runs and biases
toward escalation, because a confidently wrong cheap answer is the failure
mode to avoid (doc 09.5). The rule classifier is the explainable reference and
the behavioral target for the small trained model that replaces it.
Verify-then-cite, not cite-then-trust. The engine parses the model's
inline markers into per-sentence claims with byte spans, checks each claim
against the passages it cited, and re-inserts markers only for the survivors.
An answer with an unsupported claim comes back with that claim dropped and
Verifiedfalse, never with an unsupported citation shown (doc 09.2).Eval is judgment-in, like rank/eval. The metrics take entailment verdicts
rather than calling a model, so the arithmetic is a pure function and stays
comparable to the literature; a test pins the harness against the published
ASQA best-system numbers (doc 09.6).
Not yet built (tracked)
The as-built note
2050/code/09_answer_engine.mdlists the deferred pieces: theserved synthesis model and the NLI model behind their seams, the agentic
multi-hop loop (doc 09.4), the conflict evaluator's CRAG path, and semantic
caching. The mixer that places the answer as a vertical on the page (doc 08.5)
lands with the wire serving client.
Checks
gofmt,
go vet, and golangci-lint clean; full module green under the racedetector;
go mod tidyadds no new dependency. All prose and comments arehuman-written with no em-dashes.