[Maya-Kid]

Follow-up to #968: the two follow-ups its review asked for.

Change

The entry carries the sentence.

• graph::first_live_evidence returns the quote of the evidence it picks, along with the chunk. The order now ends in the quote's position within the chunk, so the pick is deterministic when a chunk holds several evidence rows for one fact.
• Evidence::mark still numbers by chunk through cite, so one chunk keeps one number. It then adds the fact's quote to that entry's quotes (tools::add_quote), deduplicated and in citation order.
• A chunk cited for two facts carries both sentences.
• A chunk a search registered first gains the sentence when a graph tool cites it; one a graph tool registered first keeps its sentence when a search hits it.
• MCP has no source list and is unchanged.

The stream publishes the change. cite used to only append, so the producer sent a sources frame when the list grew. An added quote does not grow it, so the frame now goes out when the list changed at all. Stored sources are what they were, with quotes where a graph tool cited.

The web marks it.

• Source gains quotes.
• The preview marks those sentences in the passage with the search-hit style (u-mark), and the list row leads with the first sentence instead of the chunk's opening.
• "Open original" passes the first sentence as a quote search parameter, and the document page marks it in the cited chunk.
• Marking is a pure function (quotes.ts):
  • each sentence is found at its first occurrence;
  • whitespace matches whitespace, so a quote's space matches the passage's line break;
  • overlapping sentences merge;
  • a sentence not in the passage is not marked.

Docs. docs/design/chat-and-mcp.md says what the entry carries.

Known limits

• One number for several sentences. When two facts are read from one chunk, their shared number opens the chunk with both sentences marked. "Open original" passes the first.
• Quotes that are not in the passage. A quote whose words are not in the passage (extraction paraphrased it) is listed but not marked.

Verification

Run in containers on a Linux host: Rust 1.98.1 and PostgreSQL 16 with pgvector, Node 20, and Playwright's Chromium for the browser tests.

New tests

In chat_graph_citation_tests.rs, on #968's fixture:

• a_graph_number_opens_the_sentence_it_was_read_from
  • Two facts read from two sentences of the charter chunk share one number, and the entry carries both sentences.
  • After the charter chunk is superseded, Zhang San's number opens the recap chunk and carries the recap's sentence.
  • With as_of before the supersession, it is the charter chunk and the charter's sentence again.
  • MCP lists no sources.
• a_search_and_a_graph_tool_share_one_number: search_chunks registers the handover chunk, then entity_facts cites Li Si's fact with the same number, and the entry gains the sentence. In the other order, the search reuses the graph's number and the sentence stays.
• a_quote_added_to_a_search_hit_is_published: a scripted chat searches, then calls entity_facts for Li Si, whose chunks the search already registered. The second sources frame has the same entries plus their quotes, and the stored sources equal the last frame.

web/src/quotes.test.ts covers:

• a sentence marked where it stands, including across a line break;
• a sentence not in the passage;
• overlapping and separate sentences;
• characters a regular expression would read as syntax;
• the list row's lead;
• the document route's quote parameter.

On dev (0fbcd38) with the new tests

The control branch is dev plus the new tests, with a stub of today's behaviour for the web helper.

• Server. The three new tests fail at their first quote check: the entry has no quotes. The chat test fails because no second frame is sent when only a quote changes.
• Web. 6 of the 7 tests fail. The one that passes checks a sentence that is not in the passage.
• Graph answers cite the sentence each fact was read from #968's two existing citation tests pass there, as they do here.

This branch

• Server (the citation, source-publishing and api:: tests): 202 passed.
• cargo fmt --all --check is clean.
• cargo clippy --locked --workspace --all-targets -- -D warnings is clean.
• UTOPIA_TEST_REQUIRE_PDFTOTEXT=1 cargo test --locked --workspace passes.
• Web:
  • pnpm test: 203 passed;
  • pnpm build passes (style guard, tsc, vite);
  • tests/chat-view.test.mjs in Chromium: 30 passed.

[WaylandYang]

Thank you. The sentence travels with the entry, one chunk keeps one number, the frame goes out when the list changed and not only when it grew, and the marking is a pure function that returns nodes. Both follow-ups from #968 are covered, including the shared number between a search and a graph tool.

I pushed one commit with three bounds:

• A source entry keeps at most eight sentences of at most 400 characters. The list is resent after every tool call and goes into the final request, and a chunk that yields many facts should not grow it.
• The quote in the document page's address is dropped above 300 characters, whole rather than cut, since a cut sentence cannot be found in the passage. It stays in browser history and access logs, and a long Chinese sentence outgrows a request header on reload.
• The two ordering terms on the quote are gone. fact_evidence has one row per fact and chunk, so they never decided anything.

Landing it. Thanks again.

[WaylandYang]

Maintainer review: sound, landing.