[Maya-Kid]

Closes #943.

Problem

When asked to shorten, translate or rephrase the previous answer, the assistant answers from the transcript without gathering anything, as the system prompt tells it to. It copies the answer's [1] and [2] along with the words.

Each turn starts with an empty ToolSink, so the restating turn has no sources. Its marks open nothing, and the answer is labelled as citing no sources.

Change

This implements option 1 from #943, with a number check.

• When a turn keeps the previous answer's sources. Both of these must hold:

  1. The turn gathered nothing: it has no tool step, having called only no_evidence_needed or no tool at all, and it has no source of its own.
  2. It cites at least one number, and every number it cites was also cited in the previous answer's text.

• What it keeps. It takes the previous answer's stored sources that it cites, unchanged: the same objects with the same numbers.

  • They are saved with the turn and sent in its sources frame. So the marks open the same passages, both live and after a reload.
  • If either condition fails, nothing changes, and marks without a source stay as they are today.

• History gains last_sources. It is read from the same last assistant message as last_tool_exchange. This follows the existing design: the last exchange is replayed precisely so that "translate it" or "say it shorter" has its evidence in view.

  • recent_context now reads its rows into a small FromRow struct instead of a tuple. A seventh tuple field would have tripped clippy's type_complexity.

• cited_numbers reads citations in the same shape that the page draws as marks (citeRe in citations.ts):

  • brackets that hold only digits, separated by , or ，;
  • 0 is not a citation number.

  It is a small hand parser, because the workspace has no regex dependency.

• Design doc: chat-and-mcp.md gains half a sentence.

Limit

Suppose a follow-up restates an older answer, and every citation number it cites also appears in the previous answer. Then the marks open the previous answer's passages.

A stricter rule could leave marks unmapped whenever an earlier answer also cited the same number. But nearly every answer cites [1], so that rule would also drop the common case: restating the last answer after a few exchanges.

Verification

Run in containers on a Linux host (Rust 1.98.1, Postgres 16 with pgvector).

• New tests in chat_restated_tests.rs run two turns in one conversation through the real chat handler, against a scripted endpoint:

  1. The first turn reads a document and cites [1]; the second calls no_evidence_needed and restates with [1]. The second answer's stored sources and its last sources frame are the first answer's source 1.
  2. The second turn cites [2], which the first answer never cited. It keeps no sources.
  3. The second turn searches (with no result) and then cites [1]. It keeps only its own sources, so none.
  4. Unit test cited_numbers_reads_the_shapes_the_page_marks.
     • It reads [1][3], [4, 6] and [2，5].
     • It rejects [ 1], [1 ], rows[0], [a1], [1,], [+1] and [].
     • It finds [2] inside [[2]].

• On dev (b07af8f), with only tests 1–3 added:

  • Test 1 fails: the second turn keeps no sources.
  • Tests 2 and 3 pass, as they should. They guard the cases where nothing may be kept.

  Test 4 uses the new function, so it was left out of that run.

• With this change:

  • cargo fmt --all --check passes.
  • cargo clippy --workspace --all-targets -- -D warnings passes.
  • cargo test -p utopia-server api::: 175 passed.
  • cargo test --workspace with UTOPIA_TEST_REQUIRE_DB=1 and UTOPIA_TEST_REQUIRE_PDFTOTEXT=1: 1190 passed, 0 failed, 5 ignored (the existing #[ignore] tests).
  • cargo build --workspace passes.

[WaylandYang]

Option 1 from #943 with the number check that makes it safe: a turn that gathered nothing and cites only numbers the previous answer cited keeps those sources, stored with the turn so replays resolve them too. The three handler cases cover keep, a foreign number and a turn that searched; the limit you name is accepted. Merging.