Repository navigation
Embed the question once per chat turn, not on every graph lookup - #971
Merged
Merged
Conversation
Signed-off-by: dada-yan <BinjunYann@gmail.com>
WaylandYang
approved these changes
Sep 27, 2026
WaylandYang
left a comment
Contributor
There was a problem hiding this comment.
Thank you. The cache lives on the sink, so it cannot outlive a turn, only successes are kept, and the vectors are the same so nothing ranks differently. The request-count table against dev is the proof. Landing it. search_chunks and mapping_index::relevant sharing the same cache would be a fine follow-up. Thanks again.
WaylandYang
approved these changes
Sep 27, 2026
WaylandYang
left a comment
Contributor
There was a problem hiding this comment.
Maintainer review: sound, landing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a name matches more than one entity, the graph tools rank the candidates by the question.
rank_by_questionembeds the turn's question and compares it with each candidate's context profile.The question is the same for the whole turn, but it is embedded again on every lookup by name:
find_entities;entity_facts,neighborsandtimelinecall that takes a name;paths_between, once for each end.Each time,
ToolCtx::embedreads the workspace's model settings and calls the embedding service.A predicate that matches no fact is embedded the same way, on every call with that word (
aligned_predicates).Change
ToolSinkgainsembeddings: the texts embedded in this turn, with their vectors.Shared.sink) and MCP one per call, so nothing outlives a turn.ToolCtx::embed_once(sink, text)returns the kept vector, or callsembedand keeps the result.rank_by_questionandaligned_predicatesuse it. They andfilterednow take the sink, which every caller already holds.Not in scope, and they could share this later:
search_chunksembeds its own query throughretrieval;mapping_index::relevantalso embeds the question.Verification
Run in containers on a Linux host: Rust 1.98.1 and Postgres 16 with pgvector.
New tests
chat_embed_once_tests.rsuses a fake/embeddingsendpoint that records every request. The workspace's embedding model points at it.The seed is two entities named Acme:
Tools, with one sink standing for one turn:
find_entitiesmakes no request and lists the busier Acme first.find_entities,entity_factsby name andneighborsby name all pick the closer Acme;entity_factstwice with a predicate that matches no fact adds one request, for the word.Chat: a scripted model calls
find_entities, thenentity_factsby name. The endpoint gets one request.On dev (f1a36a5) with only the new tests
Both tests fail on their request counts. The ranking checks before those counts pass on dev too: dev picks the same entities, it only embeds more often.
A copy of the tests that prints the requests instead of asserting them gives the full count:
find_entities, thenentity_factsby namefind_entities,entity_factsandneighborsby nameentity_factstwice with a predicate that matches no factfind_entitiesin the next turnOn dev, every lookup by name re-embeds the question. The two predicate reads cost two requests each, one for the question and one for the word.
This branch
cargo fmt --all --checkis clean.cargo clippy --locked --workspace --all-targets -- -D warningsis clean.cargo test --locked -p utopia-server --bin utopia-server api::passes 189 tests.UTOPIA_TEST_REQUIRE_PDFTOTEXT=1 cargo test --locked --workspacepasses.The branch sits on f1a36a5 and merges cleanly onto today's dev, which has #967 and #968.