Where embeddings live, and what gets embedded

[antejavor]

Part of #390

Question

The eval embeds turns, entity names and edge texts with bge-small on the host, cached beside the run. Memgraph's in-container embeddings.text with bge-m3 was OOM-killed under the Docker VM's memory. For the product, decide:

• which model;
• whether vectors live in Memgraph vector indexes or elsewhere;
• which units are embedded;
• when embedding runs (at ingest, or lazily);
• the memory budget.

Depends on

The lift ticket.

[antejavor]

Decisions (grilling, 2026-10-05)

1. Where vectors live: in Memgraph, computed inside it by MAGE's embeddings.text. Nothing is embedded on the host.

2. Model: BAAI/bge-small-en-v1.5 as the configured default ([recall] embedding_model in config.toml). It's what the 84/100 was measured with. The model name and dimension are recorded with the vectors, so a model change is detected rather than mixing vectors. bge's query instruction is prepended to questions only. Checked: bge-small runs through embeddings.text in a 2 GiB container, using about 750 MiB. bge-m3 is the model that ran out of memory.

3. What is embedded: the eval's three units.

   • User and assistant messages. actions-graph writes a plain text property on message actions when it creates them, replacing the eval's ensure_turn_text_index workaround.
   • Entity names.
   • Edge source sentences (r.text).

   Tool calls and results are not embedded, and long messages are not chunked, until Does the graph lane help on coding sessions, not just LongMemEval? #395 can measure either.

4. When:

   • Messages are embedded by a detached step at SESSION_END, independent of auto_reconcile. It costs no LLM.
   • Entities and edges are embedded during reconciliation.
   • A catch-up command embeds anything missing.
   • The hook never waits on the model. A message is findable by full-text search as soon as it has text.

5. Requirements and degradation:

   • MAGE is required, with a 2 GiB minimum for bge-small. Setup docs and bootstrap move from memgraph/memgraph to memgraph-mage.
   • doctor checks that the module exists and the model loads.
   • Without it, failed embed steps are left for the catch-up command, and recall runs its full-text and graph lanes and says in its header that the vector lanes are off.

6. Scoping to one user: vector_search.search returns the global top-k with no filter. So each vector lane does an exact Cypher scan of the user's own rows with vector_search.cosine_similarity, the same semantics as the eval's scoped top-k. To verify: latency on the eval's largest user and on a synthetic 100k-message user. If it's too slow, fall back to a global ANN index with a measured over-fetch.

Consequence for the score: the eval switches to the shared recall, so its vectors come from embeddings.text instead of the host embedder. One run on the rebuilt graph confirms the 84/100 holds.

[antejavor]

Implemented by #416, with scoping in #417.