Repository navigation
Apply E5 query/passage prefixes via the prefixing provider defaults (FastEmbed asymmetry) #1284
Description
Activity
Claude finished @phernandez's task in 49s —— View job
Triage
Type: Enhancement — this proposes new default behavior (per-model-family role prefixes for asymmetric embedding models), not a fix for broken code. A config-based workaround (
semantic_embedding_query_prefix/semantic_embedding_document_prefix) already exists today.Priority: Medium — retrieval quality degrades silently for asymmetric FastEmbed model families (E5), but it's opt-in territory (users choosing a non-default embedding model) and a workaround is already available, so it's not blocking core functionality for the default
bge-small-en-v1.5path.Component: Not cloud-specific. This lives entirely in local embedding-provider infrastructure —
src/basic_memory/repository/prefixing_provider.py,embedding_provider_factory.py, andfastembed_provider.py. I confirmedPrefixingEmbeddingProvider(prefixing_provider.py) already exists withdocument_prefix/query_prefixfields wired intoidentity_key(), matching the issue's description of the reindex-on-identity-change path. I also confirmedFastEmbedEmbeddingProvider.embed_query(fastembed_provider.py:322-324) currently just callsembed_documents([text])— i.e. there's noquery_embed/role-prefix distinction at all in the current tree, consistent with the issue's premise that the asymmetry problem is still open.Complexity: Medium — the mechanism (prefixing provider + identity-based reindex) already exists; the work is (1) a per-model-family default prefix map (E5 at minimum, with an open question on whether BGE's query instruction belongs in the same map), (2) deciding whether/how to layer a genuine
query_embedcall for models that implement it (e.g. Jina v3), and (3) a regression test shaped like the real FastEmbed dispatch. Main design judgment call is the scope of the default map (E5-only vs. E5+BGE) — worth confirming before implementation.Duplicates: None found — this is the only open issue referencing FastEmbed prefix/asymmetry behavior; it correctly builds on #1264 and PR #1265.
Closing as not planned. Basic Memory already supports explicit document/query prefixes, and we do
not want the runtime to own a growing map of model-specific prompts.#1302 documents the required multilingual E5 configuration and keeps custom model contracts under
operator control.
Follow-up to #1264 and PR #1265 (credit to @Aryan-Pardeshi for the initial attempt and to the #1264 reporter for the cosine-margin measurements showing the retrieval impact).
Problem
FastEmbed queries are embedded through the passage path, so asymmetric model families (E5:
query:/passage:prefixes) lose their role prefixes and retrieval quality degrades (#1264).PR #1265 rerouted
embed_queryto FastEmbed'squery_embed, but in fastembed 0.8.0query_embedis a passthrough toembedfor both the defaultBAAI/bge-small-en-v1.5(OnnxTextEmbedding) and the E5 family (PooledEmbedding) — onlyJinaEmbeddingV3overrides it. The change therefore produces byte-identical vectors for the reported case.Proposed fix
query:/passage:) through the existingPrefixingEmbeddingProviderpath (src/basic_memory/repository/prefixing_provider.py). Prefixes are already hashed into provider identity (prefixing_embedding_identity, wired inembedding_provider_factory.py), so changed document semantics automatically trigger reindex.query_embedcall for models that genuinely implement it (Jina v3), layered under the prefixing wrapper.query_embeddelegates toembed, and recorded inputs still receive both role prefixes.Workaround available today
semantic_embedding_query_prefix/semantic_embedding_document_prefixconfig already applies prefixes for any provider, including FastEmbed E5 models.Notes
bge-*models also document a recommended query instruction ('Represent this sentence for searching relevant passages: ') — worth deciding whether the default map covers BGE too or E5 only in the first pass.