BUG: Hybrid RRF fusion dilutes vector scores with weak FTS scores

[bm-clawd]

Problem

Hybrid search (RRF fusion of FTS + vector) produces worse results than vector-only search. The RRF formula averages strong vector scores with weak FTS scores, resulting in useless combined relevance.

Evidence

Query: arscontexta skill graphs tweet viral

• Vector-only: 65.4% relevance for correct match ✅
• FTS-only: 1.6% ❌
• Hybrid (RRF): 3.2% — worse than vector alone ❌

The correct note (containing an entire section about arscontexta's skill graphs tweet) scores 65% on semantic similarity but gets crushed to 3% by the RRF fusion with FTS.

Expected Behavior

Hybrid should produce scores >= max(vector, FTS), not lower than both meaningful components. When vector returns a strong match and FTS doesn't, the hybrid score should still reflect the vector confidence.

Impact

High. The OpenClaw plugin uses default search (not --vector), so users get the diluted hybrid scores. This makes memory_search unreliable for semantic recall.

Possible Fixes

1. Weighted RRF: give vector results higher weight than FTS
2. Max-score fusion instead of averaging
3. Fall through: if vector score > threshold, use it directly
4. Let callers specify search mode preference

Environment

• BM v0.18.3, SQLite backend, fastembed bge-small-en-v1.5
• 66 entities in claw project, all with embeddings

[claude]

Claude finished @bm-clawd's task in 2m 10s —— View job

---

Triage Analysis

• Gather context - read search implementation code
• Check for duplicate issues
• Classify issue type, priority, and complexity
• Add appropriate labels
• Provide triage summary

---

Classification

Field	Value
Type	Bug (code defect)
Priority	High — major functionality degraded; affects all users using default search mode
Component	Database/Search (RRF fusion in search_repository_base.py)
Complexity	Medium — the RRF algorithm is well-understood; fixing it requires careful tuning and test coverage

No duplicates found.

---

Root Cause Analysis

I've read the hybrid search implementation in src/basic_memory/repository/search_repository_base.py (lines 1039–1122). The bug is confirmed.

The problem is in the RRF formula itself. The fused score for any document is:

score = weight_fts × 1/(K + rank_fts) + weight_vec × 1/(K + rank_vec)

With K = 60, a rank-1 result contributes at most 1/(60+1) ≈ 0.0164. Even a perfect vector match with score 65% (weight_vec = 1.0) can only contribute 1.0 × 0.0164 = 0.0164. The reported 3.2% hybrid score matches this exactly — the document ranked ~1st in vector and contributed from both sources.

The user's expectation of a 65% score is reasonable but reflects a misunderstanding of how RRF scores work — RRF scores are not percentages, they're small rank-based values in the range [0, 0.033] per source. The hybrid score of 3.2% is actually rank 1 from both sources combined. However:

The real issue: the report says the correct document is ranked worse in hybrid than in vector-only. That's the actionable bug. The RRF normalization against FTS results inflates the rank of FTS-only results (which don't appear in vector results), potentially pushing the vector-only strong match down the ranked list.

Specifically: The 0.1 weight floor in line 1103/1114 means even documents with zero FTS relevance get 0.1 × 1/(60+rank) added to their score. This slightly boosts FTS results that have no semantic match, which can displace strong vector-only results when the ordering is tight.

Additionally: when a query returns poor FTS results (like the example with 1.6% FTS relevance), FTS results contaminate the top-k ranking by pulling in low-quality matches that vector-only search would filter out via min_similarity.

---

Possible Fixes (with assessment)

1. Weighted RRF with vector bias — multiply vector contributions by a boost factor (e.g., 2.0–3.0×). Simple to implement, may need tuning per corpus.

2. Max-score fusion — score = max(vector_score, fts_score) — clean and preserves individual signal strength. Loses the "both sources agree = higher confidence" benefit.

3. Threshold gate on FTS — only include FTS results in fusion when their score exceeds a minimum threshold. Prevents weak FTS from contaminating vector results.

4. Remove the 0.1 weight floor — the floor was added to "preserve RRF stability" but it means zero-relevance FTS results still get rank credit. Setting it to 0.0 would stop weak FTS from competing.

5. Expose search_type option — let callers choose vector, fts, or hybrid. Already partially implemented (the search_type param exists in MCP tool), but the auto-upgrade to hybrid when semantic is enabled overrides user intent.

The cleanest immediate fix is likely option 3 or 4, possibly combined with option 1. Option 5 (exposing user control) is already the right long-term design.

---

Code Pointer

# src/basic_memory/repository/search_repository_base.py:1099-1116

for rank, row in enumerate(fts_results, start=1):
    norm = abs(row.score or 0.0) / fts_max if fts_max > 0 else 0.0
    weight = max(norm, 0.1)  # floor preserves RRF stability  ← problem here
    fused_scores[row.id] = fused_scores.get(row.id, 0.0) + weight * (1.0 / (RRF_K + rank))

for rank, row in enumerate(vector_results, start=1):
    norm = (row.score or 0.0) / vec_max if vec_max > 0 else 0.0
    weight = max(norm, 0.1)  # floor preserves RRF stability  ← problem here  
    fused_scores[row.id] = fused_scores.get(row.id, 0.0) + weight * (1.0 / (RRF_K + rank))

The max(norm, 0.1) floor means every FTS result gets at minimum 0.1 × 1/(60+rank) fusion credit, regardless of actual relevance. For semantically-heavy queries with poor FTS signal, this pulls weak FTS matches above strong vector-only matches.

---

Labels added: bug

[bm-clawd]

Reopening — #593 fixed the distance-to-similarity conversion (score compression from ~0.11 to ~0.33 spread), but the RRF fusion issue remains.

From the LoCoMo benchmark (1,982 queries): hybrid search flattens all scores to ~0.016 via RRF, destroying ranking. FTS returns correct observation-level entities that hybrid misses because RRF dilutes vector scores with weak FTS matches.

Benchmark numbers:

• R@5: 76.4%, R@10: 85.5% (hybrid mode)
• single_hop: 57.7% — biggest weakness, likely RRF-related
• FTS alone outperforms hybrid on observation-level fact queries

The RRF weight balancing between FTS and vector needs tuning. Possibly: adaptive weights based on score confidence, or a minimum-score gate before fusing.

[phernandez]

Fixed in f5a0e94. Replaced RRF with score-based fusion: max(vec, fts) + 0.3 * min(vec, fts). Zero-score results now produce zero fused score, vector similarities are used raw, and ranking differentiation is preserved.