chore(train): merge train 2026-10-12 #5 — #18348 one BM25 (#18183) by mrveiss · Pull Request #18358 · mrveiss/AutoBot-AI

mrveiss · 2026-10-12T04:19:16Z

Thinking Path

Merge train #5, under the owner decisions of 2026-10-11 and 2026-10-12 (every approved PR lands through a train). It is built on main f194061 (after train #4). It holds one constituent so the free slot is not left idle while the next constituents finish their delta reviews; they form train #6. It lands with a merge commit, and the drift-prone guards were pre-run locally on the combined tree.

What Changed

Constituent (#PR @ approved head):

refactor(search): one BM25 formula for the KB and the memory graph, and the validator-only hybrid engine retired onto the canonical one (#18183, #17599) #18348 @ 2e5d1b6: one BM25 formula shared by the KB and the memory graph (memory-graph ranking numerically pinned against the old scorer; KB ranking unchanged). The validator-only HybridSearchEngine is removed; it had zero code callers.

Train-only commit:

Hub union in repo_tests/_reach_policy.py and repo_tests/reach_scope_claim_17844_test.py.

Closes #18183
Closes #17599

Verification

refactor(search): one BM25 formula for the KB and the memory graph, and the validator-only hybrid engine retired onto the canonical one (#18183, #17599) #18348 was reviewed at its head: no reversal of newer main, numerically identical ranking, and both ACs met for each issue.
The drift-prone guards (reach declarations, scope, floor migration, guard window, egress, symbol-fork, comment stripper, threat-model anchors) on this tree: 683 passed locally before push.
This PR's CI is the full-suite verification. It is never merged red.

Model Used

Claude Opus 5.5 (merge orchestration); constituent review by Sonnet.

🤖 Generated with Claude Code

Summary by CodeRabbit

Search
- Keyword relevance scoring is now consistent across knowledge-base and memory-graph search.
- Knowledge-base hybrid search uses a single search path. Health checks now verify that search results include the expected source details and warn when information is incomplete.

…he memory-graph scorer (#18183)

…em validator exercises the canonical BM25+RRF HybridSearcher (#17599)

…ed BM25 loop; the #18143 deferral is closed (#18183)

coderabbitai · 2026-10-12T04:20:16Z

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: mrveiss/AutoBot-AI/.coderabbit.yaml
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 5f6198a8-84ca-4f00-867c-7d81ecfeee58

📥 Commits

Reviewing files that changed from the base of the PR and between 78c666b and a060ea5.

📒 Files selected for processing (1)

autobot-backend/utils/system_validator.py

📝 Walkthrough

Walkthrough

The changes share BM25 scoring between the knowledge and memory-graph scorers. They remove a separate hybrid-search module and update system validation to use knowledge-base hybrid search. New repository tests check for duplicate BM25 formulae and hybrid-search implementations.

Changes

Search implementation consolidation

Layer / File(s)	Summary
Shared BM25 scoring `autobot-backend/knowledge/search_components/bm25.py`, `autobot-backend/autobot_memory_graph/semantic_search.py`, `autobot-backend/tests/knowledge/search_components/bm25_shared_18183_test.py`	The knowledge scorer and memory-graph scorer use shared BM25 calculations. Tests compare scores and ranking order.
Knowledge-base hybrid-search validation `autobot-backend/utils/hybrid_search.py`, `autobot-backend/utils/system_validator.py`, `autobot-backend/knowledge/search.py`, `autobot-infrastructure/shared/config/complete.yaml`	The separate `HybridSearchEngine` module and the top-level search configuration are removed. The validator calls knowledge-base search in hybrid mode and checks result fusion provenance.
Repository implementation guards `repo_tests/one_bm25_formula_18183_test.py`, `repo_tests/one_hybrid_search_17599_test.py`, `repo_tests/_reach_policy.py`, `repo_tests/reach_scope_claim_17844_test.py`, `repo_tests/kb_read_visibility_allowlist.py`, `repo_tests/kb_read_visibility_guard_test.py`	Repository tests scan for duplicate BM25 formulae and hybrid-search classes. Reach-scope and visibility checks are updated to register these scans and the validator probe.

Priority: ➖ Normal

Merge Risk: 🟡 Moderate · up to 78c66

The system health check that should confirm hybrid search works actually runs plain vector search. It reports success even though the hybrid path is never tested. The BM25 consolidation itself looks sound. Fix the validator call before merging so the health check means what it says.

Pre-merge checks | 3 | 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Linked Issues check		For [`#18183`], the shared `bm25_score_terms` implementation is used by `HybridScorer` and `BM25Scorer`. The added tests pin scores and ranking and guard against a second formula. For [`#17599`], the reta…	For [`#17599`], carry each required capability into the retained implementation, or add explicit documentation that identifies each dropped capability as unwanted and gives its reason. Add or update automated tests for any retained capability…
Docstring Coverage		Docstring coverage is 42.11% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 38 functions across 11 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly identifies this as merge train `#5` and highlights the main change: using one BM25 formula for issue `#18183`.
Out of Scope Changes check		The changes remain connected to [`#18183`] and [`#17599`]. The shared BM25 helpers, scorer tests, duplicate-formula guard, hybrid-search guard, validator migration, obsolete configuration removal, visibil…

Full details: Linked Issues check

Explanation

For [#18183], the shared bm25_score_terms implementation is used by HybridScorer and BM25Scorer. The added tests pin scores and ranking and guard against a second formula. For [#17599], the retained HybridSearcher uses BM25 and system_validator uses kb.search in hybrid mode. However, deleting HybridSearchEngine also deletes keyword extraction, duplicate removal, fallback search, explanations, and the global accessor. The reviewed changes do not establish that each dropped capability was carried over or explicitly recorded as unwanted with a reason. The statement that the engine had zero callers does not record the disposition of those capabilities.

Resolution

For [#17599], carry each required capability into the retained implementation, or add explicit documentation that identifies each dropped capability as unwanted and gives its reason. Add or update automated tests for any retained capability.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

Commit to this branch
Create a new PR

🧪 Generate unit tests (beta)

Commit to this branch
Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-10-12T04:21:27Z

✅ SSOT Configuration Compliance: Passing

🎉 No new hardcoded values of either class — ssot and other both block.

Known backlog in pipeline-scripts/hardcoded_values_baseline.txt is suppressed and tracked in #14371.

coderabbitai

Actionable comments posted: 7

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: mrveiss/AutoBot-AI/.coderabbit.yaml
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 67c8a282-2610-451a-a599-63ef7ce11b0a

📥 Commits

Reviewing files that changed from the base of the PR and between f194061 and 78c666b.

⛔ Files ignored due to path filters (2)

repo_tests/python_file_size_ratchet_baseline.py is excluded by !repo_tests/python_file_size_ratchet_baseline.py
scripts/python_file_size_known_large.py is excluded by !scripts/python_file_size_known_large.py

📒 Files selected for processing (13)

autobot-backend/autobot_memory_graph/semantic_search.py
autobot-backend/knowledge/search.py
autobot-backend/knowledge/search_components/bm25.py
autobot-backend/tests/knowledge/search_components/bm25_shared_18183_test.py
autobot-backend/utils/hybrid_search.py
autobot-backend/utils/system_validator.py
autobot-infrastructure/shared/config/complete.yaml
repo_tests/_reach_policy.py
repo_tests/kb_read_visibility_allowlist.py
repo_tests/kb_read_visibility_guard_test.py
repo_tests/one_bm25_formula_18183_test.py
repo_tests/one_hybrid_search_17599_test.py
repo_tests/reach_scope_claim_17844_test.py

💤 Files with no reviewable changes (2)

autobot-infrastructure/shared/config/complete.yaml
autobot-backend/utils/hybrid_search.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

… exercises HybridSearcher (#17599) KnowledgeBase.search selects its enhanced pipeline (the only one that honours mode) when limit/tags/category are passed; with top_k alone it takes _search_basic and drops mode, so the validator never reached HybridSearcher and #17599's 'system_validator validates the retained implementation' was not met. Pass limit=5 and read results from the dict response. Found by CodeRabbit on train #18358; verified at search.py (enhanced = limit is not None ...).

mrveiss added 4 commits October 12, 2026 00:10

refactor(search): one BM25 formula shared by the knowledge base and t…

54e1f0c

…he memory-graph scorer (#18183)

refactor(rag): remove the validator-only HybridSearchEngine; the syst…

5eb3d7b

…em validator exercises the canonical BM25+RRF HybridSearcher (#17599)

refactor(search): memory-graph HybridScorer.bm25_score calls the shar…

2e5d1b6

…ed BM25 loop; the #18143 deferral is closed (#18183)

Merge #18348 @ 2e5d1b6 into train-2026-10-12-5 (hub union)

78c666b

coderabbitai Bot reviewed Oct 12, 2026

View reviewed changes

mrveiss mentioned this pull request Oct 12, 2026

chore(train): merge train 2026-10-12 #6 — 5 approved PRs, stacked on #18358 (#18293) #18364

Closed

mrveiss mentioned this pull request Oct 12, 2026

tech-debt(search): BM25/hybrid follow-ups from train #18358 — avg_doc_length guard, stale-exemption check, behavioural validator test #18369

Open

5 tasks

mrveiss merged commit 13be1b4 into main Oct 12, 2026
80 checks passed

mrveiss deleted the train-2026-10-12-5 branch October 12, 2026 06:07

This was referenced Oct 12, 2026

tech-debt(search): a second BM25 implementation lives in the memory-graph HybridScorer beside the knowledge BM25Scorer #18183

Closed

tech-debt(rag): two hybrid-search implementations — one reachable only from a validator #17599

Closed

Navigation Menu

Repository navigation

chore(train): merge train 2026-10-12 #5 — #18348 one BM25 (#18183) - #18358

chore(train): merge train 2026-10-12 #5 — #18348 one BM25 (#18183)#18358
mrveiss merged 5 commits into
mainfrom
train-2026-10-12-5

mrveiss commented Oct 12, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Oct 12, 2026 •

edited

Loading

Review limit reached

Walkthrough

Changes

❌ Failed checks (2 warnings)

Uh oh!

github-actions Bot commented Oct 12, 2026

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

mrveiss commented Oct 12, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Thinking Path

What Changed

Verification

Model Used

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Walkthrough

Changes

❌ Failed checks (2 warnings)

Uh oh!

github-actions Bot commented Oct 12, 2026

✅ SSOT Configuration Compliance: Passing

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

mrveiss commented Oct 12, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Oct 12, 2026 •

edited

Loading