refactor(rag): de-fork graph retrieval — one neighbour contract, one graph blend, shared config (#18139, #18141)

[mrveiss]

Thinking Path

Two issues share the RAG graph-retrieval files, so one branch and one PR. #18139: the get_neighbors contract was forked across PPR (to_node dicts, min_weight kwarg), the PostgreSQL adapter (tuples, no min_weight), staleness BFS and edge sync, so PPR wired with the adapter raised TypeError on first use. #18141: graph-RAG and the Neural Mesh retriever blended graph signals with different formulas, weights, config and rerank weights, and the mesh branch skipped the stale-chunk filter. Fix each at its root with one definition per concept.

What Changed

#18139

• services/mesh_brain/neighbor_store.py: the one NeighborStore Protocol, get_neighbors(node_id, min_weight=0.0) -> list[tuple[str, float]], and decode_neighbors.
• PPR, MeshDBAdapter, the Redis staleness adapter and MeshEdgeSync all conform; the duplicate MeshGraph and PPR NeighborStore Protocols are gone. The raw dict-row query on MeshDB is renamed fetch_neighbor_rows (it is not a neighbour-contract implementation).
• New test runs PPR over the real MeshDBAdapter. Before: MeshDBAdapter.get_neighbors(node_id) took no min_weight, so PPR's get_neighbors(node_id, min_weight=...) raised TypeError.

#18141

• knowledge/search_components/graph_blend.py: blend_graph_score, the one pre-rerank blend; compute_blended_score stays the one post-rerank blend.
• RAGConfig.graph_weight (env AUTOBOT_RAG_GRAPH_WEIGHT, registered) and graph_traversal_depth; defaults in constants/rag_graph_constants.py.
• Graph-RAG reads the shared config; lifespan no longer builds a private RAGConfig(timeout_seconds=10.0) or restates 0.3. The graph path keeps its 10 s default as the env-backed RAGConfig.graph_timeout_seconds field.
• NeuralMeshRetriever takes config, blends with blend_graph_score and passes config.rerank_weights (incl. MMR) on every rerank; rag_service passes its config. The mesh branch now runs _filter_stale_chunks, on RankedResult objects (reranker dicts are converted by MeshRetrievalResult.ranked_chunks).
• Traversal depth default declared once for graph-RAG search, TraverseRequest, and the multi-source graph request; PersonalizedPageRank.rank(max_hops=...) takes its default from the same module.

Behavioural differences resolved

Place	Before	Kept	Why
Graph-RAG blend	(1-w)*base + w*base*strength, w=0.3	same, now blend_graph_score(base, base*strength, w)	proven equal on a 4x11x11 grid
Mesh blend	score + ppr_boost (unweighted, unbounded)	convex blend_graph_score(score, ppr, w)	stays in 0-1, which the cross-encoder blend assumes; behaviour CHANGE, test updated
Mesh-only nodes	raw PPR score	w * ppr	same blend with base 0
Graph-RAG timeout	10 s via a private RAGConfig	10 s via RAGConfig.graph_timeout_seconds (AUTOBOT_RAG_GRAPH_TIMEOUT_SECONDS)	unchanged latency; the private config object was the fork, not the value. The general RAG budget stays at the shared 120 s
Mesh rerank	RerankWeights() default, no MMR	RAGConfig.rerank_weights	recurrence of #2103
Mesh stale filter	skipped	applied	recurrence of #4721
PPR moderate tier	docstring "1-hop", loads 3	behaviour kept (3), docstring corrected	changing hop count is a retrieval-quality change, not a de-fork

Not changed, on purpose: the depth fields defaulting to 1 and 6 (path length) in schemas_knowledge.py are different concepts; mesh SIMPLE tier returns un-reranked chunks by design; symbol_fork_baseline.json has no cluster for the removed duplicate Protocols (they were different names), so no row to lower. Ceilings lowered for schemas_knowledge.py (5155), lifespan.py (2338), graph_rag_service.py (738) in both records.

Verification

• pytest services/ tests/test_rag_rerank_weights.py tests/test_session_adaptive_reranker.py tests/test_mmr_diversity.py tests/memory_graph knowledge/search_components advanced_rag_optimizer_test.py: 3508 passed.
• services/mesh_brain: 181 passed before the blend commit; new rag_graph_blend_18141_test.py: 12 passed.
• repo_tests: symbol_fork_ratchet_17312, python_file_size_ratchet, ceiling_record_parity_17872, guard_reach_meta: pass; reach_scope_claim_17844 alone: 10 passed.
• A wider knowledge/ run showed egress_vendor_behaviour_test.py failures only under ordering ("Event loop is closed"); in isolation it passes (105 passed with the graph-RAG and memory-graph tests). knowledge/chat_knowledge_system_e2e_test.py needs live services and was not run.
• Pre-push passed its pytest selection with no override.

Closes #18139
Refs #18141 (blend, config, rerank weights, stale filter and depth done; KAG has no retrieval branch to test, only the dispatcher routes it)

Model Used

Claude Sonnet 5.5

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 47 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: Repository: mrveiss/AutoBot-AI/.coderabbit.yaml
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: a30184dd-57f1-4a4f-8004-c15771cb506e

---

📥 Commits

Reviewing files that changed from the base of the PR and between df211d3 and 3b120cf.

---

⛔ Files ignored due to path filters (2)

• repo_tests/python_file_size_ratchet_baseline.py is excluded by !repo_tests/python_file_size_ratchet_baseline.py
• scripts/python_file_size_known_large.py is excluded by !scripts/python_file_size_known_large.py

---

📒 Files selected for processing (26)

• autobot-backend/api/graph_rag.py
• autobot-backend/api/schemas_knowledge.py
• autobot-backend/constants/rag_graph_constants.py
• autobot-backend/initialization/lifespan.py
• autobot-backend/knowledge/search_components/graph_blend.py
• autobot-backend/services/graph_rag_service.py
• autobot-backend/services/graph_rag_service_test.py
• autobot-backend/services/mesh_brain/edge_sync.py
• autobot-backend/services/mesh_brain/edge_sync_test.py
• autobot-backend/services/mesh_brain/mesh_db.py
• autobot-backend/services/mesh_brain/mesh_db_adapter.py
• autobot-backend/services/mesh_brain/mesh_db_adapter_test.py
• autobot-backend/services/mesh_brain/mesh_db_test.py
• autobot-backend/services/mesh_brain/neighbor_store.py
• autobot-backend/services/mesh_brain/ppr.py
• autobot-backend/services/mesh_brain/ppr_test.py
• autobot-backend/services/mesh_brain/staleness_propagator.py
• autobot-backend/services/neural_mesh_retriever.py
• autobot-backend/services/neural_mesh_retriever_test.py
• autobot-backend/services/rag_config.py
• autobot-backend/services/rag_graph_blend_18141_test.py
• autobot-backend/services/rag_mesh_results.py
• autobot-backend/services/rag_service.py
• autobot_shared/env_registry_ai.py
• autobot_shared/env_registry_rag.py
• docs/developer/ENV_VARS.md

---

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Notice: 29 open PRs — past the runaway threshold (25)

This is not a request to defer this PR. The actual limit is the open-PR cap (AUTOBOT_OPEN_PR_CAP, default 40), enforced at pre-push against NEW branches only -- pushing more commits to this PR's own branch is never blocked by it.

This notice only means the count is high enough to be worth a glance for a runaway — something opening PRs in a loop, or a merge pipeline that has stalled so nothing is draining.

Currently open:

• refactor(captcha): one CaptchaType in autobot_shared (#18131) #18418 refactor(captcha): one CaptchaType in autobot_shared (tech-debt(captcha): two captcha_solver modules whose CaptchaType enums disagree — a kind one can classify is UNKNOWN to the other #18131)
• fix(memory): host context calls get_related_entities, the canonical graph traversal (#18140) #18416 fix(memory): host context calls get_related_entities, the canonical graph traversal (bug(memory): security memory integration calls traverse_relations, which the memory graph does not have #18140)
• feat(guard): a kit component cannot be forked in either app, and kit stories render in both Storybooks (#14776) #18409 feat(guard): a kit component cannot be forked in either app, and kit stories render in both Storybooks (Retire the local Base* component forks in autobot-frontend into the shared kit #14776)
• feat(agents): read-only agent-ledger APIs for Company OS and the SLM (#18391) #18408 feat(agents): read-only agent-ledger APIs for Company OS and the SLM (feat(agents): a GUI for the agent ledger — who is working on what, per company and install-wide in the SLM #18391)
• fix(kb): add_document chunks, partial-success ingest, stable upload rejection codes (#17830) #18405 fix(kb): add_document chunks, partial-success ingest, stable upload rejection codes (bug(kb): add_document never chunks — a whole transcript or PDF becomes one embedding, against its own docstring #17830)
• feat(approvals): one approval record, shared RiskTier, pending inbox query (#13421 S1) #18404 feat(approvals): one approval record, shared RiskTier, pending inbox query (ui(approvals): five approval surfaces with no shared record or component — build one approval inbox #13421 S1)
• feat(llm): needs_reauth registry skip + typed error, header headroom harvest, pre-dispatch headroom routing (#15022, #15026, #15028) #18403 feat(llm): needs_reauth registry skip + typed error, header headroom harvest, pre-dispatch headroom routing (llm: degradation store cannot distinguish a dead credential from a transient failure — dead tokens retry every TTL forever #15022, llm: harvest provider rate-limit response headers into a cross-worker quota headroom store (the missing quota monitor) #15026, llm: route on observed headroom before dispatch instead of on a 429 after it; reconcile guessed RPM buckets with observed limits #15028)
• fix(slm-auth): SSO callbacks redirect with a one-time code, not the session JWT (#18394) #18399 fix(slm-auth): SSO callbacks redirect with a one-time code, not the session JWT (security(sso): SSO callbacks redirect with the session JWT in the query string — now live after #18322 #18394)
• refactor(api): the main app's raw fetch tail goes through fetchWithAuth, and ApiClient gains an opt-in circuit breaker for polled dashboards (#12363) #18398 refactor(api): the main app's raw fetch tail goes through fetchWithAuth, and ApiClient gains an opt-in circuit breaker for polled dashboards (canonical(frontend): no single backend contract — 8 independent HTTP clients (2 generic) + 22 raw fetch sites, and generated api.ts is bypassed (4 importers vs 74 hand-typed responses) and stale (2424 vs 2152 paths) #12363)
• fix(guards): mypy resolves autobot_shared so enum aliases stay types; the embedding-literal guard covers namespaced ids and the infra builders (#18345) #18393 fix(guards): mypy resolves autobot_shared so enum aliases stay types; the embedding-literal guard covers namespaced ids and the infra builders (types: an enum alias of the canonical Severity used as an annotation is 'not valid as a type' in the backend mypy run (6 instances) #18345)
• test(tools): differential test of the tool-approval planes (#13416) #18387 test(tools): differential test of the tool-approval planes (test(security): the two tool-approval planes are never compared — add a differential assertion #13416)
• refactor(infra): classify 43 more infra scripts by runtime, repair 3 validators (#18289) #18380 refactor(infra): classify 43 more infra scripts by runtime, repair 3 validators (infra: 98 more infrastructure modules import backend-only packages — classify each as shared, backend tool or dead (#18288 follow-up) #18289)
• feat(shell): Refined UI step 2: top nav and icon cluster, Minimal/Full mode, Appearance settings, one brand mark (#18240) #18373 feat(shell): Refined UI step 2: top nav and icon cluster, Minimal/Full mode, Appearance settings, one brand mark (Refined UI step 2 · shell: top nav, UI mode, profile appearance #18240)
• security(api): RUM split into admin, user and bounded-open routes, and Prometheus metrics behind an SLM-issued scrape token (#16375) #18372 security(api): RUM split into admin, user and bounded-open routes, and Prometheus metrics behind an SLM-issued scrape token (security(api): 44 config-registered routers declare no auth gate, triaged by exposure (found by widening the router-auth guard, #16368) #16375)
• refactor(config): the #18094 de-fork remainder — one key reader, one feature flag, one language detector, and image-gen keys vault-first through an origin-bound resolver (#18094) #18371 refactor(config): the tech-debt: forked env defaults, shadowed constants and same-name functions — the remainder after #17419/#16177/#18093 #18094 de-fork remainder — one key reader, one feature flag, one language detector, and image-gen keys vault-first through an origin-bound resolver (tech-debt: forked env defaults, shadowed constants and same-name functions — the remainder after #17419/#16177/#18093 #18094)
• fix(codebase-analytics): honest scan outcomes, duplication detectors wired, and forks shown from the one CI detector (#18013, #18067, #17982) #18362 fix(codebase-analytics): honest scan outcomes, duplication detectors wired, and forks shown from the one CI detector (bug(codebase-analytics): /duplicates returns a fabricated "timed out after 120s" in 0.2s — the single-flight lock is never released #18013, Codebase analytics reports 18,200 problems and zero duplication, while the fork detector finds 365 clusters in the same tree #18067, bug(codebase-analytics): duplicate detection reports 'nothing scanned' as a completed scan — the one tool the dedup programme needs returns nothing for AutoBot's own codebase #17982)
• refactor(audit): audit_logs is the one system of record, Redis its projection — three stores retired, one write path, one retention table (#17225) #18343 refactor(audit): audit_logs is the one system of record, Redis its projection — three stores retired, one write path, one retention table (tech-debt(audit): #6475 closed on three parallel audit systems — there are now six #17225)
• security(auth): DB role permissions work, one strict-union permission gate for both services, MFA-pending tokens refused, SSO mints from the real user (#18259, #18321, #18322) #18338 security(auth): DB role permissions work, one strict-union permission gate for both services, MFA-pending tokens refused, SSO mints from the real user (security(rbac): UserService.get_user_permissions reads role.Permission, which does not exist — the DB-backed permission path raises and the middleware denies #18259, security(slm): get_current_user accepts MFA-pending temp tokens as full sessions (#17049 fixed only the backend) #18321, fix(slm-sso): SSO login passes a stand-in user without is_platform_admin to create_token_response, which raises #18322)
• chore(deps): bump the all-dependencies group across 1 directory with 15 updates #18273 chore(deps): bump the all-dependencies group across 1 directory with 15 updates
• fix(sandbox): execution and pooled containers run fully hardened, and no caller can widen their reach (#17267) #18232 fix(sandbox): execution and pooled containers run fully hardened, and no caller can widen their reach (refactor(sandbox): container hardening is copy-pasted across three Docker clients, with a docstring as the only thing preventing drift #17267)
• chore(deps): bump the all-dependencies group in /autobot-infrastructure/autobot-npu-worker/docker with 3 updates #18177 chore(deps): bump the all-dependencies group in /autobot-infrastructure/autobot-npu-worker/docker with 3 updates
• chore(deps): bump the all-dependencies group in /autobot-infrastructure/shared/docker/ai-stack with 4 updates #18176 chore(deps): bump the all-dependencies group in /autobot-infrastructure/shared/docker/ai-stack with 4 updates
• chore(deps): bump actions/upload-artifact from 7.0.1 to 7.0.2 in the all-dependencies group #18175 chore(deps): bump actions/upload-artifact from 7.0.1 to 7.0.2 in the all-dependencies group
• chore(deps): update asyncpg requirement from >=0.31.0 to >=0.32.0 in /autobot-slm-backend in the all-dependencies group #18172 chore(deps): update asyncpg requirement from >=0.31.0 to >=0.32.0 in /autobot-slm-backend in the all-dependencies group
• chore(deps): bump the all-dependencies group across 1 directory with 16 updates #18157 chore(deps): bump the all-dependencies group across 1 directory with 16 updates
• chore(deps): update aiohttp requirement from >=3.14.3 to >=3.14.4 in /autobot-backend/code_analysis in the all-dependencies group across 1 directory #18156 chore(deps): update aiohttp requirement from >=3.14.3 to >=3.14.4 in /autobot-backend/code_analysis in the all-dependencies group across 1 directory
• chore(deps-dev): bump the oxlint group across 1 directory with 2 updates #18152 chore(deps-dev): bump the oxlint group across 1 directory with 2 updates
• chore(deps): update transformers requirement from >=5.18.0 to >=5.19.0 in /autobot-tts-worker in the all-dependencies group across 1 directory #18150 chore(deps): update transformers requirement from >=5.18.0 to >=5.19.0 in /autobot-tts-worker in the all-dependencies group across 1 directory
• chore(deps-dev): bump the all-dependencies group across 1 directory with 2 updates #18149 chore(deps-dev): bump the all-dependencies group across 1 directory with 2 updates

If the queue is draining normally, ignore this. Otherwise:

1. Check whether CI is dispatching at all — see the ci-dispatch-watchdog status on these PRs
2. Merge the ones whose CI has finished and review has passed: gh pr merge <number> --squash --delete-branch
3. Look for a loop opening near-identical PRs

Warn-only runaway detector — .github/workflows/pr-queue-gate.yml. It never blocks a merge.

[github-actions]

✅ SSOT Configuration Compliance: Passing

🎉 No new hardcoded values of either class — ssot and other both block.

Known backlog in pipeline-scripts/hardcoded_values_baseline.txt is suppressed and tracked in #14371.

[mrveiss]

Review (Sonnet) at 8522b85: changes requested (one Medium). Every AC of #18139 is met:

• one NeighborStore.get_neighbors(node_id, min_weight) contract (mesh_brain/neighbor_store.py);
• PPR (ppr.py:155), staleness (staleness_propagator.py:129), MeshDBAdapter (mesh_db_adapter.py:75) and edge sync (edge_sync.py:74) all conform;
• the duplicate Protocol is gone;
• PPR runs against the real adapter in ppr_test.py:188-198;
• the rename to fetch_neighbor_rows leaves no stale caller.

For #18141: one blend_graph_score, plus AUTOBOT_RAG_GRAPH_WEIGHT, registered with range 0–1. Behaviour changes (a), (b) and (d) are intended and tested, and the size baselines only shrink.

Severity	Location	Finding	Fix
Medium	lifespan.py:~1159	Removing RAGConfig(timeout_seconds=10.0) moves the graph-RAG timeout from 10s to the shared timeout_seconds. rag_service.py:767 uses timeout or config.timeout_seconds, and graph_rag_service.py:493 passes timeout=None by default. Both the /search handler (api/graph_rag.py:188, request timeout defaults to None) and retrieval_dispatcher.get_context (via knowledge_rag.py:~570, which sends no timeout) await it inline. So a default user request can now wait up to the shared ceiling instead of 10s. No issue or owner decision covers that.	Give graph-aware search its own env-backed default (keeping 10s), registered, read by both paths. Otherwise state the 10s → shared change in the body as an owner decision.
Low	api/graph_rag.py:352	The metrics payload still hardcodes "timeout_seconds": 10.0, so it reports a value that no longer applies.	Read it from get_rag_config().
Low	rag_graph_blend_18141_test.py:76	The r.metadata-on-dict crash fix is covered only indirectly.	Assert that metadata and chunk_id survive coerce_ranked_results.

[mrveiss]

Delta review at 3b120cf: approve. The timeout Medium is resolved.

• Default restored: graph search defaults to RAGConfig.graph_timeout_seconds, built from AUTOBOT_RAG_GRAPH_TIMEOUT_SECONDS (10.0) and validated > 0 (rag_config.py:68,195). graph_rag_service.py:246 applies timeout or config.graph_timeout_seconds, so /search and the dispatcher are back to 10s and an explicit caller timeout still wins.
• Registry: the new autobot_shared/env_registry_rag.py is loaded through env_registry_ai.py:33, so its vars are registered, and ENV_VARS.md is updated.
• Metrics: get_metrics reports the real value (graph_rag_service.py:730). The old hardcoded 10.0 is now a placeholder in a docstring example.
• Tests: default, env override, and the caller-timeout precedence. The graph_rag_service ceiling went 738 → 734.

Queued for the next merge train.