fix(unwired): wire ClaimVerifier, FailurePatternDetector, self-improvement loop, NPU events

Task 6 — Wire in unwired services & the self-improvement loop

Theme: Built-but-unwired. Whole services and the self-improvement write-path are implemented but have zero production callers (matches the prior Ornith-1.0 "read wires disconnected" finding). Per project policy these are wire-in, never deletion.

Problem & subtasks

6.1 Wire ClaimVerifier into grounding/agent flow — services/claim_verifier.py:121 (zero non-test callers)
- Step: invoke from GroundedAgent/knowledge_grounding; surface requires_human_review
- Step: dedup with services/claim_classifier.py:59 (also unused; GroundedAgent._extract_claims:338 reimplements it inline)
6.2 (DONE — get_pattern_detector() wired into workflow_runner failure path) Wire FailurePatternDetector into error/task-completion flow — services/failure_pattern_detector.py:78,361 (singleton never accessed)
- Step: call detect_pattern/learn_pattern from a central error handler or task-completion hook
6.3 Reconnect the self-improvement WRITE path — judges/task_outcome_judge.py:41, agents/task_pattern_learner.py:51
- Step: the read side (agents/agent_orchestration/routing.py:_check_learned_strategy) reads learned strategies from Redis, but nothing writes them. Both writers are dead: phase_progression_manager.py:748 (callers execute_automated_progression/trigger_manual_progression have zero callers) and llc/kb/diary_writer.py:248 (nothing ever sets context_snapshot["outcomes"])
- Step: feed real outcome dicts into diary context_snapshot["outcomes"], OR schedule execute_automated_progression
- Step: verify learned strategies now appear in Redis and influence routing
6.4 (DONE via feat(npu): publish worker added/status/metrics events + avg_response_time_ms (#10602 6.4, #10698) #10951) Publish missing NPU worker events — services/npu_worker_manager.py:991
- Step: subscribers exist (api/websockets.py:978, services/npu_pipeline/invalidation.py:57) for npu.worker.added/status.changed/metrics.updated but only removed is published — emit the other three at the manager mutation sites so cache invalidation + live frontend updates fire
6.5 Consolidate dual subagent systems / enable reflection — enhanced_orchestration/subagent_dispatcher.py:71,129,232 vs services/agents/subagent_spawner.py:88
- Step: get_subagent_dispatcher() has zero callers and enable_reflection defaults False & is never set; converge onto the live spawner and pass enable_reflection=True where valuable
6.6 Stop discarding agent-recommendation scores — enhanced_orchestration/agent_router.py:42-57
- Step: return scores so callers can threshold-gate/audit; feed orchestration/performance_tracker.py updates back into the recommender at runtime
6.7 Use lightweight_mode_used to short-circuit heavy path — llm_shared/models.py:138, chat_workflow/manager.py:576
- Step: when trivial-tier, skip expensive RAG fetch/rerank/validation instead of only tagging metadata

Verification

Each subtask: add a test proving the previously-dead path now executes (e.g. learned strategy written→read; NPU event received by subscriber)

Notes

ABSOLUTE: never delete this code — wire it in. Several items are independently shippable; split into separate PRs if a single PR grows too large.

Part of umbrella #10603 (AutoBot Efficiency & Precision).

Premise re-verification (2026-06-28, current base after py3.14 + 19 merged PRs)

Re-checked each claim against the current code before implementing. The base moved — several claims are now stale or deeper than originally recorded:

#	Item	Current status	What a real wire-in needs
1	ClaimVerifier / ClaimClassifier	Still unwired (`services/claim_verifier.py:121`, `claim_classifier.py:482` zero prod callers)	Replace GroundedAgent inline claim logic with these services — behavior-changing (grounding quality)
2	FailurePatternDetector	Still unwired — BOTH sides (`get_pattern_detector` `:364` no callers; no reader either)	Needs a WRITE hook (`learn_pattern` on failure, e.g. `workflow_runner._handle_workflow_execution_failure:315`) and a READER (`detect_pattern` at retry, or stats API) — writer alone = orphan data
3	Self-improvement write-path	Still unwired (`execute_automated_progression`/`trigger_manual_progression` zero callers; nothing sets `context_snapshot["outcomes"]`)	High effort: pick trigger (scheduler vs outcome-hook)
4	NPU events	Mostly already wired — `added`/`status.changed`/`removed` now published (`npu_worker_manager.py:893/775/991`). Only `metrics.updated` missing, and there is no metrics producer at all (`npu:worker:{id}:metrics` is only read/deleted)	Add a metrics WRITER first, then emit
5	Dual subagent systems	Both unwired (`get_subagent_dispatcher` AND `subagent_spawner` have zero prod callers)	Product decision on canonical system first
6	agent_router scores discarded	Confirmed (`agent_router.py:57`), path is live (`/agents/recommend`)	Additive, but scores only reach an API response with no current UI consumer
7	`lightweight_mode_used`	NON-ISSUE — the expensive-work short-circuit already works via the separate `lightweight_mode` bool (`chat_workflow/manager.py:2864` → `llm_handler.py:760/788`). The flagged field is a legitimate write-only UI cost indicator.	No action — removing from scope

Net: #7 drops out; #4 is deeper (needs a producer); #1/#2/#3/#5/#6 are genuine but each is a small feature (consumer + writer, sometimes a product call), not a one-line wire. Recommend tackling one at a time, each with its consumer, rather than a bulk PR.

feat(reliability): wire FailurePatternDetector into workflow failure …

feat(orchestration): surface agent recommendation scores (#10660) (#1…

Decision doc — §6.5 dual subagent systems (needs a canonical-choice decision)

There are two parallel-subagent implementations, both currently unwired in production (only test references). They have complementary strengths, so wiring requires picking the canonical one first.

	`SubagentDispatcher` (`enhanced_orchestration/subagent_dispatcher.py`)	`SubagentSpawner` (`services/agents/subagent_spawner.py`)
Purpose	"autonomous subagents for parallel workstreams"	"parallel subagents for independent tasks"
Reflection	✅ score-and-revise pass (`enable_reflection`)	❌
Depth/count safety	❌	✅ `MAX_SUBAGENT_DEPTH` + `MAX_SUBAGENTS_PER_PARENT`
State persistence	❌ (in-memory)	✅ Redis-backed (`pending_tasks`/`active_subagents`)
Partial-return	`spawn_parallel_tasks` (awaits all)	`wait_for_all` flag (fire-and-forget option)
Tests	`tests/orchestration/test_subagent_dispatcher_reflection.py`	(none found)

Your decision (pick one)

A — Spawner canonical, port reflection in (Recommended): Spawner has the production-safety guards (recursion depth + fan-out caps + Redis state) that any real multi-agent system needs; port the Dispatcher's reflection pass onto it as an opt-in, then retire the Dispatcher.
B — Dispatcher canonical, port guards in: keep reflection as the centerpiece, add depth/count caps + Redis state from Spawner.
C — Keep both for distinct call sites (more surface to maintain; not recommended).

Once you pick, the wire-in (into the orchestrator's parallel-workstream execution path) is a follow-up PR. Until then this stays blocked — I won't guess the architecture.

Decision docs — remaining §6.1 / §6.3 / §6.4 (need a design choice each)

(§6.2 FailurePatternDetector shipped in #10631; §6.6 agent scores shipped in #10661; §6.5 dual-subagent decision doc posted above; §6.7 lightweight_mode = non-issue.)

§6.1 — Wire ClaimVerifier into grounding

Gap: services/claim_verifier.py + claim_classifier.py have zero production callers; GroundedAgent reimplements claim extraction inline. But GroundedAgent itself isn't on the chat path (see §3.5), so wiring ClaimVerifier into GroundedAgent improves an unused component.
Options: A — replace GroundedAgent's inline logic with ClaimVerifier/ClaimClassifier (dedups; readies it for §3.5). B — defer until §3.5 decides whether GroundedAgent goes on the chat path.
Recommendation: A as a prep refactor (low user-visible risk, dedups), OR bundle with §3.5. Risk: low-med. Sequence with §3.5.

§6.3 — Reconnect the self-improvement write-path

Gap: TaskPatternLearner.learn_from_outcomes (the WRITE side of the learning loop) has two dead feeders: phase_progression_manager.execute_automated_progression/trigger_manual_progression (zero callers) and diary_writer reading context_snapshot["outcomes"] (never set). The READ side (routing reads learned strategies from Redis) works but nothing writes.
Options: A — set context_snapshot["outcomes"] at the point task outcomes are recorded, so diary_writer runs the learner (event-driven). B — schedule execute_automated_progression on a Celery beat cadence (batch). C — both.
Recommendation: A (event-driven, closes the loop at the natural write point) — but needs care to feed real outcome dicts. Risk: med-high (touches the agent outcome-recording path). Your call on trigger.

§6.4 — NPU `metrics.updated` event

Gap: subscriber exists (api/websockets.py), but npu.worker.metrics.updated is never published — and there is no metrics producer at all (npu:worker:{id}:metrics is only read/deleted; NPUWorkerMetrics is never written). Only avg_response_time is derivable from existing pulse data; success_rate/requests_per_minute need new request/error counters.
Options: A — build the producer: add request/error counters in the worker dispatch path, compute+store NPUWorkerMetrics on the health/pulse loop, then emit metrics.updated. B — emit partial metrics (only avg_response_time from pulse p95) now, fill the rest later.
Recommendation: A (do it properly) — but it's an instrumentation feature, and its runtime value depends on active NPU workers + a frontend consumer. Risk: med. Lowest priority of the three.

All await your pick before I implement.

feat(npu): produce + emit worker metrics (#10697) (#10699)

feat(npu): publish worker added/status/metrics events + avg_response_…

feat(agent): wire ClaimVerifier + self-improvement write-path + subag…

Delivered via PR #10990 — ClaimVerifier + self-improvement write-path + subagent reflection wired into hot paths (config-gated); FailurePatternDetector/NPU/scores/lightweight-mode confirmed already wired. 17 tests.

Navigation Menu

Repository navigation

fix(unwired): wire ClaimVerifier, FailurePatternDetector, self-improvement loop, NPU events #10602

Task 6 — Wire in unwired services & the self-improvement loop

Problem & subtasks

Verification

Notes

Premise re-verification (2026-06-28, current base after py3.14 + 19 merged PRs)

Decision doc — §6.5 dual subagent systems (needs a canonical-choice decision)

Your decision (pick one)

Decision docs — remaining §6.1 / §6.3 / §6.4 (need a design choice each)

§6.1 — Wire ClaimVerifier into grounding

§6.3 — Reconnect the self-improvement write-path

§6.4 — NPU `metrics.updated` event

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

fix(unwired): wire ClaimVerifier, FailurePatternDetector, self-improvement loop, NPU events #10602

Description

Task 6 — Wire in unwired services & the self-improvement loop

Problem & subtasks

Verification

Notes

Activity

mrveiss commented on Jun 28, 2026

mrveiss commented on Jun 28, 2026

Premise re-verification (2026-06-28, current base after py3.14 + 19 merged PRs)

mrveiss commented on Jun 29, 2026

Decision doc — §6.5 dual subagent systems (needs a canonical-choice decision)

Your decision (pick one)

mrveiss commented on Jun 29, 2026

Decision docs — remaining §6.1 / §6.3 / §6.4 (need a design choice each)

§6.1 — Wire ClaimVerifier into grounding

§6.3 — Reconnect the self-improvement write-path

§6.4 — NPU metrics.updated event

mrveiss commented on Jul 6, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

§6.4 — NPU `metrics.updated` event