Repository navigation
Capability audit: LLC adapter layer + voice pipeline — confirmed gaps #13096
Description
Activity
Audit close-out — two remaining items resolved
DISPROVED: retry is never applied around a streaming response
Suspected hazard: a retry wrapper re-attempting a call after SSE bytes were
already consumed would replay already-emitted text/tool deltas to the client.Checked and the invariant holds — no fix needed:
llm_shared/base_provider.py:157—execute_with_retry(_attempt, …)wraps
chat_completion(), which returns a fully-bufferedLLMResponse. Retrying a
buffered response cannot replay partial deltas.stream_completion()(base_provider.py:246) is a separate abstract method and
is not wrapped by the backoff handler.- No retry/backoff wrapper on any provider's
stream_completionimplementation. fallback_chain.pyandmodel_fallback_coordinator.pycontain no streaming
references — fallback operates on non-streaming completions only, so there is no
mid-stream provider switch that could replay output.
Recorded so this is not re-investigated. If streaming retry or mid-stream fallback
is ever added, this invariant must be re-asserted at that point.CONFIRMED and filed: #13110 — vision not wired into the voice loop
The one item left un-audited in the original pass. Confirmed: the voice path has
no visual-input reference at all (everyframehit in the voice composables is an
audio frame — the AudioWorklet render quantum). All the vision components exist;
nothing connects them to the voice loop. Filed as #13110.Umbrella task list updated below.
Two further findings filed after reconciling every item from the audit against what was actually opened:
- LLC scheduler: only rate-limit failures have a recovery path — every other failure is an opaque str(exc) retried on cron #13116 — only rate-limit failures have a designed recovery path (
heartbeat_scheduler.py:20-24,71-72); every other failure recordserror=str(exc)(:498-501) and is retried on the next cron tick with the same outcome. Sibling of LLC adapters: no read-only sign-in probe before dispatch — signed-out agents burn scheduler slots #13098 / LLC adapters: no stall or first-output watchdog — a hung run holds a slot for the full 3600s timeout #13099, which supply two of the classes. - EncryptionService accepts a weak master key with only a log warning #13117 —
EncryptionServiceaccepts a weak master key with only alogger.warning(encryption_service.py:53-59) and proceeds to derive keys from it. Filed outside this umbrella as a security-configuration issue.
#13110 re-scoped into a vision epic (#13119-#13123) — see its own thread.
- LLC scheduler: only rate-limit failures have a recovery path — every other failure is an opaque str(exc) retried on cron #13116 — only rate-limit failures have a designed recovery path (
Sequence map: this umbrella is track U#13096 in #13483, which places all 282 open issues into ordered tracks.
Derived order for this track's 9 open children:
1. #13097 [P0] LLC adapters: agent cancel/timeout orphans the whole subprocess tree 2. #13098 [--] LLC adapters: no read-only sign-in probe before dispatch — signed-out agen 3. #13099 [--] LLC adapters: no stall or first-output watchdog — a hung run holds a slot 4. #13100 [--] LLC adapters: no version floor or compat assertion on any external agent C 5. #13101 [--] LLC adapters: commentary vs final answer is tag-based, so it cannot work f 6. #13102 [--] Voice: TTS text is not shaped for speech — markdown, file paths, URLs and 7. #13103 [--] Voice: first audio waits for a full terminated sentence — no sub-sentence 8. #13105 [--] Voice: end-of-turn is purely temporal — a mid-sentence pause commits the t 9. #13116 [--] LLC scheduler: only rate-limit failures have a recovery path — every otherHow much to trust this: track membership is reliable. The order is derived from stated blockers (only 9 exist backlog-wide), then priority label, then issue number — it is a starting default, not a proven critical path. Where a child carries its own
## Implementation order, that wins.64% of open issues have no priority label, which is why some rows sort last for no good reason — see #13485.
- added sub-issues
on Sep 1, 2026
Context
A capability audit of two AutoBot subsystems — the LLC external-agent adapter
layer and the voice conversation pipeline — against established practice for
each. This umbrella collects every confirmed gap, each verified by reading the
code and recorded with the grep/file evidence in the child issue.
Findings are grouped by subsystem. Every item below was confirmed absent, not
assumed absent.
What the audit confirmed we already do well
Recorded so these are not re-litigated:
/api/voice/clone-voiceis permission-gated andaudit-logged on both denied and success paths (
autobot-backend/api/voice.py:253-281)sequential fallback queue (fix(voice): assistant text dropped/truncated — fallback aborts prior audio + cursor drift + tail not flushed #12502)
realtime-WebRTC (feat(voice): WebRTC Realtime mode in voice conversation UI #7345)
messaging (voice: zero voices → silent TTS failure with no user-visible message (no tts-worker in small topologies) #9999), untrusted-cert mic diagnostics (Voice: Hands-free and full-duplex modes fail with 'Mic access required' on self-signed cert #1059), 3-tier CLI PATH
resolution for service accounts (LLC: claude_code adapter can't run on deployed instance — 'claude' CLI not on service PATH (heartbeat run skipped) #12478)
coordinator, cross-worker rate limiting, circuit breakers; 429/503/529 handled
--session-idpassed explicitly so the stored idis one the CLI actually knows (LLC claude_code adapter: every agent run fails on Claude 2.1.219 — fresh cmd missing --verbose (stream-json+print requires it), and resume hard-fails on stale session id with no fresh fallback #12683)
Tasks
LLC adapter layer
scheduler slots every heartbeat
full 3600 s timeout
external agent output
failure is an opaque
str(exc)retried on cronVoice pipeline
phantom turn
streamed playback starves and stutters (live bug, reopened; PR fix(voice): pre-roll audio against a below-real-time TTS worker + make the real-time factor observable (#12460) #13736
delivers the client pre-roll + real-time-factor telemetry, host load
remains)
the SLM admin tool, the LLM vision path exists (feat(chat): vision-in-the-loop — thread browser/VNC screenshots into LLM chat context #11538), nothing connects them
Filed separately (not a capability gap)
EncryptionServiceaccepts a weak master key with only a logwarning; a missing key correctly raises but a weak one proceeds. Security
configuration, so it sits outside this umbrella's task list.
Disproved during the audit
deltas) — checked, does not occur. Retry wraps only the buffered
chat_completion()path;stream_completion()is unwrapped and the fallbackchain has no streaming path. Detail in the close-out comment below.
Deliberately NOT proposed
Recorded with reasons so they are not re-raised without new information:
one means a new worker-VM deployment plus an extra hop inside the turn path. The
latency and ops load plausibly cancel the accuracy gain in our topology. The
cheap heuristic (V4) captures most of the value. Revisit only with measurements.
version of that pattern only works where a normalizing protocol sits in front of
every agent. Ours carry genuine per-agent logic — CLI flags, credential
injection, session resume, rate-limit output scanning, token parsing — precisely
because there is no such protocol. Flattening would relocate that logic somewhere
worse. The precondition is a protocol, not a refactor. See discovery(orchestration): AgentRegistry fragmentation — 4 different implementations + 1 Protocol with same name #6828 for the prior
round of this.
output, in-flight permission relay, and mid-session model switching — real
capabilities our detached headless-CLI model cannot offer. But it would mean
depending on a protocol under a year old, with exact-pinned third-party adapter
versions. Our current model is less capable and materially more stable. Worth
re-evaluating in ~2 quarters, not now.
Sequencing
#13097 first — it is a live bug, and #13099 (stall-kill) makes the orphaning worse until
it is fixed. Then #13098 (highest value-per-line), then #13102 + #13104 (most
user-visible), then #13100, #13103, #13105, #13101. #13110 is now its own epic
(#13119-#13123) and runs in parallel — de-fork + wire-in work, not new construction.