test(voice): judge first audio by chunks released, not a wall-clock race (#16407)

[mrveiss]

Refs #16407 (AC2 delivered; AC1, the CI-history question, is still open, see the issue)

Thinking Path

• test_streaming_synthesize_reaches_first_audio_far_sooner failed on shard 8 of fix(ci): let the system-package retry recover from its own timeout, and fetch before unpacking (#15515) #16367: streaming time-to-first-audio was 1.238s, against 0.522s for blocking.
• Only one assertion there can flake: the upper bound streaming_ttfb < blocking_ttfb / 2, a limit on wall time on a shared runner. The blocking check is a lower bound on an asyncio.sleep, and a sleep never finishes early.
• 1.238s is more than twice the whole simulated utterance (0.5s). Buffering doesn't explain it either: a route that buffered everything would read about 0.5s. Before its first yield, the streaming path does a mocked request, one 0.1s paced read and a WAV parse. Something delayed the event loop on that runner, and I could not determine what.
• The fake worker already offers a signal that doesn't depend on load. _PacedStreamReader releases a chunk only when the route pulls one. When the first audio byte leaves, a streaming route has pulled exactly 1 chunk. The perf(voice): 4.7-10.1s to first audio — the voice path uses blocking /tts/synthesize while the streaming route sits unused #13215 defect, buffering the whole utterance, has pulled all CHUNK_COUNT.

What Changed

Only autobot-backend/api/voice_latency_test.py:

• _PacedStreamReader.released. _make_worker_http_client keeps each stream-route reader on client.stream_readers, read through _stream_progress(http_client).
• _time_to_first_byte samples an optional progress callable on each ASGI send and returns its value at the first body byte.
• test_streaming_synthesize_reaches_first_audio_far_sooner becomes test_streaming_synthesize_sends_first_audio_after_the_first_chunk. It asserts released == 1 instead of the wall-clock ratio. The blocking lower bound (>= 0.8 × SYNTH_SECONDS) stays.
• New contrast test, test_a_route_that_buffers_the_utterance_is_caught. It patches api.voice._framed_audio_chunks with a version that drains the stream before yielding, and asserts released == CHUNK_COUNT. This shows the new signal catches the defect it guards against.

Verification

• Not run locally: repo policy is not to run repository code, so CI's python-suite is the check. The file parses (AST), and ruff, black and isort are clean.
• Traced by hand:
  • Streaming: synthesize_stream calls readexactly(4), which releases frame 0, then reads the rest from the buffer and yields. StreamingResponse sends it with progress at 1.
  • Buffering: the route drains all 5 frames, and the next readexactly ends the stream before the first send, so progress is 5.
• AC1 (CI history): a sweep of recent failed python-suite shards for this test is running. The result will go on test(voice): voice_latency_test's streaming-vs-blocking TTFB race looks flaky under CI load (found triaging #16367) #16407.

Deliberately out of scope

• .test_durations still keys the old test name. test-durations.yml regenerates that file, and pytest-split tolerates stale keys.
• What stalled the event loop on that runner: I can't determine it from the code or the one log.

Model Used

Claude Opus 5 (claude-opus-5).

Single-issue rationale: a test-only fix to one voice-latency test; no other open issue shares this file or the voice test scope.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 795232e1-9c9a-43af-bac0-9f47b53fcad7

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Notice: 29 open PRs — past the runaway threshold (25)

There is no PR queue limit, and this is not a request to defer this PR. Work proceeds one issue at a time without a cap on open PRs; review capacity is the constraint.

This notice only means the count is high enough to be worth a glance for a runaway — something opening PRs in a loop, or a merge pipeline that has stalled so nothing is draining.

Currently open:

• fix(security): require sign-in on seven ungated routers, and admin where they change shared state or reach outside (#16375) #16418 fix(security): require sign-in on seven ungated routers, and admin where they change shared state or reach outside (security(api): 44 config-registered routers declare no auth gate, triaged by exposure (found by widening the router-auth guard, #16368) #16375)
• chore(ci): merge train 2026-09-12-a, test vehicle, do not merge (#15937) #16417 chore(ci): merge train 2026-09-12-a, test vehicle, do not merge (Enable the merge queue — no PR merges against a stale base #15937)
• feat(preflight): run every reproducible required check through the workflow's own command (#15933) #16416 feat(preflight): run every reproducible required check through the workflow's own command (Local preflight must cover the 10 required checks #15933)
• fix(guards): fold shell continuations in the hooksPath guard via one shared helper (#15961) #16414 fix(guards): fold shell continuations in the hooksPath guard via one shared helper (core.hooksPath override silently disables every hook in a worktree #15961)
• fix(security): deny an SLM token when its revocation check cannot run, HS256 and RS256 (#16387, #16412) #16413 fix(security): deny an SLM token when its revocation check cannot run, HS256 and RS256 (security(slm-auth): token revocation checks fail open when Redis errors, so a revoked token is accepted #16387, security(slm-auth): RS256 authority-token revocation check fails open when Redis errors #16412)
• fix(security): generate the vault root key from 32 random bytes, not alphanumeric characters (#16405) #16410 fix(security): generate the vault root key from 32 random bytes, not alphanumeric characters (security(secrets): the vault root key is base64 of 32 alphanumeric chars (~190 bits), not 32 random bytes #16405)
• chore(docker): retire the four agent Dockerfiles whose modules no longer exist; agents run in-process (#16361) #16409 chore(docker): retire the four agent Dockerfiles whose modules no longer exist; agents run in-process (chore(docker): four agent Dockerfiles under autobot-infrastructure/autobot-backend/docker/ are built by nothing: wire in or retire #16361)
• chore(deps): carry dependabot's npm security bumps onto Dev_new_gui (hono 4.13.7 and transitive; supersedes #16171) #16408 chore(deps): carry dependabot's npm security bumps onto Dev_new_gui (hono 4.13.7 and transitive; supersedes chore(deps): bump the npm_and_yarn group across 6 directories with 7 updates #16171)
• fix(tests): stop gating a disk-bound model load on a CPU work budget, and prove it stays offline and import-free (#15055, #15235) #16404 fix(tests): stop gating a disk-bound model load on a CPU work budget, and prove it stays offline and import-free (test(perf): startup benchmarks assert absolute wall-clock budgets and fail on runner load — passed and failed on the same code minutes apart #15055, test(perf): three calibrated budgets are drifting or disk-bound — the startup margin is 1.4x, not the 1.9x its note records #15235)
• fix(ci): keep the drift-checked SLM agent mirror out of the duplication scan (#16401) #16402 fix(ci): keep the drift-checked SLM agent mirror out of the duplication scan (ci(duplication-guard): the SLM scope counts the drift-checked agent mirror, so every agent change fails the absolute pin #16401)
• fix(ci): bring CI's pytest environment up to its declared floors, and gate on them (#16264) #16391 fix(ci): bring CI's pytest environment up to its declared floors, and gate on them (ci: the pytest environment sits below two declared floors, and the floor banner tells CI readers the opposite #16264)
• fix(security): the dev proxy stops granting admin without a login, and the session check endpoint is tested (#16382, #16385) #16390 fix(security): the dev proxy stops granting admin without a login, and the session check endpoint is tested (SLM Vite dev-server proxy injects X-Internal-API-Key on /autobot-api with no session check #16382, SLM backend: add direct test coverage for GET /api/auth/me #16385)
• fix(security): the SLM proxy checks the caller's SLM session before it adds the backend key (#16374) #16386 fix(security): the SLM proxy checks the caller's SLM session before it adds the backend key (security(slm): the SLM nginx /autobot-api/ proxy adds the internal API key for every caller, so anyone reaching the SLM host is a backend admin #16374)
• fix(ci): retire the CodeQL suite file nothing loads, and record the real guard behind alert #1030 (#16283, #16302) #16377 fix(ci): retire the CodeQL suite file nothing loads, and record the real guard behind alert feat: Voice conversation mode — Hands-free with VAD (Tier 2) #1030 (ci(security): codeql.yml is disabled, so the reviewed codeql-config.yml exclusions apply to nothing #16283, ci(security): codeql-config.yml's file-scoped query-filters exclude nothing, under either setup #16302)
• fix(security): write the service-key export only to its configured directory, never into a checkout, and keep one (#16348) #16372 fix(security): write the service-key export only to its configured directory, never into a checkout, and keep one (security(service-auth): generate_service_keys.py writes its live-key export relative to the working directory, so a run inside a checkout drops keys into the work tree #16348)
• fix(security): authenticate every skills route, and require admin to change, fetch or execute (#16368) #16370 fix(security): authenticate every skills route, and require admin to change, fetch or execute (security(api): every route in api/skills.py (catalog fetch/install, enable, execute) is unauthenticated, and the router-auth guards can't see feature routers #16368)
• fix(ci): let the system-package retry recover from its own timeout, and fetch before unpacking (#15515) #16367 fix(ci): let the system-package retry recover from its own timeout, and fetch before unpacking (ci(runner): apt provisioning times out (exit 124) three attempts in a row and fails the shard before any test runs #15515)
• fix(guards): tell a baseline entry that matches nothing apart from one that under-matches (#16334) #16365 fix(guards): tell a baseline entry that matches nothing apart from one that under-matches (bug(guards): --audit-baseline says "no longer match anything" for an entry that still matches, so the reader deletes a live exemption #16334)
• fix(ci): stop the regen bot's approved runs cancelling the job that approved them (#16360) #16363 fix(ci): stop the regen bot's approved runs cancelling the job that approved them (ci: the regen bot's own push cancels the job that approves its parked runs, so a PR's CI can silently never run #16360)
• fix(ci): keep line numbers out of the secrets baseline, so a line move no longer rewrites it (#16353) #16362 fix(ci): keep line numbers out of the secrets baseline, so a line move no longer rewrites it (ci(secrets): .secrets.baseline stores a line number per finding, so every PR that shifts lines in a file with findings rewrites it and conflicts with every other such PR #16353)
• test(guards): name the deliberate fixed sleeps, and stop a random temp name deciding a language test (#16255, #16341) #16358 test(guards): name the deliberate fixed sleeps, and stop a random temp name deciding a language test (tests: 32 more fixed sleeps are followed by an assertion, and some decide the outcome by the clock #16255, test(voice): test_no_filename_hint flakes when tempfile's random name forms a language-code token (e.g. _lv_) #16341)
• fix(ci): gate the frontend npm audit on one report, retry it, and report 'could not check' apart from advisories (#16337) #16357 fix(ci): gate the frontend npm audit on one report, retry it, and report 'could not check' apart from advisories (ci(security): the npm audit gate fails an audit-endpoint error exactly like a high advisory — retry, and report 'could not check' distinctly #16337)
• fix(docker): retry apt fetches and fail apt-get update on a lost index, set once for every image (#16291) #16356 fix(docker): retry apt fetches and fail apt-get update on a lost index, set once for every image (ci(docker): an unreachable Ubuntu mirror surfaces as a misleading 'held broken packages' error; apt steps neither retry nor fail on index errors #16291)
• feat(slm): Redis control and detailed health live in the SLM only; remove the unreachable user-GUI copies (#16245, #16252) #16352 feat(slm): Redis control and detailed health live in the SLM only; remove the unreachable user-GUI copies (tech-debt(frontend): move SettingsPanel's detailed system health and Redis service management into the SLM #16245, bug(frontend): RedisServiceControl calls /api/service-monitor/services/* and no mounted router serves those paths #16252)
• fix(code-sync): delete files removed from source through the updater, and check drift across every file (#16310, #16322) #16351 fix(code-sync): delete files removed from source through the updater, and check drift across every file (deploy: the builtin updater never removes files deleted from source, so deleted modules pile up on every host #16310, deploy: ansible synchronize never deletes files removed from source on remote fleet nodes #16322)
• fix(pre-commit): run detect-secrets serially and refresh the baseline's line numbers (#16343) #16349 fix(pre-commit): run detect-secrets serially and refresh the baseline's line numbers (pre-commit: detect-secrets runs in parallel batches that race on .secrets.baseline, so large commits and base merges can't commit #16343)
• fix(kb): make three summarisation paths do what they claim (#14840, #15700, #16110) #16345 fix(kb): make three summarisation paths do what they claim (context-overflow summarization can never succeed — _get_gateway imports a module and a symbol that do not exist #14840, sec(kb): the chat-to-knowledge summariser interpolates whole transcripts unframed, and the result is persisted to the knowledge base #15700, bug(kb): summary quality is scored against text[:200] — a 300-word summary judged on a 200-character window #16110)
• ci(python-suite): queue the shards first in, first out across PRs, with a per-shard job-level queue: max group (#16320) #16328 ci(python-suite): queue the shards first in, first out across PRs, with a per-shard job-level queue: max group (ci: queue python-suite shards FIFO across PRs (queue: max), so one PR's suite finishes before the next starts #16320)
• docs(env): move the generated env-var table into ENV_VARS.md and drop its count line, so env-var PRs stop conflicting (#16317) #16318 docs(env): move the generated env-var table into ENV_VARS.md and drop its count line, so env-var PRs stop conflicting (docs: the generated env-var table and its count line live in CLAUDE_RULES.md, so every env-var PR conflicts with every other #16317)

If the queue is draining normally, ignore this. Otherwise:

1. Check whether CI is dispatching at all — see the ci-dispatch-watchdog status on these PRs
2. Merge the ones whose CI has finished and review has passed: gh pr merge <number> --squash --delete-branch
3. Look for a loop opening near-identical PRs

Warn-only runaway detector — .github/workflows/pr-queue-gate.yml. It never blocks a merge.

[github-actions]

✅ SSOT Configuration Compliance: Passing

🎉 No new hardcoded values of either class — ssot and other both block.

Known backlog in pipeline-scripts/hardcoded_values_baseline.txt is suppressed and tracked in #14371.

[mrveiss]

Code review of b878e8c2b: APPROVE, no defects. It was traced against the code, not just the diff.

• released == 1 is guaranteed, not incidental.
  • StreamingResponse.stream_response sends straight from async for chunk in body_iterator and never reads ahead.
  • The chain _framed_audio_chunks → stream_or_synthesize → synthesize_stream is sequential async generators, with no create_task or start_soon anywhere.
  • readexactly(4) releases frame 0. The next readexactly(20) is served from the buffer.
  • _SynthesisThroughput.start()/observe() are synchronous, so there is no extra suspension before the first send.
  • So the count can't be 0 or 2.
• The contrast test exercises the patch. _synthesized_audio_response looks up _framed_audio_chunks as a module global each time it's called. _buffered calls the test module's own import-time binding, which is the original, so it can't recurse.
• The main test catches perf(voice): 4.7-10.1s to first audio — the voice path uses blocking /tts/synthesize while the streaming route sits unused #13215. A buffering route releases all 5 frames before the first send, so released == 1 fails.
• The remaining wall-clock check can't flake. blocking_ttfb >= 0.8 × SYNTH_SECONDS is a lower bound on asyncio.sleep, which never returns early.
• Every caller of _time_to_first_byte was updated for the 3-tuple return; there are no callers outside this file.

The nits (the bare list annotation, two pre-existing truthiness asserts) need no action.