feat(asr): add ElevenLabs Scribe v2 provider#807
Conversation
PR Reviewer Guide 🔍(Review updated until commit 76603c5)Here are some key observations to aid the review process:
|
appergb
left a comment
There was a problem hiding this comment.
Verdict: REQUEST_CHANGES
我独立复核了 issue #795 对应的完整 20 文件 functional diff。当前有 1 个 P2 阻塞项:
[P2] QA 浮窗中的当前听写在 ElevenLabs 转写阶段取消,会被展示为“识别失败”
位置:openless-all/app/src-tauri/src/coordinator/qa_session.rs:288-300,以及同文件 138-145。
wait_for_overlay_dictation_cancel 返回后,ElevenLabs 分支先调用 asr.cancel(),随后返回 Err("dictation cancelled")。上层 take_current_dictation_transcript_for_qa 对所有 Err 都调用 finish_qa_with_error("识别失败: ...") 并直接返回,因此 148-156 行原本用于静默处理取消的 cancelled 检查永远不会执行。结果是请求虽然停止了,但用户主动取消会得到错误胶囊,违背 #795 的取消行为要求。
请把“用户取消”与“转写失败”分开表达,例如使用结构化结果;或者在上层 Err 分支先检查 cancelled / session_id 是否已变化,恢复状态并返回 Ok(None),不要调用 finish_qa_with_error。请补一个回归测试:在 QA 浮窗的 ElevenLabs 当前听写转写阶段触发取消,断言最终为 idle 且不发 Error/识别失败事件。
非阻塞:
[P3] 新增的 persistence/credentials.rs:958 set_for_asr_provider 签名不符合当前 rustfmt 输出。仓库 beta 本身已有大量格式债务,但 cargo fmt --check 对 PR 新增这一处也给出 diff,建议随修复一起格式化该函数。
其余独立核验:
- ElevenLabs 官方 API 文档确认 POST /v1/speech-to-text、xi-api-key、multipart file/model_id、scribe_v2 及 timestamps_granularity=none;本实现匹配。
- 检查了 HTTPS/loopback 限制、DNS 固定、禁止重定向、动态超时、1 MiB 流式响应上限、错误体脱敏与 API key 不落日志。
- 检查了 provider-scoped 凭据读写和 active provider 路由。CredentialsVault::get 会按 root.active.asr 取当前 provider,而切换时先更新 active;未复现机器人评论所述的凭据错配。
- 检查了普通听写、QA、重转写和资源取消路由,以及 5 个 locale 和设置页变更;未见大范围 UI 重构。
- CI run 29311411804 在当前 head 3f87cfd 上全绿:macOS 625/625、Linux 611/611、Windows Rust-only 123/123,Android cargo check 通过;12 个 ElevenLabs 单测在 macOS/Linux 通过。
- git diff --check 通过,差异中未发现硬编码密钥。cargo fmt --check 整体仍失败,主要是 beta 既有格式债务,另含上面的新增 P3。
当前 gh 账号 appergb 与 PR 作者相同,GitHub 不允许正式 APPROVE/REQUEST_CHANGES,因此此处按要求提交 COMMENTED review;审阅结论仍是 REQUEST_CHANGES,修复 P2 后再批准。
|
快修p2 |
3f87cfd to
76603c5
Compare
|
Persistent review updated to latest commit 76603c5 |
appergb
left a comment
There was a problem hiding this comment.
Verdict: APPROVE
复审 head 76603c5 后,没有发现新的 P1/P2/P3 阻塞项。此前 review #4691507297 的两个问题均已解决:
- P2 取消错误路径已修复:transcribe_overlay_dictation_asr 现在通过 OverlayDictationTranscribeOutcome::Cancelled 显式区分用户取消和转写失败;take_current_dictation_transcript_for_qa 对取消路径恢复会话并调用 finish_qa_idle_silently,返回 Ok(None),不再进入 finish_qa_with_error。
- 新增 overlay_elevenlabs_cancel_finishes_idle_without_error_capsule 回归测试,实际启动 ElevenLabs 请求后触发 cancel_session,并断言主会话与 QA 都为 Idle、QA 取消标志复位、最后胶囊状态为 Idle。旧实现会停在 Error,因此该测试能捕获原回归。
- P3 rustfmt 已修复:set_for_asr_provider 现在与 rustfmt 输出一致。cargo fmt --check 仍会报告 qa_session.rs:60 和 credentials.rs 的既有 beta 格式债务,但不再报告此前新增函数。
- ElevenLabs 响应测试 helper 改写保持生产边界:read_response_limited 对每个 reqwest bytes_stream chunk 都调用生产函数 append_response_chunk;该函数以累计长度限制 MAX_RESPONSE_BYTES。测试验证恰好 1 MiB 可接受,随后再加 1 byte 会拒绝,等价覆盖生产累计边界,不是测试专用旁路。
验证结果:
- 对比旧 head 3f87cfd 与新 head,实际修复仅涉及 qa_session.rs、persistence/credentials.rs 和 elevenlabs.rs;完整 PR 仍为 20 文件。
- git diff --check origin/beta...HEAD 通过,独立 worktree 无修改。
- 当前 PR mergeable。分支比 beta 少安全依赖提交 66d0847,但 GitHub CI 使用的 merge ref b307c4e 的两个父提交正是 beta 66d0847 与 head 76603c5,因此已验证实际合并结果。
- CI run 29314019430 全绿:Android cargo check、Linux、macOS、Windows。Linux 612/612、macOS 626/626、Windows Rust-only 123/123;两个重点测试 response_limit_accepts_one_megabyte_and_rejects_the_next_byte 与 overlay_elevenlabs_cancel_finishes_idle_without_error_capsule 在 Linux/macOS 均通过。
- PR-Agent 检查也通过。
当前 gh 账号 appergb 与 PR 作者相同,GitHub 不允许正式 APPROVE 自己的 PR,所以按要求提交 COMMENTED review;逻辑结论为 APPROVE,代码层面可合并,仍需其他账号完成仓库要求的正式批准。
User description
Summary
Replaces #797 with a clean branch based on the current
betatip.Fixes #795.
Adds ElevenLabs Scribe v2 as a batch ASR provider and wires it through dictation, QA, retranscription, cancellation, provider validation, provider-scoped credentials, settings, and localized provider labels.
Review finding fixed
ElevenLabs defaults
timestamps_granularitytoword, which adds a potentially largewordsarray even though OpenLess only consumes the top-level transcript text. The client now explicitly sendstimestamps_granularity=none, keeping long-recording responses text-only and within the existing defensive 1 MiB response cap. A boundary test proves that exactly 1 MiB is accepted and the next byte is rejected.Official API reference: https://elevenlabs.io/docs/api-reference/speech-to-text/convert
Security and compatibility
xi-api-keyor audio.scribe_v2model.Verification
timestamps_granularitywas absent.cargo test --locked --manifest-path src-tauri/Cargo.toml asr::elevenlabs::tests— 12 passed, 0 failed.cargo test --locked --manifest-path src-tauri/Cargo.toml— 625 passed, 0 failed.npm ci --no-audit --no-fund— completed.npm run build— completed; only the pre-existing Android IPC manual-chunk circular warning remains.npm audit --json— 0 vulnerabilities.git diff --check origin/beta...HEAD— clean.qwen-asrsubmodule initialized atb00b789b17051aea61e9717458171100662318a4.GitNexus classifies the complete provider integration as critical blast radius because it touches 34 execution flows across dictation, QA, cancellation, and provider dispatch. The focused reviewer-finding fix itself is low risk (one file, eight symbols, zero mapped flows). This PR requires independent agent review before merge.
PR Type
Enhancement, Tests, Documentation
Description
Add ElevenLabs Scribe batch ASR client
Wire through dictation, QA, retranscribe, and cancel
Extract SSRF validation to shared module
Add per-provider credential isolation and frontend preset
Update i18n for 5 languages and privacy descriptions
Diagram Walkthrough
flowchart LR AudioBuffer["Audio Buffer (16kHz PCM)"] --> EncodeWav["Encode WAV"] EncodeWav --> MultipartForm["Multipart POST /v1/speech-to-text"] MultipartForm --> SecCheck["SSRF Check & Redirect Block"] SecCheck --> ElevenLabsAPI["ElevenLabs API"] ElevenLabsAPI --> ParseResponse["Parse JSON Response"] ParseResponse --> Transcript["Transcript Text"] Transcript --> App["Application"]File Walkthrough
15 files
New ElevenLabs Scribe ASR clientExport ElevenLabs moduleAdd per-provider credential read/writeAdd ElevenLabs provider validationIntegrate ElevenLabs ASR into coordinatorAdd ElevenLabs provider detectionHandle ElevenLabs dictation sessionsHandle ElevenLabs QA transcription with cancelAdd ElevenLabs cancel supportExtract SSRF validation moduleRegister endpoint_security moduleAdd per-provider credential methodsUpdate credential IPC for provider scopeAdd ElevenLabs preset and per-provider fieldsAdd ElevenLabs ASR preset definition5 files
Add English i18n for ElevenLabsAdd Japanese i18n for ElevenLabsAdd Korean i18n for ElevenLabsAdd Simplified Chinese i18n for ElevenLabsAdd Traditional Chinese i18n for ElevenLabs