Skip to content

feat(stt): transcribe with omp's own speech-to-text models, including a ChatGPT/Codex subscription - #249

Open
andrebrait wants to merge 2 commits into
kahme247:mainfrom
andrebrait:feat/codex-stt
Open

andrebrait wants to merge 2 commits into
kahme247:mainfrom
andrebrait:feat/codex-stt

Conversation

@andrebrait

Copy link
Copy Markdown
Collaborator

Problem

Voice input needed an OpenAI-compatible STT endpoint (OMP_WEB_STT_ENDPOINT): an API key or a separately hosted STT server. Users with a ChatGPT/Codex subscription already logged in to omp could not dictate with it.

Changes

  • Transcribe with an omp model. Leave OMP_WEB_STT_ENDPOINT unset and set OMP_WEB_STT_MODEL to an omp speech-to-text model, such as openai-codex/codex-transcribe (ChatGPT subscription, added in can1357/oh-my-pi#15391). An endpoint, when set, still wins, so existing setups are unchanged.
  • omp keeps the credentials. Jobs go through omp auth-gateway stdio (lib/omp/auth-gateway.ts): one shared child, started on first use and stopped after 5 minutes idle, sending base64 JSON requests. omp-web never reads omp's credential store. A crashed child rejects its in-flight jobs and the next job respawns it; a child that never reports ready is killed after 60 seconds.
  • Size cap. The gateway limits a JSON body to 25 MB and base64 grows audio by 4/3, so this path rejects recordings over 18 MB with a clear error instead of an opaque 413.
  • The 501 message names both variables. README (en/ja/zh-CN), CHANGELOG and AGENTS.md describe the setting.

Verification

  • npm test: 1356 pass, 0 fail. lib/stt-omp-gateway.test.mjs runs real jobs against a stub omp that speaks the stdio protocol: a transcript, an upstream error, a crash (error carries omp's stderr, next job respawns the gateway), and the format following the browser's file name (webm, mp4). The 501 test now clears both variables.
  • tsc --noEmit clean; eslint has no errors.
  • Live, against a real Codex subscription with the omp branch from feat(ai): transcribe speech with a ChatGPT/Codex subscription can1357/oh-my-pi#15391: the job routes with OMP_WEB_STT_MODEL=openai-codex/codex-transcribe transcribed the Whisper JFK clip correctly, 5.9 s cold (gateway start) and 1.1 s warm.
  • Endpoint probes: WebM (Chrome), MP4 (Safari) and Ogg/Opus (Firefox) recordings all transcribe.

Needs an omp release that includes can1357/oh-my-pi#15391 for the Codex model; other omp STT models (for example openai/whisper-1 with an OpenAI key in omp) also need its auth-gateway stdio selector fix.

…stdio

Without OMP_WEB_STT_ENDPOINT, OMP_WEB_STT_MODEL names an omp speech-to-text
model (e.g. openai-codex/codex-transcribe on a ChatGPT subscription). Jobs run
through a shared, idle-killed omp auth-gateway stdio child, so omp-web never
touches credentials and needs no API key or separate STT server.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant