Self-hosted video-to-text and audio-to-text transcription with speaker diarization, timestamps, batch processing, translation, AI summaries, and subtitle exports.
中文文档 · Quick start · Features · FAQ
Video2Text Studio converts public video URLs and local audio/video files into structured, searchable transcripts. Unlike plain speech-to-text scripts, it preserves speakers, segment and word timestamps, confidence data, and source text as a reliable fact layer. Optional OpenAI-compatible models can then create translations, readable transcripts, and summaries without rewriting those facts.
It is designed for podcasts, interviews, lectures, research videos, meeting recordings, multilingual subtitles, media archives, and batch transcription workflows.
Video2Text Studio is an independently modified distribution based on AI-Video-Transcriber, originally created and maintained by GitHub user wendy7756. This repository preserves the upstream Git history and Apache-2.0 license while publishing additional product, workflow, persistence, credential handling, documentation, and branding changes.
Thank you to the original author and contributors for making the source project available. Video2Text Studio is not an official release of the upstream project; issues and changes specific to this distribution should be reported in this repository.
| Capability | What you get |
|---|---|
| Video and audio input | YouTube, Bilibili, TikTok, other yt-dlp URLs, or local media uploads |
| Accurate structure | Speaker diarization, segment timestamps, word timing, confidence, and preserved source text |
| Cloud or local transcription | Speechmatics Batch by default; optional Faster-Whisper for local processing |
| Batch workspace | Persistent SQLite queue, checklist, controlled concurrency, pause/resume, retry, live progress, and ZIP export |
| Translation and summaries | Segment-ID-validated translation plus readable transcripts and summaries through an OpenAI-compatible API |
| Production-ready exports | JSON, Markdown, SRT, and WebVTT in source-language and bilingual variants |
| Privacy-conscious credentials | Request-memory storage by default and optional macOS Keychain support; keys are not written to browser storage or SQLite |
- Category: self-hosted video-to-text, audio-to-text, speech-to-text, and subtitle generator.
- Inputs: public video URLs, audio files, video files, and text files.
- Transcription engines: Speechmatics Batch and optional Faster-Whisper.
- Outputs: structured JSON, readable/bilingual Markdown, SRT, WebVTT, summaries, and batch ZIP archives.
- Best for: podcasts, interviews, courses, lectures, research, media monitoring, accessibility, and multilingual subtitle production.
- License: Apache-2.0.
- Video URLs supported by yt-dlp, plus local audio, video, and text uploads.
- Complete Chinese and English UI for setup, single tasks, batch workflows, progress, results, and runtime feedback, with the preference restored after refresh.
- A common provider interface for Speechmatics Batch and optional Faster-Whisper.
- Speechmatics Mandarin (
cmn), Mandarin/English code-switching (cmn_en), and English (en). - Speaker diarization, word timestamps/confidence, and preserved disfluencies.
- ID-validated, segment-level translation through an OpenAI-compatible API.
- JSON, source/bilingual Markdown, SRT, and WebVTT exports.
- Readable transcript and summary are derived outputs; they never replace the structured fact source.
- Rename
S1/S2for exports without retranscribing or translating. - SSE progress, cancellation, timeouts, retries, and best-effort remote cleanup.
- Persistent batch queue for mixed URL/file imports, review-before-start, pause/resume/cancel, retries, batch SSE, validation, and ZIP export.
Python 3.10+ and FFmpeg (including ffprobe) are required.
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env
python start.pyOpen http://localhost:8000. The default requirements and Docker image do not install Faster-Whisper, CTranslate2, torch, or Whisper models.
docker compose up --build- Sign up or sign in at the Speechmatics Portal.
- Open Settings → API Keys, create a key, and store it securely.
- Expand AI Settings in the web UI, keep “Speechmatics Batch (recommended),” and enter the key.
- Click Test connection. This makes a read-only backend
GET /jobs?limit=1request. It uploads no audio, creates no job, and consumes no audio transcription allowance. - After connection succeeds, you may explicitly open Use short audio test and select media you are authorized to upload (5–60 seconds, up to 10 MB).
On macOS, both the Speechmatics key and the OpenAI-compatible model key can be explicitly saved to Keychain. The page displays only ******; the original value is never returned to the browser.
The short test creates a real Batch job and consumes allowance for the submitted duration. It previews language, speaker count, segment count, timestamps, labels, and transcript only—no translation, summary, or LLM cleanup. The remote job and local test file are removed best-effort after success or failure.
Language choices are Mandarin (cmn, Simplified Chinese output locale), mixed Mandarin/English (cmn_en), and English (en). Automatic language detection is intentionally not shown because this version does not integrate reliable Language ID.
Speaker diarization and disfluency preservation are enabled by default. Speaker sensitivity controls how readily similar voices are kept separate; “prefer current speaker” helps reduce false flips between similar voices.
- Speechmatics with diarization enabled always analyzes audio; ordinary platform captions cannot silently replace it.
- Speechmatics with diarization disabled may use the platform-caption fast path.
- Faster-Whisper may use platform captions first, then fall back to the local model.
The per-request browser key takes precedence over SPEECHMATICS_API_KEY. A formal Speechmatics job uploads normalized audio, polls asynchronously, retrieves JSON-v2, saves the structured result locally, and only then attempts to delete the remote job. Cleanup failure is a redacted warning and never destroys a successful local result. This proactive deletion minimizes retention in addition to Speechmatics' own retention policy, but cannot guarantee deletion during an upstream/network outage.
Provide an OpenAI-compatible key, Base URL, and model to translate and summarize. Translation uses structured segment batches and validates that returned IDs are complete, unique, and known. Failed batches retry and fall back per segment; source text is always preserved and failed translations remain empty with their IDs recorded.
The selected Summary Language is also the translation target. The LLM never generates timestamps or speakers.
Available result downloads:
transcript_*.jsonwith the completeTranscriptResult, including words;- source and bilingual Markdown;
- source and bilingual SRT;
- source and bilingual WebVTT;
- readable transcript Markdown;
- summary Markdown.
Speaker names affect rendered exports only. Raw speaker IDs remain unchanged in JSON.
The web UI has a Batch tasks tab beside the single-task entry. Add one public URL per input, press Enter or Add link for another row, or paste multiple newline-separated URLs to expand them automatically. You can also drop multiple local audio/video/text files. The import is staged first: invalid links, duplicate URLs, duplicate file SHA-256 values, unsafe filenames, and per-file/total-size failures are shown in the checklist. Nothing is submitted to a provider until Start batch is clicked.
The checklist can be titled, reordered, deleted, and given per-item language overrides before starting. Batches default to strict sequential processing (concurrency=1). Speechmatics can use 1–3 controlled workers; Faster-Whisper is always capped at one worker. More concurrency can increase quota consumption, memory use, and rate-limit risk. Pause stops new items while allowing the active item to finish; resume continues queued work; cancel stops queued items and best-effort cancels active remote work. Failed items can be retried individually or with Retry failed. SSE updates are reconciled by a lightweight active-batch poll, and each newly completed item automatically opens as the current result. Use Open on any completed checklist item to reuse the single-task result workspace for transcript/summary/translation viewing, speaker naming, and per-format JSON, Markdown, SRT, or VTT downloads.
Each batch has its own SQLite-backed queue and item state. GET /api/batches/{batch_id}/stream sends one reconnectable SSE stream with a complete batch.snapshot, item/batch updates, terminal batch.completed, and ping. The page can disconnect without stopping the backend scheduler. A successful or partially successful batch can export a ZIP without retranscribing; only successful items are included by default:
batch-name/
batch-manifest.csv
batch-summary.json
001-safe-title/
transcript.json
transcript-source.md
transcript-bilingual.md
transcript-source.srt
transcript-bilingual.srt
transcript-source.vtt
transcript-bilingual.vtt
transcript-readable.md
summary.md
batch-manifest.csv includes position, title, redacted source, input_type, status, provider, language, audio_duration, speaker_count, segment_count, warnings, error, and output_folder. The ZIP never contains input media, API keys, sensitive query parameters, or raw upstream provider responses. ZIP creation is streamed to a batch-owned directory and atomically replaced; export can be retried independently.
Batch API endpoints are POST /api/batches, POST /api/batches/{id}/files, PATCH /api/batches/{id}, PATCH/DELETE /api/batches/{id}/items/{item_id}, POST /start|pause|resume|cancel|retry-failed, per-item retry|cancel, GET /api/batches, GET /api/batches/{id}, GET /items, /stream, /prepare-export, /export-status, /download, and per-item downloads. The existing /api/process-video, /api/process-upload, single-task SSE and downloads remain compatible.
Batch settings are allow-listed snapshots and never contain credentials. Request keys remain in JavaScript memory and the backend's in-memory CredentialVault, unless the user explicitly saves either provider key to macOS Keychain. Keys are not stored in localStorage, SQLite, SSE, logs, URLs, DOM data attributes, or ZIPs. The browser persists only non-sensitive workspace state (provider, language, model choice, mode, and current task/batch ID), so refresh can reopen the same progress view. A batch with a sensitive URL query parameter (for example token, signature, authorization, api_key, or secret) shows only a stripped public URL and requires the original URL again after a restart. A normal public URL is persisted without query or fragment.
Batch state lives in temp/state.sqlite3 (WAL mode); files live only under temp/batches/{batch_id}/inputs, working, items, and exports. The default limits are 50 items, 2 GB total uploads, 24-hour input retention, and 168-hour output/ZIP retention. A safe cleanup pass runs at service startup for terminal batches and validates that every target is directly inside the batch-owned directory; finished batches can also be deleted explicitly through the UI/API. The scheduler is an in-process single-application scheduler. Run one Uvicorn worker; multiple workers need an external queue/lease deployment plan even though item leases are stored for defensive claiming.
After a service restart, unfinished batches become interrupted; items that had not been submitted become waiting_for_credentials and no new paid job is submitted automatically. Click Resume and provide the credential again. A saved Speechmatics remote job ID may be retained for diagnosis, but this version does not silently resume polling without explicit user confirmation and a credential.
The structured JSON is saved before remote deletion. If a later local export fails, call POST /api/tasks/{task_id}/retry-export to regenerate files from that fact source. Single-task results remain under temp/ and should be cleaned according to the deployment's retention policy; batch inputs, outputs, and ZIPs follow the batch retention windows described above.
pip install -r requirements-local.txtSet WHISPER_MODEL_SIZE=base (or another supported model size) if needed. The import and model load are lazy: the application and Speechmatics provider start normally when Faster-Whisper is absent. Selecting the missing local provider returns an installation hint. This backend does not diarize speakers and uses UNKNOWN labels.
SPEECHMATICS_API_KEY=
SPEECHMATICS_BASE_URL=https://eu1.asr.api.speechmatics.com/v2
SPEECHMATICS_OPERATING_POINT=standard
SPEECHMATICS_MAX_WAIT_SECONDS=7200
SPEECHMATICS_TEST_MAX_WAIT_SECONDS=300
SPEECHMATICS_TEST_MAX_MB=10
SPEECHMATICS_TRUST_ENV=false
OPENAI_API_KEY=
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_TRANSLATION_MODEL=gpt-4o
UPLOAD_MAX_MB=200
BATCH_MAX_ITEMS=50
BATCH_DEFAULT_CONCURRENCY=1
BATCH_MAX_CONCURRENCY=3
BATCH_MAX_TOTAL_UPLOAD_MB=2048
BATCH_INPUT_RETENTION_HOURS=24
BATCH_OUTPUT_RETENTION_HOURS=168
CORS_ALLOWED_ORIGINS=http://localhost:8000,http://127.0.0.1:8000Current Speechmatics documentation uses regional SaaS hosts. Existing deployments can override the base URL with another compatible Jobs API endpoint, including the legacy https://asr.api.speechmatics.com/v2. Every request for a job must remain in the region where it was created.
Process-level HTTP proxy variables are ignored by default, preventing malformed NO_PROXY/IPv6 entries from breaking client initialization. Set SPEECHMATICS_TRUST_ENV=true only when Speechmatics must use a managed proxy and its variables are valid for httpx.
- Speechmatics and OpenAI-compatible keys use password inputs and are never stored in localStorage, sessionStorage, cookies, URLs, DOM
data-*,tasks.json, SQLite, SSE, API responses, transcript metadata, logs, or downloads. - On macOS, either key can be explicitly saved as a separate Keychain item. The browser receives only
saved: true/falseand displays******, never the stored value. - Request-scoped Speechmatics and OpenAI keys live in an independent in-memory
TaskContext; references are cleared after completion, failure, or cancellation. - Browser calls go only to this backend. Sensitive responses use
Cache-Control: no-store, and upstream errors are mapped to redacted messages. - Remote non-HTTPS pages display a warning; localhost HTTP remains usable for development.
- CORS defaults to explicit localhost origins, with no wildcard/credential combination.
Refreshing the page restores the non-sensitive setup and reconnects to the current single task or batch. Unsaved keys are cleared; Keychain-backed keys remain usable. After a service restart, an in-flight remote task still requires explicit Resume before any new paid work is submitted.
backend/
api/ # batch HTTP routes and request models
transcribers/ # base contract, registry, providers
transcript_models.py # TranscriptResult/Segment/Word fact source
transcript_pipeline.py # subtitle/text normalization and policy
translation_pipeline.py # ID-preserving segment translation
exporters.py # JSON/Markdown/SRT/VTT
task_context.py # non-persistent task secrets/cancellation
batch_models.py # batch/item states and settings snapshot
batch_repository.py # SQLite WAL repository and event cursor
batch_scheduler.py # bounded in-process workers and leases
batch_exporter.py # streaming safe ZIP export
credential_vault.py # process-memory batch credentials
result_validator.py # transcript/export quality checks
item_runner.py # shared single-item provider pipeline
main.py # API and orchestration
A future provider implements BaseTranscriber and registers a factory; upload, translation, and export paths remain provider-neutral.
pip install -r requirements-dev.txt
pytest -q
python -m compileall -q backendAll Speechmatics and OpenAI-compatible calls are mocked. The automated suite does not consume real API allowance.
Video2Text Studio is a self-hosted web application for converting video and audio into structured text. It adds speaker labels, timestamps, batch processing, translation, summaries, and subtitle exports around cloud or local transcription engines.
Yes. It accepts public URLs supported by yt-dlp, including YouTube, Bilibili, TikTok, and many other platforms. Only process media you are authorized to access and transcribe.
Yes. Faster-Whisper is available as an optional local backend. Speechmatics Batch remains the default when speaker diarization and richer structured speech data are required.
No. Translation, readable transcripts, and summaries are derived outputs. The LLM is not allowed to generate or rewrite timestamps, speaker IDs, word confidence, or source transcript facts.
The application is open source under Apache-2.0. External services such as Speechmatics or an OpenAI-compatible model provider may charge for their usage. Faster-Whisper can run locally on your own hardware.
- Manual end-to-end validation requires your own authorized media and API key; no third-party sample is bundled.
- There is no automatic language option without reliable Language ID.
- Faster-Whisper has no diarization.
- Text uploads have no real audio timeline, so deterministic synthetic segment timing is used.
- In-memory active-task coordination targets a single application process. Multi-worker/host deployment needs an external queue and shared state.
- A disconnected browser does not normally cancel a formal backend task; reconnect and query or cancel the task explicitly.
- Batch scheduling is deliberately single-process. Run one Uvicorn worker; SQLite is the durable queue, but multi-worker scheduling still needs an operational deployment lock or external queue.
- Batch input/output retention cleanup runs on startup; deployments needing continuous cleanup should restart or add a separate scheduler with the same path checks.