Skip to content

[codex] Improve YouTube transcript extraction - #86

Merged
Talljack merged 9 commits into
mainfrom
codex/youtube-transcript-parity
Jul 13, 2026
Merged

Talljack merged 9 commits into
mainfrom
codex/youtube-transcript-parity

Conversation

@Talljack

Copy link
Copy Markdown
Owner

What changed

  • add resilient YouTube caption extraction through mobile player clients, watch-page tracks, timed-text tracks, and the existing package fallback
  • support watch, Shorts, live, embed, and youtu.be URLs with strict host validation
  • normalize JSON3 captions and preserve the existing import API response contract
  • cap direct extraction at six YouTube requests with 300ms, 600ms, and 1200ms backoff for retryable failures
  • stop extraction after aborts and add focused route/parser/fallback/budget tests

Why

The previous route relied only on youtube-transcript, so videos available through YouTube's other caption sources frequently returned no transcript. Unbounded fallback attempts could also amplify failures and trigger YouTube network protection.

Impact

YouTube imports recover more manual and automatic captions while limiting request bursts during rate limits or upstream failures. Existing API consumers keep the same response shape.

Validation

  • pnpm test src/lib/youtube-transcript.test.ts src/app/api/import/youtube/route.test.ts — 22/22 passed
  • pnpm typecheck — passed
  • pnpm lint — passed
  • pnpm build — passed
  • real public-video smoke test recovered 61 English segments / 2089 characters before later network throttling
  • full suite: 720/721 passed; the sole failure is the pre-existing daily-plan.test.ts:283 assertion

@vercel

vercel Bot commented Jul 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
echo-type Ready Ready Preview, Comment Jul 13, 2026 1:45pm

@Talljack
Talljack marked this pull request as ready for review July 13, 2026 14:07
@Talljack
Talljack merged commit 96ae1b4 into main Jul 13, 2026
6 checks passed
@Talljack
Talljack deleted the codex/youtube-transcript-parity branch July 13, 2026 14:08

This branch was successfully deployed

1 active deployment
Preview — 7f64811c Deployed Jul 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant