Index and display developer-focused YouTube channels, playlists, and videos #120

What would you like to do?

Allow developer-focused YouTube channels, playlists, and individual videos to be submitted, indexed, and displayed in DevFeed. Readers should discover relevant videos alongside articles and open them on YouTube, while keeping existing relevance and publication checks.

Current behavior

Verified on master at b302fc8849b2ccdc5b29efe9bb82718bb28e2380:

• Source ingestion is built around RSS/Atom and publisher/aggregator semantics, with no dedicated YouTube collection adapter.
• ContentFormat already includes video, but publication_blockers requires page_kind=article. The analysis and publication pipeline needs an explicit contract for substantive video content.
• ArticleCard currently displays Read and article-specific action labels.
• Existing source relevance assessment uses bounded title/summary evidence and reviews entries individually, including mixed-topic sources.

Required behavior and implementation

Submission and stable identity

• Accept channel URLs/handles, playlist URLs, and individual video URLs through source/content submission, admin import, and CLI flows as applicable.
• Resolve channel handles/URLs to stable channel IDs; persist playlist and video IDs. Normalize supported YouTube hosts and URL variants, including youtu.be, mobile URLs, and tracking/timestamp parameters.
• A watch URL containing both a video ID and playlist ID should preview the detected resources and make the submission target explicit; it must not silently import an entire playlist when the user submits one video.
• Model channels and playlists as recurring collections, and individual videos as single content submissions with provenance through their channel. Submitting one video must not subscribe to all channel uploads or implicitly approve its channel.
• Keep provider/resource identity separate from existing publisher/aggregator semantics. Define the additive schema/API changes needed rather than forcing YouTube resources to have an arbitrary RSS URL.

YouTube indexing adapter

• Implement a bounded provider adapter using documented public YouTube metadata APIs: resolve channels and their uploads collection, enumerate playlist items, and retrieve video metadata.
• Use stable IDs, batched metadata reads, resumable pagination/checkpoints, and configurable initial-import limits. Channel/playlist refreshes must discover new items without repeatedly importing the entire history.
• Persist canonical video identity, title, description, channel attribution, original publication time, thumbnail, duration when available, language evidence, collection memberships, and retrieval/availability state.
• Store video publication time separately from playlist insertion time and DevFeed discovery time. Reordering a playlist must not make an old video appear newly published.
• Deduplicate globally by video ID across direct submissions, channels, playlists, and URL variants, while preserving all collection origins. Repeated refreshes and retries must be idempotent.
• Removing a video from one playlist removes that membership; it must not withdraw a still-public video available through another approved origin.
• Integrate recurring collection refreshes with existing scheduler/job ownership, source approval, retry/backoff, and active-job coalescing. Coordinate with adaptive scheduling in Add gated adaptive source sync intervals based on publication velocity #116 without making this feature depend on it.
• Handle unavailable/private/deleted videos, missing metadata, partial pages, provider outages, and quota/rate-limit errors without failing unrelated source ingestion. Revalidate availability and remove unavailable items from public results.
• Configure any required provider credentials server-side, redact them from logs, and define clear missing-configuration/quota-exhausted states. Bound API use and avoid repeated search calls when collection IDs are known.
• Preserve existing URL validation, guarded network access, timeout/response limits, and redirect protections.

Developer relevance and publication

• Assess channel/playlist relevance from a bounded sample, then assess individual videos using their own evidence. A relevant channel does not automatically make every upload relevant.
• Apply the existing developer/software audience policy to tutorials, technical talks, engineering discussions, and other substantive developer content. Unrelated entertainment, generic promotion, and insufficient evidence must remain distinguishable.
• Adapt enrichment, analysis, and publication checks to accept evidenced substantive videos with content_format=video; do not relabel every YouTube page as an article or bypass relevance, source approval, summary, language, and topic requirements.
• Keep summaries grounded in retrieved metadata/evidence. Do not invent claims about a video's contents from its title. Sparse descriptions should remain pending for additional evidence or explicit review.
• Indexing must work without requiring transcripts; use additional evidence only when available through a supported access path. Handle unknown language explicitly.

Reader and administration UX

• Display published videos in feeds, search, topic/source pages, bookmarks, and detail views through shared reader components.
• Show a Video label, thumbnail, channel attribution, and duration when known. Use Watch on YouTube and appropriate accessible action labels rather than Read original article.
• Open the canonical YouTube video URL from the watch action. Likes, bookmarks, sharing, and recommendations should use the same stable content identity.
• Preserve channel/playlist provenance and provide collection pages with indexed videos and follow controls. Make collection kind and indexing/review/error state visible in source/admin workflows.
• Define whether watch-link opens participate in the existing activity metric; keep copy explicit that an open does not prove playback or completion.
• Update public/user/admin API contracts and regenerate admin clients where changed.
• Verify the web reader and both built Chrome/Microsoft Edge extensions, including routes, provider links, images, and engagement actions.

Acceptance criteria

• A developer-focused channel, playlist, and individual video can each be submitted, reviewed, indexed, and displayed through the intended workflows.
• Channel/playlist refreshes discover new videos with bounded API usage, persisted checkpoints, and safe retries.
• Repeated imports and URL variants produce one content item per video, retaining multiple origins and accurate timestamps.
• Mixed-topic collections do not publish unrelated videos; sparse evidence does not produce unsupported summaries or automatic publication.
• Approved, substantive videos pass the video-aware analysis/publication contract; existing article policy behavior remains covered.
• Deleted/private videos are withheld, and removing one playlist membership does not remove an otherwise valid video.
• Backend tests cover URL/resource detection, mocked provider pagination/metadata, deduplication, checkpoints, quotas, unavailable content, relevance, and publication decisions.
• Disposable-database integration tests verify migrations, origins, repeated imports, scheduler/job ownership, and public filtering.
• Browser tests verify video display and Watch on YouTube, collection navigation, like/bookmark/share behavior, and responsive/accessibility states on web, Chrome, and Edge.

Provider implementation references

• Channel lookup supports stable IDs and handles, and channel content details expose uploads information: YouTube channels.list.
• Enumerate collection entries through YouTube playlistItems.list; fetch video resources through YouTube videos.list.
• Implement bounded requests against the documented YouTube API quota model.