YouTube Live Chat Reader

A high-performance system for efficiently reading YouTube Live Chat messages.

🚀 Features

• Real-time Processing: Reading YouTube Live Chat messages with the most efficient method, streamList
• Scalable & High Available: Horizontally scalable reading workers; if one fails, another one takes over immediately
• Observable: Comprehensive metrics, tracing, and logging
• Kubernetes-Native: Designed to run in containerized environments

🏗️ Architecture

The system consists of two main components:

1. Finder Service

• Periodically checks configured YouTube channels for upcoming live streams
• Emits events when new live streams are discovered
• Runs as a Kubernetes CronJob for scheduled execution

2. Reader Service

A multi-binary service with the following components:

Consumer

• Listens for newly discovered live stream events
• Creates the initial live stream progress state
• Runs as a Kubernetes Deployment

Worker

• Reads and stores live chat messages using YouTube’s streamList
• Distributed locking with Redis to ensure exactly-once live stream processing
• Runs as a Kubernetes Deployment

🚀 Quick Start

Prerequisites

• Kubernetes cluster (e.g., Docker Desktop)
• Helm 3+
• kubectl

Installation

1. Clone the repository:

   git clone https://github.com/natsoman/youtube-chat-reader.git
   cd youtube-chat-reader

2. Deploy to your Kubernetes cluster:

   # Navigate to the deployment directory and follow instructions
   cd deploy/local/k8s/

⚙️ Configuration

Configuration is managed through Kubernetes resources:

Common Configuration

• ConfigMap
• Secrets

Component-Specific Configuration

Finder

• CronJob
• Secrets

Reader Worker

• Deployment
• Secrets

Reader Consumer

• Deployment

Coverage