Skip to content

[feat] : Implement complete FFmpeg worker and HLS video processing pipeline#11

Merged
maulik-mk merged 2 commits intodevfrom
feat/hls-encoding-pipeline
Mar 21, 2026
Merged

[feat] : Implement complete FFmpeg worker and HLS video processing pipeline#11
maulik-mk merged 2 commits intodevfrom
feat/hls-encoding-pipeline

Conversation

@maulik-mk
Copy link
Copy Markdown
Owner

This pull request introduces a comprehensive modernization and expansion of the video processing worker project. The changes include a major overhaul of the Docker build pipeline to use Ubuntu and add advanced codec support, the introduction of production-ready configuration and environment files, new database schema for video processing, and improvements to package management and code formatting. These updates collectively enhance the build reproducibility, security, maintainability, and operational capabilities of the service.

Docker Build & Runtime Modernization:

  • Replaces the Alpine-based Docker build with a multi-stage Ubuntu 24.04 pipeline, adding robust support for advanced codecs (x264, x265, fdk-aac, VMAF, Opus, SoXR, zimg) and VMAF quality metrics, and creates a secure, production-ready runtime with Node.js and a non-root user. The Dockerfile now includes detailed documentation, OCI labels, and a healthcheck. [1] [2]

Configuration & Environment Management:

  • Adds a detailed .env.example file outlining all required environment variables for both development and production, including Azure, Redis, Postgres, and pipeline settings.
  • Introduces a .dockerignore file to optimize Docker build context and exclude unnecessary files.

Database Schema for Video Processing:

  • Adds a new SQL schema (sql/main.sql) defining tables for videos, video metadata, and video renditions, complete with indexes, constraints, and triggers for automatic timestamp updates.

Package Management & Formatting:

  • Updates package.json to reflect the new repository, expands dependencies/devDependencies for Fastify, BullMQ, Azure SDK, and more, and adds scripts for building, formatting, and end-to-end testing.
  • Adds a Prettier configuration file for consistent code formatting.
  • Updates pnpm-workspace.yaml to ignore certain built dependencies for workspace management.

Most Important Changes:

1. Build & Runtime Modernization

  • Major Dockerfile overhaul: switches to Ubuntu 24.04, adds multi-stage builds for FFmpeg with advanced codecs (x264, x265, VMAF, etc.), builds VMAF from source, and sets up a secure, production-ready Node.js runtime with health checks and non-root execution. [1] [2]
  • Adds .dockerignore to reduce build context size and improve build efficiency.

2. Configuration & Environment

  • Introduces a comprehensive .env.example for both production and development, documenting all required environment variables for system services, Azure, and testing.

3. Database Schema

  • Adds sql/main.sql defining normalized tables for videos, metadata, and renditions, including constraints, indexes, and triggers for automatic timestamp updates.

4. Package & Dependency Management

  • Updates package.json with new dependencies (Fastify, BullMQ, Azure SDK, etc.), scripts for build/test/dev, and repository metadata; introduces Prettier config for formatting. [1] [2]

5. Workspace Configuration

  • Updates pnpm-workspace.yaml to ignore specific built dependencies, improving workspace management.

…ture

- Set up DDD structure (application, domain, infrastructure)

- Implement FFmpeg core adapters and HLS playlist generation

- Add BullMQ worker and Azure Blob Storage integration
@maulik-mk
Copy link
Copy Markdown
Owner Author

CI build failed

Screenshot

@maulik-mk
Copy link
Copy Markdown
Owner Author

Fix in commits

CI build failed

Screenshot

@maulik-mk maulik-mk merged commit aa1f2d8 into dev Mar 21, 2026
1 check passed
@maulik-mk maulik-mk deleted the feat/hls-encoding-pipeline branch April 5, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant