WatcherX is a serverless video ingestion and processing platform built on Google Cloud Platform (GCP). It enables authenticated users to upload videos, which are automatically transcoded to 360p resolution, stored, and tracked for efficient streaming and management. The system is designed for scalability, security, and automatic processing using a modern serverless architecture with minimal operational overhead.
- Secure Uploads: Only authenticated users can request signed URLs for uploading raw videos via Firebase Authentication
- Serverless Architecture: Utilizes Cloud Functions, Pub/Sub, Cloud Run, Cloud Storage, and Firestore—no VM management required
- Automated Transcoding: Videos are transcoded to 360p using FFmpeg in Cloud Run workers
- Auto-Scaling: Cloud Run workers scale automatically based on message queue demand
- Public Access: Processed videos are publicly accessible from Cloud Storage for streaming
- Metadata Tracking: Processing status, timestamps, and video information stored in Firestore
- CORS-Enabled: Cloud Storage configured for cross-origin requests
Note
The web-client is currently implemented primarily for authentication purposes. Full UI/UX features are planned for future development.
The platform consists of three main layers:
- API Service - Cloud Functions for authentication & signed URL generation
- Processing Workers - Cloud Run service for video transcoding with FFmpeg
- Web Client - Next.js application for user interface
- Authentication: User logs in via web-client using Firebase Authentication
- Request Signed URL: Web-client makes HTTPS request to Cloud Functions API endpoint
- Generate Signed URL: Cloud Functions generates a temporary signed URL for secure upload
- Upload Video: User uploads raw video directly to Cloud Storage using the signed URL
- Event Trigger: Video upload to Cloud Storage triggers a Cloud Pub/Sub message
- Worker Activation: Cloud Run worker (
vidProcessService) receives and consumes the message - Video Transcoding: Worker processes video using FFmpeg to transcode to 360p resolution
- Store Processed Video: Transcoded video is saved to
/processed-videosdirectory in Cloud Storage - Update Metadata: Processing status and video metadata are written to Firestore
- Public Delivery: Processed videos become publicly accessible from Cloud Storage
WatcherX/
├── api-service/
│ ├── firebase.json
│ ├── firestore.rules
│ ├── firestore.indexes.json
│ ├── functions/
│ │ └── (Cloud Functions code for signed URL generation)
│ ├── package.json
│ ├── public/
│ └── remoteconfig.template.json
├── vidProcessService/
│ ├── Dockerfile
│ ├── .dockerignore
│ ├── src/
│ │ └── (TypeScript video processing worker code)
│ ├── package.json
│ ├── tsconfig.json
│ └── (Cloud Run containerized service)
├── web-client/
│ ├── app/
│ │ └── (Next.js application pages)
│ ├── public/
│ ├── next.config.mjs
│ ├── package.json
│ ├── tsconfig.json
│ └── .eslintrc.json
├── utils/
│ └── gcs-cors.json
├── archi.png
├── README.md
└── .gitignore
| Component | Technology |
|---|---|
| Cloud Functions | Node.js runtime for generating signed URLs |
| Video Processing | Cloud Run + FFmpeg + TypeScript |
| Containerization | Docker (for Cloud Run deployment) |
| Web Client | Next.js + TypeScript + ESLint |
| Storage | Google Cloud Storage (GCS) |
| Database | Firestore for metadata tracking |
| Messaging | Cloud Pub/Sub |
| Authentication | Firebase Authentication |
| Infrastructure | Google Cloud Platform (GCP) |
- Google Cloud SDK (
gcloudandgsutil) - Node.js (>=14.x)
- FFmpeg installed on system or container
- A GCP project with billing enabled
- Firebase project configuration
- GenerateUploadUrl: Cloud Functions endpoint that generates signed URLs for authenticated uploads
- vidProcessService: Cloud Run worker service that handles video transcoding and metadata updates
Contributions are welcome! Please open an issue or submit a pull request:
- Fork the repo
- Create a feature branch (
git checkout -b feature-name) - Commit your changes (
git commit -m 'Add new feature') - Push to the branch (
git push origin feature-name) - Open a pull request
This project is licensed under the MIT License. See the LICENSE file for details.
Built with ❤️ using Google Cloud Platform