Embeddish

Embeddish is a Phoenix web application that allows you to download videos (Originally intended for Instagram Reels dishes, hence the name), analyze their content using multimodal AI (Google Gemini), and perform semantic searches across your video library.

It leverages the power of Google Gemini for transcribing and embedding video content, and pgvector for high-performance vector similarity search within PostgreSQL.

Features

• Video Downloading: Automatically downloads videos from supported platforms (powered by yt-dlp).
• AI-Powered Analysis:
  • Uses Google Gemini to generate accurate transcriptions of video content.
  • Creates vector embeddings for semantic understanding.
• Semantic Search: Search through your video library using natural language queries to find relevant content, even if the exact keywords aren't present.
• Background Processing: Robust handling of long-running tasks (downloads, AI processing) using Oban.

Tech Stack

• Framework: Phoenix Framework (Elixir)
• Database: PostgreSQL with pgvector extension
• AI Model: Google Gemini 1.5 Flash (via Req)
• Background Jobs: Oban
• Styling: Tailwind CSS
• Media Handling: yt-dlp

Prerequisites

Before you begin, ensure you have the following installed:

1. Elixir: v1.18 or later.
2. PostgreSQL: With pgvector extension installed.
   • Tip: If using Docker, you can use the pgvector/pgvector:pg16 image.
3. yt-dlp: Command-line tool for downloading videos.
   • macOS: brew install yt-dlp
   • Linux/Windows: Follow official installation instructions.
4. Google Gemini API Key: Get one from Google AI Studio.

Setup

1. Clone the repository:

   git clone https://github.com/your-username/embeddish.git
   cd embeddish

2. Install dependencies:

   mix deps.get

Now you can visit localhost:4000 from your browser.

Usage

1. Add a Video: On the home page, enter a valid video URL (e.g., YouTube) and click "Process".
2. Wait for Processing: The system will download the video, upload it to Gemini for analysis, and store the results. You can monitor the progress via the UI or logs.
3. Search: Use the search bar to find videos based on their content. For example, "Show me a recipe for pasta" or "Explain how neural networks work".