The AI Video Insights Assistant is a powerful backend service that uses advanced AI orchestration to analyze YouTube video content and answer user-submitted questions in a contextually accurate manner. Built on either LangChain4j or Spring AI for robust orchestration, this app leverages embeddings and similarity search for precise content-based responses, while also including fallback options for comprehensive coverage.
screen-capture.mp4
- Robust Transcript Processing with Error Handling: The assistant retrieves and processes video transcripts, generating meaningful answers based on video content. If the transcript cannot be processed, users receive clear, actionable error messages.
- Embedding and Vector Storage: Video content embeddings are stored in a vector database for efficient retrieval.
- Similarity Search for Relevant Answers: The assistant performs similarity searches on stored embeddings to identify the most relevant answers to user questions. If the relevance of all matching documents falls below a cosine similarity threshold of 0.75, a fallback mechanism activates.
- Dynamic and User-Friendly API: Designed with intuitive API, users can smoothly browse collections, select pieces of interest, and read detailed descriptions. The UI ensures that each piece of information is easy to access and visually engaging.
- Fallback to Tavily API Search: For low-relevance document matches, the assistant queries the Tavily API, using the “Include Answer” option to retrieve high-quality answers along with the top 3 related URLs from Tavily results, ensuring that users receive the most comprehensive response available.
- Advanced Q&A on Video Content: Users can submit any question about a video, and the assistant respond using contextually appropriate information derived from the video transcript.
- Dual Data Source Configuration for AI Content Retrieval: This feature enables AI to access information from two distinct data sources, each mapped to a unique identifier for efficient retrieval and context-specific responses.
- Java 21
- SpringBoot 3.3.3: Backend framework for building fast and scalable applications.
- Tavily API: Tavily is a search engine tailored for AI agents.
- Qdrant: Open Source and Purpose Built — Qdrant Filters Enable You to Apply Arbitrary Business Logic on Top of a Similarity Search.
- Together AI: Provides models for describe image. Model: Meta Llama 3.2 90B Vision Instruct Turbo
- Langchain4j: Supercharge your Java application with the power of LLMs.
- Set up the following environment variables.
- export QDRANT_GRPC_HOST=your_host;
- export QDRANT_API_KEY=your_api_key;
- export TOGETHER_AI_API_KEY=your_api_key;
- export TAVILY_API_KEY=your_api_key;
- Run the command: mvn exec:java -Dspring.profiles.active=local
- Open the following link in your browser: http://localhost:8208/api/swagger-ui/index.html#/
Feel free to open issues or submit pull requests to improve the project. Contributions are welcome!