Table of Contents
The AI Video Retrieval (AIVR) system enhances video search by integrating deep learning models for speech recognition, image captioning, and embedding generation. It uses txtai for indexing and Django for integration, enabling real-time video processing and semantic search. The system retrieves precise video timestamps based on natural language queries. A usability study confirms its improved retrieval accuracy and efficiency over traditional methods. Future extensions include OCR, object detection, and action recognition for enhanced relevance.
To develop an AI-powered video retrieval system that enables precise semantic search within videos by leveraging deep learning models for automatic speech recognition, image captioning, and embedding generation. The system aims to enhance retrieval accuracy, efficiency, and usability by indexing multimodal data and providing timestamp-aligned search results, with potential extensions for OCR, object detection, and action recognition. The content will be formatted for various platforms, including:
- Develop an AI-driven semantic video retrieval system – Leverage deep learning models for speech recognition, image captioning, and embedding generation to enable accurate and efficient video search.
- Enhance retrieval accuracy and timestamp precision – Implement multimodal indexing and vector embeddings to improve search relevance and ensure precise timestamp alignment for retrieved segments.
- ** Ensure scalability and real-time processing** – Design a framework that supports real-time video uploads, indexing, and search queries while allowing future enhancements like OCR, object detection, and action recognition.
This section should list any major frameworks/libraries used to bootstrap your project. Leave any add-ons/plugins for the acknowledgements section. Here are a few examples.
The project can be viewed and run locally by installing the following tools.
- Install Docker Container
https://www.docker.com/products/docker-desktop/- Install GitHub
https://docs.github.com/en/desktop/installing-and-authenticating-to-github-desktop/installing-github-desktop- Use Git Clone to create a local repo
git clone https://github.com/iaminhri/semanticSearch.git
cd semanticSearch- Build the docker project by using the docker-compose-deploy.yml file.
docker-compose -f docker-compose-deploy build- Run the Project:
docker-compose -f docker-compose-deploy up- Access The Website:
127.0.0.1:8080
See LICENSE.txt for more information.
The authors wish to acknowledge the Responsible & Applied Machine Learning Laboratory (RAML Lab) at the Department of Computer Science, Brock University, Canada.