A production-ready, enterprise-grade Retrieval-Augmented Generation system built with Next.js 15 and FastAPI. Processes and analyzes multimodal content (text, images, audio, video) with advanced knowledge graph capabilities, hybrid search, and comprehensive evaluation frameworks.
| Component | Status | Version |
|---|---|---|
| Frontend | Production Ready | Next.js 15.1.3 |
| Backend | Production Ready | FastAPI 0.104.1 |
| Testing | Comprehensive | Jest, Playwright, pytest |
| Deployment | Docker + K8s | Helm charts available |
- Text: PDF and TXT ingestion with OCR capabilities
- Images: Object detection, scene recognition, text extraction
- Audio: Speech-to-text with speaker diarization (Whisper)
- Video: Frame extraction and audio transcription
- Hybrid Search: Combines vector, graph, and keyword search with reranking
- Cross-Modal Discovery: Find related content across different file types
- Entity Navigation: Interactive knowledge graph exploration
- Real-time Results: Sub-second search response times
- CrewAI-powered specialized agents (orchestrator, retrieval, graph, vector, QA, synthesis)
- Workflow types: factual lookup, reasoning, multimodal
- Automatic query classification and routing
- WebSocket infrastructure supporting 10,000+ concurrent connections
- Live document processing status with multi-stage visualization
- Automatic reconnection with heartbeat monitoring
- JWT authentication with secure WebSocket token handling
- SQL injection prevention via validated enums
- Role-based access control with audit logging
- CORS protection with explicit allowlists
- Framework: Python 3.11, FastAPI, Uvicorn/Gunicorn
- Databases: PostgreSQL, Neo4j 5.15, Qdrant, Redis
- Processing: Celery for background jobs
- AI/ML: OpenAI, Anthropic, sentence-transformers, Whisper, spaCy, CrewAI
- Observability: Prometheus, OpenTelemetry, Sentry, structlog
- Framework: Next.js 15.1.3, React 18, TypeScript
- UI: Tailwind CSS, shadcn/ui, Radix UI
- State: Zustand, TanStack Query v5
- Visualization: Recharts, Cytoscape, vis-network
- Testing: Jest, React Testing Library, Playwright
- Docker Compose (development/production)
- Kubernetes with Helm charts
- Terraform for AWS resources
- GitHub Actions CI/CD
- Node.js 18.17+
- Python 3.11+
- Docker 24.0+ and Docker Compose
- 16GB RAM minimum
# Clone the repository
git clone https://github.com/yourusername/multimodal-rag-system.git
cd multimodal-rag-system
# Start all services with Docker
docker-compose -f docker-compose.development.yml up -d
# Frontend development
cd frontend
npm install
npm run dev
# Access the application
# Frontend: http://localhost:3000
# Backend API: http://localhost:8000
# API Docs: http://localhost:8000/docs| Role | Password | |
|---|---|---|
| Admin | admin@multimodal-rag.com | admin123 |
| Demo | demo@multimodal-rag.com | demo123 |
# Run backend tests
pytest tests/ --cov=src
# Start backend locally
uvicorn src.main:app --reload --port 8000
# View logs
docker-compose -f docker-compose.development.yml logs -f backendcd frontend
npm run dev # Development server
npm run build # Production build
npm run test # Unit tests
npm run test:e2e # E2E tests with Playwright
npm run type-check # TypeScript validation
npm run validate # Full validation suiteComprehensive documentation is available in the docs/ directory:
| Category | Description |
|---|---|
| Architecture | System design and component architecture |
| Deployment | Deployment strategies and CI/CD guides |
| Database | Schema documentation and migration guides |
| Security | Security audits and authentication strategies |
| Testing | Testing guides and validation reports |
| Operations | Monitoring, runbooks, and observability |
| API | API documentation and OpenAPI specs |
| Guides | Quick start and usage guides |
The system tracks RAG quality using the RAG Triad:
| Metric | Threshold | Description |
|---|---|---|
| Answer Relevancy | >70% | Response relevance to query |
| Faithfulness | >90% | Grounding in retrieved context |
| Contextual Relevancy | >70% | Retrieved context relevance |
| Latency | <2000ms | Response time target |
| Hallucination Rate | <10% | Unsupported claims threshold |
| Service | URL | Port |
|---|---|---|
| PostgreSQL | localhost | 5432 |
| Neo4j | bolt://localhost | 7687 |
| Qdrant | http://localhost | 6333 |
| Redis | redis://localhost | 6379 |
.
├── backend/ # FastAPI application
│ ├── src/
│ │ ├── api/ # Route handlers
│ │ ├── services/ # Business logic
│ │ ├── models/ # SQLAlchemy models
│ │ └── core/ # Configuration, auth, middleware
│ └── tests/ # Backend tests
├── frontend/ # Next.js application
│ ├── app/ # App Router pages
│ ├── src/
│ │ ├── components/ # React components
│ │ ├── hooks/ # Custom hooks
│ │ ├── store/ # Zustand stores
│ │ └── services/ # API clients
│ └── tests/ # Frontend tests
├── docs/ # Documentation
├── infrastructure/ # Terraform, K8s configs
└── docker-compose.*.yml # Docker configurations
- Create a feature branch from
develop - Write tests for new functionality
- Ensure all tests pass:
npm run validateandpytest - Submit a pull request
MIT License - see LICENSE for details.