AI-Powered RAG Assistant with Screenshot Integration

A highly secure, offline-first, and multimodal-ready RAG (Retrieval-Augmented Generation) assistant. Engineered with Spring Boot 3 (Spring AI), React 18 (Vite), and a Qdrant Vector Database, it runs entirely locally using Ollama. The application stands out by extracting both plain text and embedded illustrations, screenshots, and diagrams from documents, letting the AI ground its responses with high fidelity and display visual evidence alongside answers.

---

System Architecture

This system is built as a three-tier local architecture. It processes, embeds, indexes, and queries documents completely within your machine's perimeter.

graph TD
    %% Styling
    classDef default fill:#1a1b26,stroke:#7aa2f7,stroke-width:2px,color:#a9b1d6;
    classDef frontend fill:#1f2335,stroke:#7dcfff,stroke-width:2px,color:#7dcfff;
    classDef backend fill:#1f2335,stroke:#9ece6a,stroke-width:2px,color:#9ece6a;
    classDef database fill:#1f2335,stroke:#e0af68,stroke-width:2px,color:#e0af68;
    classDef ollama fill:#1f2335,stroke:#f7768e,stroke-width:2px,color:#f7768e;
    classDef cloud fill:#1f2335,stroke:#bb9af7,stroke-width:2px,color:#bb9af7;

    subgraph Client["Client Tier (React & Vite)"]
        UI["React Web App (Glassmorphism Layout)"]:::frontend
    end

    subgraph Server["Application Tier (Spring Boot 3)"]
        SecurityFilter["Spring Security & JWT Filter"]:::backend
        Controller["Upload & Chat Controllers"]:::backend
        Extractor["Document Processing Suite <br/>(Apache Tika + PDFBox / POI)"]:::backend
        AIService["Spring AI Integration Layer"]:::backend
        ImgStore["Local Image Upload Store"]:::backend
    end

    subgraph DataInf["Data & Inference Tier"]
        VectorDB[("Qdrant Vector Database")]:::database
        LocalInf["Local Inference <br/>(Ollama: phi3:mini / nomic-embed)"]:::ollama
        CloudInf["Cloud/Hybrid Inference <br/>(OpenRouter / OpenAI Cloud APIs)"]:::cloud
    end

    %% Ingestion Flow
    UI -->|1. Drag-and-Drop Documents| SecurityFilter
    SecurityFilter -->|2. Authorize Request| Controller
    Controller -->|3. Ingest stream| Extractor
    Controller -->|4a. Extract plain text chunks| AIService
    Extractor -->|4b. Carve embedded screenshots/images| ImgStore
    AIService -->|5. Generate Vector Embeddings| LocalInf
    AIService -->|5. Generate Vector Embeddings| CloudInf
    AIService -->|6. Index text + payload metadata| VectorDB
    ImgStore -.->|Link paths in payload| VectorDB

    %% RAG Query Flow
    UI -->|7. Ask Natural Language Query| SecurityFilter
    SecurityFilter -->|8. Authorize Request| Controller
    Controller -->|9. Forward prompt| AIService
    AIService -->|10. Fetch relevant document chunks| VectorDB
    VectorDB -->|11. Returns top matches & image paths| AIService
    AIService -->|12. Feed query + context to LLM| LocalInf
    AIService -->|12. Feed query + context to LLM| CloudInf
    LocalInf -->|13. Generate grounded answer| AIService
    CloudInf -->|13. Generate grounded answer| AIService
    AIService -->|14. Return structured JSON answer with sources and images| UI

Loading

---

Why This Technology Stack?

Every element in this architecture is selected to deliver maximum privacy, lightning-fast processing, and enterprise-grade extensibility on consumer-grade hardware.

Technology	Role	Why We Selected It
Spring Boot 3 & Spring AI	Backend Framework	Spring Boot 3 brings unmatched type-safety, rapid dependency injection, and native compilation capabilities to enterprise Java. Spring AI abstracts vector store operations, prompt engineering, and LLM integrations cleanly, allowing us to swap components or models with zero changes to core business logic.
React 18 & Vite	Frontend Interface	Vite delivers instantaneous Hot Module Replacement (HMR) and highly optimized production builds. React 18 allows us to create a premium, responsive glassmorphism UI with smooth asynchronous states during heavy document uploads and real-time streaming chat rendering.
Qdrant Vector Database	Vector Indexing	Written in Rust, Qdrant is an ultra-fast vector database engineered for production. It allows us to perform high-speed cosine similarity searches, and seamlessly supports complex payload filtering (allowing us to bind document metadata, sections, and extracted screenshot paths directly to the text vectors).
Ollama (phi3:mini & nomic-embed-text)	Local Inference	Ollama runs AI models locally on your CPU/GPU. nomic-embed-text provides high-quality 8192 token-context embeddings, and phi3:mini is an exceptionally powerful, lightweight 3.8B parameter instruct model. This ensures 100% data privacy and zero API costs.
Apache Tika & PDFBox / POI	Content Extraction Suite	Apache Tika handles multi-format parsing (PDF, DOCX, TXT, HTML) under a unified interface. Apache PDFBox and Apache POI carve out embedded screenshots, illustrations, and figures directly from the binary layouts of PDFs and Word documents, enabling our multimodal-like pipeline.

---

Key Features

• Grounded Inline Citations (Next-Gen Transparency): Replaces "black-box" responses. AI answers are annotated with inline citation markers like [1] matching dotted underlines. Hovering displays a glassmorphic tooltip containing the exact document source, page number, and original grounding context snippet.
• Generative Document Writer & Exporter: Features a dedicated, in-context technical drafting editor. Offers quick-pick templates (SLA Agreement, Tech Spec, Project Roadmap, Privacy Policy) and outputs beautifully styled drafts. Exporters run 100% offline via client-side Blob downloads and custom serif Print-to-PDF framers.
• High-Fidelity Motion Animations (motion.dev): Fluid, physics-based transitions driven by Framer Motion. Features glide-sliding active mode pills with spring physics, enter/exit sliding list elements inside the files hub, staggered mounting chat bubbles, and rotating smooth grounding accordion logs.
• Perfect Viewport Height Lock & Layout Integrity: Restricts workspace height dynamically to ensure a zero-scrollbar desktop-grade experience. Features stacked two-row file cards to eliminate text clipping or metadata badge overlaps, and isolates scrolling strictly to the ingested file lists.
• Token-Refresh Auto-Retry Resiliency: An automatic interceptor that catches any 401 Unauthorized or 403 Forbidden API responses, silently fetches a fresh signed JWT session from the backend, and transparently retries the failed API call. Also features proactive token validation on mount to guarantee a zero-interruption browser refresh experience.
• Isolated Multi-Workspaces (Qdrant Vector Pools): Segregates documents securely. Integrates Spring AI's .withFilterExpression("workspace == ...") payload queries in Qdrant, enabling isolated workspace swaps with instant context transitions.
• Visual RAG Pipeline: Ingests PDFs and DOCXs, automatically carves out embedded illustrations and screenshots, and showcases them in a lightbox gallery inside AI answers.
• Self-Correction Relevance Grader (Corrective RAG): Features a strict anti-hallucination guard that runs a rapid corrective grading step, gracefully refusing to answer if matching ground-truth is missing.
• Mobile-First Responsive UX: Employs a tailored, modern Outfit + Inter design system. Adapts cleanly across screens: full rows on Web, sliding drawer workspaces on Tablets, and touch bottom navigation bars on Mobile.
• Search Engine Optimized (SEO): Built with search-crawling directives, robot indexes, structured title tags, semantic markup, and descriptive meta keywords.

---

Setup & Execution

1. Download Local AI Models

Open a terminal on your host machine and run the following commands to pull the necessary models via Ollama:

# Pull the instruction-tuned chat model
ollama pull phi3:mini

# Pull the text embedding model
ollama pull nomic-embed-text

2. Launch the Application Stack

From the project root directory, run Docker Compose to build and spin up the frontend, backend, and Qdrant container:

docker-compose up --build

3. Access Services

• Web App UI: http://localhost:5173/ (or port 80 if running production Nginx)
• Backend API: http://localhost:8080/
• Qdrant DB Console: http://localhost:6333/dashboard

---

Security & Authentication Architecture

This project is built with a production-ready Zero-Trust Security Design:

• JSON Web Tokens (JWT): Secure endpoint boundaries are enforced for chat (/api/chat/**), ingestion (/api/documents/**), and writer (/api/writer/**).
• Sleek Client Auth Layer: Establishes silent background session validation when the app mounts, saving JWTs in localStorage and injecting standard Authorization: Bearer <token> headers into every call.
• Cross-Platform Path Traversal Protection: Implements robust directory anchoring and path normalization checks in Spring Boot controllers to ensure malicious players cannot retrieve system-level files using relative backslash attacks.
• Production Reverse Proxying: Routes React assets and API endpoints seamlessly through Nginx, bypassing raw browser CORS locks on public environments.

---

Repository Structure

rag-assistant/
├── backend/            # Spring Boot 3 Java Service
│   ├── src/            # Parsing, chunking, security, and Spring AI logic
│   ├── pom.xml         # Maven dependencies (Optimized: No H2 dependency)
│   └── .env.example    # Backend environment template
├── frontend/           # React 18 + Vite Web Application
│   ├── src/            # Login, ChatWindow, and FileHub components
│   ├── nginx.conf      # SPA routing & API reverse proxy configuration
│   └── package.json    # Node scripts and dependencies
├── uploads/            # Local carved image volumes (gitignored)
├── qdrant_data/        # Persistent database storage (gitignored)
├── .env.example        # Root environment variable templates
└── docker-compose.yml  # Docker multi-container orchestrator