Talk-to-a-Folder

A lightweight Retrieval-Augmented Generation (RAG) system that allows users to query documents inside a Google Drive folder and receive grounded answers with citations.

This project demonstrates an end-to-end workflow including document ingestion, semantic retrieval, and LLM-based answer generation.

---

Features

• Load documents directly from a Google Drive folder
• Automatic document parsing (.txt, .pdf, .docx)
• Word-based document chunking
• Embedding-based semantic retrieval
• LLM-powered grounded answers
• Inline source citations
• Simple and responsive UI

---

Architecture

Frontend

• React
• TypeScript
• Vite

Responsibilities:

• Folder input UI
• Question input
• Display answers
• Display citations

---

Backend

• FastAPI
• Google Drive API
• Sentence Transformers
• OpenAI API

Responsibilities:

• Load documents from Google Drive
• Extract text from files
• Chunk documents
• Generate embeddings
• Retrieve relevant chunks
• Generate answers

---

Pipeline

Drive → Parse → Chunk → Embed → Retrieve → Generate → Answer

---

Design Decisions

Why Google Drive Integration

Google Drive is commonly used in real workflows. Supporting Drive-based ingestion demonstrates handling external document sources rather than relying only on local files.

---

Why In-Memory Storage

Embeddings are stored in memory for simplicity.

Trade-offs:

Pros:

• Fast retrieval
• Simple setup
• Minimal infrastructure

Cons:

• No persistence
• Limited scalability

In production, this would be replaced with a vector database such as:

• FAISS
• Pinecone
• Weaviate

---

Why Sentence Transformers

Model used:

all-MiniLM-L6-v2

Reasons:

• Lightweight
• Fast inference
• Good semantic similarity performance
• Suitable for retrieval-based systems

---

How to Run

Backend

cd backend
pip install -r requirements.txt
uvicorn app:app --reload

Backend runs at:

http://127.0.0.1:8000

Frontend

cd frontend
npm install
npm run dev

Frontend runs at:

http://localhost:5173

Google Drive OAuth Setup

This project uses Google Drive API with OAuth authentication.

For security reasons, OAuth credentials are not included in the repository.

Follow these steps to set up access:

1. Go to Google Cloud Console
   https://console.cloud.google.com/

2. Create a new project

3. Enable Google Drive API

4. Create OAuth credentials:

   • Go to APIs & Services → Credentials
   • Click Create Credentials → OAuth Client ID
   • Select Desktop App
   • Download the credentials file

5. Rename the downloaded file to: client_secret.json

6. Place it inside: backend/

7. The first time you load a folder, a browser window will open to authorize access.
   A token.json file will be created automatically for future use.

Supported File Types

The system currently supports the following file types:

• .txt
• .pdf
• .docx

Files are automatically downloaded, parsed, and indexed for retrieval.

---

Example Usage

1. Paste a Google Drive folder link
2. Click Load to index the folder
3. Wait until indexing completes
4. Ask a natural language question

Example questions:

• What are the main goals mentioned in these documents?
• Summarize key risks.
• What deadlines are referenced?
• What architectural decisions were made?

---

Limitations

This implementation focuses on clarity and simplicity. The following limitations are known:

• Uses in-memory storage for embeddings
• Supports small-to-medium folders
• Retrieval limited to top-k matching chunks
• Chunking is word-based, not semantic
• No multi-turn conversation memory
• Parsing quality depends on document format

These limitations reflect deliberate trade-offs to keep the system lightweight and easy to understand.

---

Future Improvements

Potential enhancements include:

• Add vector database support (FAISS, Pinecone, etc.)
• Improve document parsing and structure detection
• Implement semantic chunking
• Add chat history support
• Deploy backend and frontend to cloud infrastructure
• Add authentication and access control

---

Tech Stack

Frontend

• React
• TypeScript
• Vite

Backend

• FastAPI
• Python

Machine Learning / NLP

• Sentence Transformers
• OpenAI API

Integrations

• Google Drive API

---

Repository Structure

talk-to-a-folder/
│
├── backend/
│   ├── app.py
│   ├── requirements.txt
│   ├── token.json
│   ├── client_secret.json
│   │
│   └── rag/
│       ├── init.py
│       ├── chunker.py
│       ├── embedder.py
│       ├── retriever.py
│       ├── llm_generator.py
│       ├── drive_loader.py
│       ├── drive_utils.py
│
├── frontend/
│   ├── package.json
│   ├── vite.config.ts
│   │
│   └── src/
│       ├── App.tsx
│       ├── App.css
│       ├── main.tsx
│
└── README.md

---

Business Impact

Systems like this help organizations:

• Reduce manual document search time
• Improve knowledge accessibility
• Enable faster decision-making
• Support internal knowledge workflows

Typical real-world use cases:

• Internal knowledge search
• Policy and documentation lookup
• Customer support automation
• Document summarization

---

Next Steps

If extended further, this system would include:

• Persistent vector storage
• Cloud deployment
• Monitoring and logging
• Query analytics
• Performance optimization

---

Author

Built as part of an engineering take-home assignment demonstrating:

• System design
• Retrieval-Augmented Generation (RAG)
• LLM integration
• Full-stack development