A bot designed to read and analyze financial PDF reports. Users can ask questions in plain English, and it will find and deliver relevant information, eliminating the need to manually search through lengthy documents.
- Conversational Q&A: Interact with financial PDFs using natural language to get clear, concise answers
- Advanced RAG Pipeline: Utilizes Retrieval-Augmented Generation to find and synthesize information
- Conversational Memory: Retains context over the last six turns for complex follow-up questions
- High-Performance Models: Powered by
google/flan-t5-basefor generation andall-MiniLM-L6-v2for semantic retrieval - Extendable: Fine-tune models on your own data to improve performance for specific domains
The application uses a Retrieval-Augmented Generation (RAG) architecture:
- PDF Ingestion: Extracts all text from the source PDF
- Chunking: Segments text into smaller, overlapping chunks to maintain context
- Embedding: Converts text chunks into numerical vectors (embeddings)
- Retrieval: Finds most relevant text chunks when a question is asked
- Synthesis: Generates a final, human-like answer using the FLAN-T5 model
- Core: Python 3.9+
- AI Frameworks: PyTorch, Hugging Face (transformers, sentence-transformers)
- PDF Processing: PyMuPDF (fitz)
- Text Utilities: NLTK
pdf_path = "your_document.pdf"python main.pyAzure
βββ modules_download
β βββ nlt.py #to download punkt
β
βββ testing #contains some testings(for personal test)
β
βββ training_model
β βββ inference.py #testing with a single prompt
β βββ train_model_answers.py #for training FLAN-T5
β βββ train_new_knowledge.py #for training all-MiniLM-L6-v2
β
βββ main.py #main file which requires pdf