🧠 ModelPulse — Stay Up to Date with the Fast-Moving World of LLMs

ModelPulse, A local, GPU-accelerated retrieval-augmented AI system to track the latest in LLM research.

---

ModelPulse is an open-source, GPU-accelerated retrieval-augmented system that helps developers and researchers stay up to date with the fast-moving world of LLMs — tracking new research, model releases, and blogs. Everything runs fully locally with Hugging Face models — no API calls required (optional: enable RAGAS evaluation with OpenAI API key).

---

📑 Table of Contents

• Overview
• Features
• Tech Stack
• Architecture
• Quick Start
• Manual Setup
• Example Queries
• Evaluation Metrics
• Repository Structure
• Configuration
• Roadmap
• License
• Acknowledgments

---

🧭 Overview

LLM research moves fast — new architectures, RAG techniques, and benchmarks appear weekly. ModelPulse acts as your personal AI radar, automatically:

• 📰 Collects updates from trusted sources like OpenAI, Anthropic, Hugging Face, and arXiv
• 🔍 Builds semantic search indexes for Q&A
• 🧠 Generates summaries and digests with citations
• 📊 Tracks faithfulness, latency, and cost metrics
• ⚙️ Adapts over time using feedback and metrics

---

Demo

Search

Perform semantic searches across the latest LLM research and documentation

---

Ask Questions with Citations

Ask questions and get grounded answers with source citations

---

Generate Research Digests

Automatically generate summaries and digests from retrieved content

---

✨ Features

🚀	Feature	Description
🧩	Hybrid Retrieval	Combines BM25 + FAISS vector search for optimal precision and recall
🧠	Grounded Summarization	Answers are cited and based on retrieved evidence
⚙️	Fully Local	Works offline with GPU inference — no API required
📊	Evaluation Dashboard	Visual metrics: latency, quality, and cost (optional with API key)
🧮	Adaptive Tuning	Learns retrieval parameters automatically (optional with API key)
📬	Topic Watchlists	Alerts you when new papers appear on your topics

---

🧰 Tech Stack

💡 Layer	🔧 Tools & Libraries
Ingestion	feedparser, beautifulsoup4, trafilatura
Embeddings	sentence-transformers, BAAI/bge-base-en-v1.5, intfloat/e5-base-v2
Retrieval	faiss-gpu, rank_bm25, cross-encoder/ms-marco-MiniLM-L-6-v2
Generation	Local LLMs (Qwen2.5-7B, Mistral-7B, Llama-3.1-8B)
Evaluation	ragas, scikit-learn, matplotlib
UI / Backend	Streamlit, FastAPI, SQLite
Deployment	Docker, docker-compose, NVIDIA GPU

---

🖥️ Architecture

┌──────────────────┐
│   Connectors     │  ← RSS, Blogs, arXiv, APIs
└───────┬──────────┘
        │
┌───────▼──────────┐
│ Indexing Layer   │  ← Chunking + Embeddings (BGE/E5)
└───────┬──────────┘
        │
┌───────▼──────────┐
│ Retrieval Layer  │  ← BM25 + FAISS + Cross-Encoder
└───────┬──────────┘
        │
┌───────▼──────────┐
│ Generation Layer │  ← Local LLMs (Qwen/Mistral/Llama)
└───────┬──────────┘
        │
┌───────▼──────────┐
│ Evaluation Layer │  ← RAGAS + latency + cost tracking
└───────┬──────────┘
        │
┌───────▼──────────┐
│  Streamlit UI    │  ← Dashboard + QA + Digests
└──────────────────┘

---

🚀 Quick Start (Docker Recommended)

# 1. Install NVIDIA Container Toolkit
./install_nvidia_docker.sh

# 2. (Optional) Configure OpenAI API key for RAGAS evaluation & adaptive tuning
# Skip this step for fully local operation without evaluation metrics
cp .env.example .env
nano .env
# Add your key: OPENAI_API_KEY=sk-proj-your-key

# 3. Start ModelPulse
./start.sh

# 4. Visit the dashboard
# → http://localhost:8501

🕐 First run: 15–30 min (downloads models and builds index). Next runs: ~30 sec (just launches the UI).

Note: Without OPENAI_API_KEY, ModelPulse runs 100% locally — ingestion, search, Q&A, and UI all work offline. The API key is only needed for RAGAS evaluation metrics and adaptive config tuning.

---

🧪 Manual Setup (Python)

git clone https://github.com/LeoFu9487/ModelPulse.git
cd ModelPulse
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Ingest and index data
python3 -m jobs.ingest_daily
python3 -m pipeline.chunk
python3 -m pipeline.embed

# Launch dashboard
python3 -m streamlit run ui/app_streamlit.py

---

💡 Example Queries

Q: What’s new in RAG evaluation this week?

A: A new metric called “context coherence” was introduced by Hugging Face [1],
   improving precision for long-form retrieval tasks [2].

Sources:
[1] https://huggingface.co/blog/ragas-update
[2] https://arxiv.org/abs/2401.01234

---

📊 Evaluation Metrics

Metric	Description	Requires API Key
Faithfulness	Alignment between generated answer and sources	Yes (RAGAS)
Answer Relevancy	Semantic relevance of generated answers	Yes (RAGAS)
Precision / Recall	Context retrieval accuracy	Yes (RAGAS)
Latency	Response time per query	No (local)
Cost	GPU compute cost per evaluation	No (local)
Confidence	Weighted similarity of top-k retrieved chunks	No (local)

Note: RAGAS-based metrics (faithfulness, relevancy, precision, recall) require OPENAI_API_KEY. All other features including latency tracking and the Streamlit dashboard work fully locally.

---

📦 Repository Structure

modelpulse/
├── connectors/        # Data sources
├── pipeline/          # Chunking & embedding
├── retriever/         # Hybrid + reranking logic
├── rag/               # Q&A and evaluation
├── ui/                # Streamlit app
├── jobs/              # Ingestion & digest tasks
├── storage/           # SQLite data
└── Dockerfile

---

⚙️ Configuration

config.yaml example:

embeddings:
  model: BAAI/bge-base-en-v1.5
retrieval:
  top_k: 8
  alpha_bm25: 0.5
generator:
  model: Qwen/Qwen2.5-7B-Instruct
  quantization_4bit: true
  temperature: 0.0

Restart after changes:

docker compose down && ./start.sh

---

🧭 Roadmap

• Active Learning Loop — feedback-based retrieval tuning
• Retrieval Compression Benchmark — dense vs sparse
• Fine-Tuned Domain Embeddings
• Multimodal Support (CLIP)
• Personalized Watchlists

---

🧾 License

MIT License © 2025 Yu-Peng FU

---

🙌 Acknowledgments

Thanks to the open-source community:

• SentenceTransformers
• FAISS
• RAGAS
• Streamlit
• Hugging Face

---

🌐 ModelPulse keeps you informed, grounded, and in sync with the latest in AI.

---