🐢 Open-Source Evaluation & Testing library for LLM Agents

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

RAG evaluation without the need for "golden answers"

Framework for testing vulnerabilities of large language models (LLM).

RAG boilerplate with semantic/propositional chunking, hybrid search (BM25 + dense), LLM reranking, query enhancement agents, CrewAI orchestration, Qdrant vector search, Redis/Mongo sessioning, Celery ingestion pipeline, Gradio UI, and an evaluation suite (Hit-Rate, MRR, hybrid configs).

Open source framework for evaluating AI Agents

This project aims to compare different Retrieval-Augmented Generation (RAG) frameworks in terms of speed and performance.

A framework for systematic evaluation of retrieval strategies and prompt engineering in RAG systems, featuring an interactive chat interface for document analysis.

RAG Chatbot for Financial Analysis

EvalWise is a developer-friendly platform for LLM evaluation and red teaming that helps test AI models for safety, compliance, and performance issues

A comprehensive evaluation toolkit for assessing Retrieval-Augmented Generation (RAG) outputs using linguistic, semantic, and fairness metrics

EntRAG - Enterprise RAG Benchmark

Python SDK

Using MLflow to deploy your RAG pipeline, using LLamaIndex, Langchain and Ollama/HuggingfaceLLMs/Groq

BetterRAG: Powerful RAG evaluation toolkit for LLMs. Measure, analyze, and optimize how your AI processes text chunks with precision metrics. Perfect for RAG systems, document processing, and embedding quality assessment.

RAG Pipeline Evaluation and monitoring on AWS using RAGAS

Advanced Retrieval-Augmented Generation (RAG) system designed as an interactive learning portal for political analytics.

AI RAG evaluation project using Ragas. Includes RAG metrics (precision, recall, faithfulness), retrieval diagnostics, and prompt testing examples for fintech/banking LLM systems. Designed as an AI QA Specialist portfolio project.