Supercharge Your LLM Application Evaluations 🚀

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

Python package for the evaluation of odometry and SLAM

The easiest tool for fine-tuning LLM models, synthetic data generation, and collaborating on datasets.

End-to-end Automatic Speech Recognition for Madarian and English in Tensorflow

A unified evaluation framework for large language models

Accelerating the development of large multimodal models (LMMs) with one-click evaluation module - lmms-eval.

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.

☁️ 🚀 📊 📈 Evaluating state of the art in AI

(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Multi-class confusion matrix library in Python

Evaluation code for various unsupervised automated metrics for Natural Language Generation.

XAI - An eXplainability toolbox for machine learning

FuzzBench - Fuzzer benchmarking as a service.

High-fidelity performance metrics for generative models in PyTorch