🗣️ Tool to generate adversarial text examples and test machine learning models against them

A Python library to help implement kurobako's solvers and problems

SCARF (System for Comprehensive Assessment of RAG Frameworks) is a modular evaluation framework for benchmarking deployed Retrieval Augmented Generation (RAG) applications. It offers end-to-end, black-box assessment across multiple configurations, supports automated testing with several vector databases and LLMs.