A comprehensive Python framework for evaluating LLM-extracted structured data against ground truth labels. Supports binary classification, scalar values, and list fields with detailed performance metrics, confidence-based evaluation, and statistical uncertainty quantification via non-parametric bootstrap confidence intervals.
pytest for LLM apps - Test for grounding failures, prompt injection, safety violations, and regressions
Deterministic validation firewall that verifies AI-generated proposals against ground-truth state using immutable rules. Zero dependencies. Patent pending.
A unified suite of MCP suppressors that prevent hallucinations, enforce grounding, and stabilise agent behaviour.
A new package that helps developers integration-test AI and LLM applications by validating structured outputs. It takes a user's test scenario or prompt as input, sends it to an LLM, and uses pattern
Setting benchmarking standards for the LLM security!
Systematic AI evaluation framework that transforms subjective assessment into objective measurement. Reduce research time by 85% while maintaining 95%+ accuracy through multi-LLM validation.
Evidence-backed validation framework for AI agents, Cursor skills, MCP orchestrations, and agentic repos. Structured rule catalog, verified evidence standard, and certification output. Free, open-source, MIT.
GSMA telecom documentation dataset creation pipeline with hard negative generation for embedding training. Features concurrent LLM validation, semantic similarity ranking, and DVC-based reproducible data processing.
GenAI Chatbot Testing Framework using Python & Pytest | Validates intent, context, prompt variations, hallucination risks, safety, and response quality scoring
MCP-native post-generation validation layer for LLM outputs
Production-grade AI quality gates for LLM applications. Automating hallucination detection, semantic accuracy scoring, and adversarial safety testing for RAG & Agentic AI systems.
C# SDK for the Guardrails AI API -- LLM validation, hallucination detection, PII protection, and content safety
Governance layer for LLM-driven agent-based models — 6-stage validation pipeline catches hallucination, logical drift, and unsafe state mutation. 3 reference impls (flood, multi-agent flood, Colorado irrigation). Multi-LLM, full audit trails, MIT.
Validate LLM outputs with deterministic rules to block bad AI proposals before they reach production.
Add a description, image, and links to the llm-validation topic page so that developers can more easily learn about it.
To associate your repository with the llm-validation topic, visit your repo's landing page and select "manage topics."