AutoResearch + PromptFoo = AutoPrompter. Run it with Neo AI Engineer
-
Updated
Apr 16, 2026 - Python
#
promptfoo
Here are 90 public repositories matching this topic...
FinSecAI: An experimental testbed for evaluating Prompt Injection, Jailbreak, and data extraction vulnerabilities in financial LLM agents via MCP and Ollama.
mcp jailbreak tcc adversarial-attacks ai-security fastapi prompt-injection llm-security ollama promptfoo
-
Updated
Oct 9, 2026 - Python
AI customer-support agent that approves, denies or escalates refunds. The LLM proposes, deterministic code decides. 100% policy-correct on 83 golden evals, 0 breaches in 120 red-team attacks, load-tested to 2,500 users. LangGraph + FastAPI + React.
react python kubernetes redis typescript postgresql load-testing customer-support ai-safety opik red-teaming fastapi guardrails ai-agent llm prompt-injection llm-evaluation promptfoo langgraph agentic-ai
-
Updated
Oct 13, 2026 - Python
Agent 测试转型手册：资深自动化测试工程师的 AI QE 学习路径 + 练手代码
qa benchmark test-automation pytest chinese quality-gate evals langfuse llm-evaluation promptfoo ragas llm-as-judge agentic-ai deepeval agent-testing agent-eval ai-qe
-
Updated
Sep 16, 2026 - Python
Orchestrator/sub-agent RCA system with a production-grade eval and observability harness — LangGraph agents, Ragas/Promptfoo evals, OTel tracing.
observability root-cause-analysis pydantic opentelemetry ai-product-management llm-evaluation promptfoo langgraph ragas agentic-ai
-
Updated
Sep 2, 2026 - Python
面向开发者的中文 Eval Harness 源码教材：解析 lm-evaluation-harness、Inspect AI、OpenAI Evals、Promptfoo、DeepEval 与 Harbor，覆盖任务、运行、评分、统计与发布门禁。
benchmark chinese source-code-analysis llm lm-evaluation-harness openai-evals promptfoo llm-as-a-judge deepeval inspect-ai eval-harness
-
Updated
Aug 27, 2026 - Python
Avaliação de qualidade e segurança (Go/No-Go) da assistente virtual Lia (LLM) para a loja Pop & Doce. Projeto final do Hackathon AWS AI FDE, envolvendo avaliação automatizada, Red Teaming e engenharia de prompt.
software-testing red-teaming quality-engineering prompt-engineering generative-ai llm-evaluation promptfoo deepeval aws-hackathon
-
Updated
Oct 19, 2026 - Python
Compare models across prompts, test domains, and scenarios.
-
Updated
Apr 5, 2026 - Python
AI Testing Framework using Promptfoo for automated LLM evaluation, red teaming, and CI/CD integration with Jenkins. Supports prompt validation, security testing, custom assertions, API-based model testing, and quality gates for enterprise GenAI applications.
jenkins framework ai ci-cd cicd jenkins-pipeline evaluation-framework llm prompt-engineering generative-ai ai-testing llmops prompt-injection llm-security llm-tools llm-evaluation llm-agents promptfoo
-
Updated
Oct 12, 2026 - Python
AISRF, the AI Security & Research Framework: every LLM request from your apps and agents becomes a ticket a human approves before it reaches the backend. Analyzers, guardrails (Rebuff, LLM Guard, NeMo, Lakera), red teaming (garak, promptfoo, PyRIT), source code review, reports, MCP server.
mcp human-in-the-loop red-teaming ai-security fastapi guardrails pyrit ai-red-team prompt-injection llm-security promptfoo llm-gateway garak owasp-llm-top10
-
Updated
Sep 27, 2026 - Python
Python + SQL toolkit for LLM threat intelligence — flags prompt injection, jailbreaks, and data exfiltration mapped to OWASP LLM Top 10 (2025) & MITRE ATLAS.
python sql threat-intelligence red-teaming ai-security detection-engineering pyrit adversarial-ml mitre-atlas prompt-injection llm-security promptfoo owasp-llm-top-10 garak
-
Updated
Aug 17, 2026 - Python
7 AI red-teaming tools (garak, promptfoo, DeepTeam, PyRIT, Azure AI Red Teaming Agent, …) vs one real Microsoft Foundry agent, scored from the wire by deterministic oracles and a tool-blind judge. Raw findings published.
benchmark azure evaluation ai-agents red-teaming ai-security pyrit prompt-injection llm-security promptfoo garak microsoft-foundry
-
Updated
Oct 9, 2026 - Python
Prompt optimization and benchmarking for local LLMs: picks the right prompt technique for your task and measures it on Ollama. DSPy-compatible.
python nlp benchmarking ai structured-output mlops dspy llm prompt-engineering chain-of-thought llmops prompt-testing local-llm ollama llm-evaluation prompt-optimization promptfoo prompt-techniques llm-benchmark prompt-compiler
-
Updated
Oct 4, 2026 - Python
Open-source eval harness for RAG and agent systems. Ragas + promptfoo + custom rubrics. Maintained by Paiteq, used by Paiteq + GetWidget.
-
Updated
May 22, 2026 - Python
AI/LLM test strategy for an e-commerce product recommendation engine — prompt regression, hallucination detection, toxicity safety gate, latency SLOs, and contract testing
portfolio ecommerce pytest pact k6 github-actions qa-automation fastapi ai-testing promptfoo hallucination-detection llm-testing prompt-regression
-
Updated
May 25, 2026 - Python
-
Updated
Jun 8, 2026 - Python
AI Testing Beyond the Basics: Ensuring Truthful and Reliable Chatbots and Agents Demos
-
Updated
Oct 12, 2026 - Python
Agent-native framework for heuristic eval feedback loops, evidence, decisions, and eval assets.
-
Updated
Jun 15, 2026 - Python
Uses Claude Agents SDK to decode job listings into the problems a company is actually hiring to solve and proposes representative projects.
job-listings ai-evaluation prompt-engineering promptfoo llm-as-judge agentic-ai claude-code ai-hiring claude-agents claude-agent-sdk
-
Updated
Sep 7, 2026 - Python
Add this topic to your repo
To associate your repository with the promptfoo topic, visit your repo's landing page and select "manage topics."