AutoResearch + PromptFoo = AutoPrompter. Run it with Neo AI Engineer
-
Updated
Apr 16, 2026 - Python
#
promptfoo
Here are 164 public repositories matching this topic...
Your guide to the Agentic AI evolution. **Prompting Blueprints** offers a curated collection of concepts and tactics for building autonomous AI workflows. Master tool-specific playbooks, backed by structured prompt packs and rigorous evaluations for the latest AI models.
mkdocs gemini llm prompt-engineering generative-ai chatgpt prompt-library promptfoo context-engineering
-
Updated
Oct 20, 2026 - HTML
This repository contains the code snippets used in "LLM Prompt Engineering For Developers"
react python nlp natural-language-processing openai gpt perplexity few-shot-learning gpt-4 llm prompt-engineering chain-of-thought chatgpt langchain chatgpt-api azure-prompt-flow promptfoo general-knowledge-prompting betterprompt
-
Updated
Apr 22, 2024
FinSecAI: An experimental testbed for evaluating Prompt Injection, Jailbreak, and data extraction vulnerabilities in financial LLM agents via MCP and Ollama.
mcp jailbreak tcc adversarial-attacks ai-security fastapi prompt-injection llm-security ollama promptfoo
-
Updated
Oct 9, 2026 - Python
AI customer-support agent that approves, denies or escalates refunds. The LLM proposes, deterministic code decides. 100% policy-correct on 83 golden evals, 0 breaches in 120 red-team attacks, load-tested to 2,500 users. LangGraph + FastAPI + React.
react python kubernetes redis typescript postgresql load-testing customer-support ai-safety opik red-teaming fastapi guardrails ai-agent llm prompt-injection llm-evaluation promptfoo langgraph agentic-ai
-
Updated
Oct 13, 2026 - Python
prompt-evaluator is an open-source toolkit for evaluating, testing, and comparing LLM prompts. It provides a GUI-driven workflow for running prompt tests, tracking token usage, visualizing results, and ensuring reliability across models like OpenAI, Claude, and Gemini.
electron react typescript datascience developer-tools ai-evaluation llm prompt-engineering prompt-testing promptfoo ai-evaluation-tools ai-evaluation-metrics ai-evaluation-framework
-
Updated
Dec 4, 2025 - TypeScript
Community Plugin for Genkit to use Promptfoo
plugin testing firebase ai evaluation prompt prompts evaluation-framework llm llmops prompt-testing llm-eval llm-evaluation llm-evaluation-framework promptfoo genkit genkitx genkit-plugin
-
Updated
Jan 3, 2025 - TypeScript
Agent 测试转型手册：资深自动化测试工程师的 AI QE 学习路径 + 练手代码
qa benchmark test-automation pytest chinese quality-gate evals langfuse llm-evaluation promptfoo ragas llm-as-judge agentic-ai deepeval agent-testing agent-eval ai-qe
-
Updated
Sep 16, 2026 - Python
Orchestrator/sub-agent RCA system with a production-grade eval and observability harness — LangGraph agents, Ragas/Promptfoo evals, OTel tracing.
observability root-cause-analysis pydantic opentelemetry ai-product-management llm-evaluation promptfoo langgraph ragas agentic-ai
-
Updated
Sep 2, 2026 - Python
面向开发者的中文 Eval Harness 源码教材：解析 lm-evaluation-harness、Inspect AI、OpenAI Evals、Promptfoo、DeepEval 与 Harbor，覆盖任务、运行、评分、统计与发布门禁。
benchmark chinese source-code-analysis llm lm-evaluation-harness openai-evals promptfoo llm-as-a-judge deepeval inspect-ai eval-harness
-
Updated
Aug 27, 2026 - Python
Avaliação de qualidade e segurança (Go/No-Go) da assistente virtual Lia (LLM) para a loja Pop & Doce. Projeto final do Hackathon AWS AI FDE, envolvendo avaliação automatizada, Red Teaming e engenharia de prompt.
software-testing red-teaming quality-engineering prompt-engineering generative-ai llm-evaluation promptfoo deepeval aws-hackathon
-
Updated
Oct 19, 2026 - Python
VCL VibeBench — open frontier-model comparisons you can fork and re-run. By Vibe Coder's Life.
-
Updated
Aug 25, 2026 - JavaScript
Compare models across prompts, test domains, and scenarios.
-
Updated
Apr 5, 2026 - Python
AI Testing Framework using Promptfoo for automated LLM evaluation, red teaming, and CI/CD integration with Jenkins. Supports prompt validation, security testing, custom assertions, API-based model testing, and quality gates for enterprise GenAI applications.
jenkins framework ai ci-cd cicd jenkins-pipeline evaluation-framework llm prompt-engineering generative-ai ai-testing llmops prompt-injection llm-security llm-tools llm-evaluation llm-agents promptfoo
-
Updated
Oct 12, 2026 - Python
AISRF, the AI Security & Research Framework: every LLM request from your apps and agents becomes a ticket a human approves before it reaches the backend. Analyzers, guardrails (Rebuff, LLM Guard, NeMo, Lakera), red teaming (garak, promptfoo, PyRIT), source code review, reports, MCP server.
mcp human-in-the-loop red-teaming ai-security fastapi guardrails pyrit ai-red-team prompt-injection llm-security promptfoo llm-gateway garak owasp-llm-top10
-
Updated
Sep 27, 2026 - Python
Python + SQL toolkit for LLM threat intelligence — flags prompt injection, jailbreaks, and data exfiltration mapped to OWASP LLM Top 10 (2025) & MITRE ATLAS.
python sql threat-intelligence red-teaming ai-security detection-engineering pyrit adversarial-ml mitre-atlas prompt-injection llm-security promptfoo owasp-llm-top-10 garak
-
Updated
Aug 17, 2026 - Python
kotlin testing mcp moderation chatbot demo-app rag quarkus langchain4j llm-evaluation promptfoo rag-chatbot mcp-client
-
Updated
Oct 19, 2026 - Kotlin
Add this topic to your repo
To associate your repository with the promptfoo topic, visit your repo's landing page and select "manage topics."