Repository navigation
feat(agent-compliance): PromptDefense + GovernanceVerifier integration example - #1837
Conversation
…ation example Demonstrates how PromptDefenseEvaluator integrates with GovernanceVerifier for pre-deployment governance checks: - Scan system prompts for missing defenses (12 attack vectors) - Block deployment if prompt grade falls below threshold - Generate audit entries for MerkleAuditChain - Batch evaluation across multiple agents Includes 4 progressive demos: weak vs strong prompt comparison, batch evaluation, and audit entry generation. Follow-up to microsoft#854 discussion with @ppcvote.
|
Welcome to the Agent Governance Toolkit! Thanks for your first pull request. |
🤖 AI Agent: breaking-change-detector — API CompatibilityAPI CompatibilityNo breaking changes detected. |
🤖 AI Agent: docs-sync-checker — Docs SyncDocs Sync
|
🤖 AI Agent: security-scanner — View detailsNo security issues found. |
🤖 AI Agent: test-generator — `prompt_defense_governance.py`
|
🤖 AI Agent: code-reviewer — Action Items:TL;DR: 0 blockers, 1 warning. Example implementation is functional and well-structured, but a minor improvement is suggested.
Action Items:
Warnings:
|
PR Review Summary
Verdict: |
|
lawcontinue — thanks for taking this through to a concrete integration example. Seeing A few notes that might tighten the demo, drawn from the production gap data we baselined this April (
I'd be glad to follow up with a PR adding (1) a "median prompt" demo case using anonymized gap-corpus data and (2) the v1.5 vector pass — let me know if either fits the project direction. — Min Yi (ppcvote / Ultra Lab) |
Imran Siddique (imran-siddique)
left a comment
There was a problem hiding this comment.
Clean integration example. Good use of fallback imports for standalone usage.
ddb581d
into
microsoft:main
…ation example (microsoft#1837) Demonstrates how PromptDefenseEvaluator integrates with GovernanceVerifier for pre-deployment governance checks: - Scan system prompts for missing defenses (12 attack vectors) - Block deployment if prompt grade falls below threshold - Generate audit entries for MerkleAuditChain - Batch evaluation across multiple agents Includes 4 progressive demos: weak vs strong prompt comparison, batch evaluation, and audit entry generation. Follow-up to microsoft#854 discussion with @ppcvote. Co-authored-by: deepsearch <deepsearch@deepsearchdeMac-mini.local>
Summary
Adds an integration example demonstrating how
PromptDefenseEvaluatorworks withGovernanceVerifierfor pre-deployment governance checks.Follow-up to the co-authoring discussion in #854 with MinYi Xie (@ppcvote).
What it shows
evaluate_batch()across multiple agentsto_audit_entry()output for MerkleAuditChain integrationis_blocking()decision based on configurable min gradeIntegration points
PromptDefenseEvaluator→evaluate()/evaluate_batch()/evaluate_file()PromptDefenseReport→is_blocking()for deployment decisionsto_audit_entry()→MerkleAuditChaincompatible audit log entriesGovernanceVerifier→verify()+evidence_checksfor full governance attestationTesting
Checklist
governed_agent.py,quickstart.py)