The diff tool for prompt engineers. Stop flying blind when you tweak system instructions.
"You wouldn't deploy code without reviewing the diff. Why would you deploy a prompt that way? Prompt-audit gives you a changelog for your model's brain."
# Clone the repository
git clone https://github.com/amanhammadK/prompt-audit.git
cd prompt-audit
# Make it executable (Unix)
chmod +x prompt_audit.py# Compare two files
python prompt_audit.py --old v1.txt --new v2.txt
# Compare raw strings
python prompt_audit.py --old "You are a helpful assistant." --new "You are a professional expert."
# Optional: Set your API key for behavioral predictions
export OPENAI_API_KEY='your-key-here'
python prompt_audit.py --old v1.txt --new v2.txt- Structural Diff: Highlight exactly what instructions were added, modified, or removed.
- Impact Scoring: A heuristic score (1-10) that flags high-risk changes (like removing safety constraints).
- Keyword Analysis: Automatically detects persona shifts, specificity increases, and volume changes.
- Behavioral Prediction: (Optional) Uses GPT-3.5 to summarize the expected change in model output.
MIT