Explainable ML Phishing Detection — SHAP-Powered Feature Attribution on a Production-Grade Ensemble
Modern phishing detection models achieve high accuracy — but tell you nothing about why a URL is dangerous. A security analyst seeing a bare "PHISHING" verdict cannot act on it, explain it to a client, or tune their defenses against it.
PhishExplain solves this by wrapping a production-grade phishing detection ensemble with SHAP (SHapley Additive exPlanations), making every prediction fully transparent:
- What features drove the classification
- How much each feature contributed to the risk score
- Why in plain English — no ML background required to understand the output
This project extends the Kavach phishing detection backend — a two-layer cascading architecture with a ROC AUC of 0.9744 on 124,902 balanced URLs — with a FastAPI service and interactive dark-theme frontend designed for security analyst workflows.
| Metric | Value |
|---|---|
| ROC AUC | 0.9744 |
| Training Dataset | 124,902 URLs (perfectly balanced) |
| Malicious Sources | PhishTank + URLhaus |
| Legitimate Sources | Tranco Top 1 Million |
| Ensemble | LightGBM (40%) + TF-IDF N-gram LR (60%) |
| URL | Verdict | Risk Score | Top SHAP Feature |
|---|---|---|---|
https://google.com |
✅ SAFE | 18.5% | https_domain (−1.94) |
http://192.168.1.104/login/secure/verify-account.php |
🚨 PHISHING | 99.9% | https_domain (+2.80) |
https://xn--80ak6aa92e.com/ |
🚨 PHISHING | 88.9% | num_path_segments (+3.14) |
URL Input
│
▼
┌─────────────────────────────────────────┐
│ LAYER 1: Lexical Ensemble │
│ │
│ URL → 26 Features → LightGBM ──┐ │
│ ├─ Weighted Score (0.0–1.0)
│ URL → TF-IDF N-grams → LR ─────┘ │
│ (40% LGBM / 60% N-gram) │
└─────────────────────────────────────────┘
│
├── Score < 0.40 → ✅ SAFE
├── Score 0.40–0.65 → ⚠️ SUSPICIOUS
└── Score > 0.65 → 🚨 PHISHING
│
▼
┌─────────────────────────────────────────┐
│ SHAP EXPLAINABILITY LAYER │
│ │
│ TreeExplainer → SHAP values per feature│
│ → Top-5 attributions │
│ → Plain-English explanations │
│ → Contribution direction (±) │
└─────────────────────────────────────────┘
│
▼
JSON Response + Interactive Frontend
PhishExplain extracts 26 structured lexical features from every URL:
| Category | Features |
|---|---|
| Structure | url_length, domain_length, path_length, query_length, num_path_segments, num_subdomains |
| Characters | num_dots, num_hyphens, num_digits, num_slashes, num_at_symbols, special_chars, has_encoded_chars |
| Entropy | url_entropy (Shannon entropy — measures character randomness) |
| Protocol | https_domain, has_port, has_ip_address |
| Domain | is_punycode, is_shortened, suspicious_tld, is_common_tld, tld_length |
| Content | keyword_count, brand_token_count, leet_brand_match |
| Infrastructure | ssl_valid, dns_num_ips, domain_age_days (Layer 2 — static in Layer 1) |
SHAP (SHapley Additive exPlanations) provides theoretically grounded feature attribution based on cooperative game theory. For each prediction, SHAP calculates the exact marginal contribution of every feature by averaging over all possible feature orderings. This means:
- Attribution values are consistent — a feature that always increases risk always has a positive SHAP value
- Attribution values are locally accurate — they sum to the model's output for that specific prediction
- Results are comparable across predictions — SHAP values for different URLs use the same scale
# Clone the repository
git clone https://github.com/Vasco1290/PhishExplain.git
cd PhishExplain
# Create virtual environment (Windows)
python -m venv phishexplain_env
phishexplain_env\Scripts\activate
# Install dependencies
pip install -r requirements.txtRequirements: Python 3.9+, FastAPI, LightGBM, SHAP, scikit-learn, tldextract, pandas, uvicorn
PhishExplain requires the pre-trained Kavach V9 model files. Place the following .pkl files in the models/ directory:
models/
├── ensemble_lgbm_v9.pkl ← LightGBM model
├── ensemble_ngram_model_v9.pkl ← Logistic Regression (N-gram)
├── ensemble_ngram_vectorizer_v9.pkl ← TF-IDF vectorizer
├── ensemble_weights_v9.pkl ← Ensemble weights
└── feature_columns_v9.pkl ← Feature column order
Model files are excluded from version control (.gitignore). The
models/directory is provided empty.
uvicorn app.main:app --reloadSuccessful startup shows:
INFO: All ensemble models loaded successfully.
INFO: SHAP TreeExplainer initialized successfully.
INFO: Application startup complete.
Navigate to http://localhost:8000 — the interactive dark-theme dashboard loads automatically.
# Predict a URL
curl -X POST "http://localhost:8000/predict" \
-H "Content-Type: application/json" \
-d '{"url": "http://paypal-secure.tk/login/verify"}'Response:
{
"url": "http://paypal-secure.tk/login/verify",
"prediction": "Phishing",
"confidence": 0.97,
"risk_score": 0.97,
"top_features": [
{
"feature": "suspicious_tld",
"value": 1,
"shap_value": 2.14,
"contribution": "Positive (increases phishing risk)",
"explanation": "URL uses a high-risk top-level domain"
},
{
"feature": "keyword_count",
"value": 2,
"shap_value": 1.87,
"contribution": "Positive (increases phishing risk)",
"explanation": "URL contains 2 suspicious keywords"
}
],
"layer": "Layer 1 (Lexical)",
"processing_time_ms": 14.2
}curl http://localhost:8000/healthPhishExplain/
├── app/
│ ├── main.py ← FastAPI app, CORS, endpoints, startup
│ ├── explainer.py ← SHAP TreeExplainer wrapper, templates
│ ├── features.py ← 26-feature URL extractor (tldextract)
│ ├── predictor.py ← Ensemble prediction, pkl loading
│ └── schemas.py ← Pydantic request/response models
├── frontend/
│ └── index.html ← Single-file dark-theme UI (pure CSS/JS)
├── models/
│ └── .gitkeep ← Place .pkl files here
├── notebooks/
│ └── shap_exploration.ipynb
├── tests/
│ └── test_explainer.py
├── requirements.txt
└── README.md
pytest tests/ -vPhishing attacks cost organizations billions annually — yet most deployed detection systems are black boxes. A security operations center analyst receiving a "PHISHING" classification with no supporting evidence cannot:
- Explain the decision to a client or management
- Identify which adversarial evasion techniques are being used
- Tune detection thresholds based on feature-level patterns
- Build institutional knowledge about evolving attack patterns
PhishExplain addresses this gap directly. By surfacing the exact features driving each classification — and expressing them in plain English — it transforms a prediction into an actionable, explainable security finding.
This project extends production cybersecurity work developed during an internship at a security company, where a phishing and malicious URL detection model was deployed in a live environment. PhishExplain adds the explainability layer that production systems routinely lack.
@misc{rawal2026phishexplain,
author = {Harshit Kishor Rawal},
title = {PhishExplain: Explainable ML Phishing Detection with SHAP},
year = {2026},
publisher = {GitHub},
url = {https://github.com/Vasco1290/PhishExplain}
}- Lundberg, S.M., & Lee, S.I. (2017). A unified approach to interpreting model predictions. NeurIPS 2017.
- Ke, G., et al. (2017). LightGBM: A highly efficient gradient boosting decision tree. NeurIPS 2017.
- Mohammad, R.M., et al. (2012). An assessment of features related to phishing websites. ICITST 2012.
- 26-feature lexical URL extraction (tldextract)
- LightGBM + N-gram ensemble prediction
- SHAP TreeExplainer with plain-English attribution
- FastAPI backend with CORS
- Dark-theme interactive frontend
- Three-tier classification (Safe / Suspicious / Phishing)
- Layer 2 infrastructure checks (DNS, SSL, domain age)
- Batch URL analysis endpoint
- SHAP summary plots for dataset-level analysis
- Docker deployment configuration
MIT License. See LICENSE for details.
Built on top of the Kavach production phishing detection system · Explainability layer powered by SHAP