🛣️ TollVision Analytics Platform

Deep-dive analytics for free-flow tolling & Intelligent Transport Systems.

Upload your passage / IVDC / transaction CSVs (or similar) and get an interactive, executive-grade dashboard — KPIs, advanced visualisations, AI analytics, and downloadable reports. No code changes to onboard new files.

---

✨ What it does

• Drag-and-drop upload — CSVs, multiple files, or a ZIP. Types are auto-detected by column signature, so any similarly-structured export works with zero code changes.
• Auto-clean — dedupe, timestamp parsing, class/lane/plaza normalisation, with a transparent quality report (nulls, duplicates, detection confidence).
• Executive KPIs — passages, transactions, completion & match rates, ANPR read rate, tolled vs video-only, duplicates, peak hour, and modelled revenue / average toll / leakage.
• Advanced charts — hourly demand, lane×hour heatmap, vehicle-mix donut, lane utilisation, plaza→class treemap, read-confidence box plots, event-class Pareto, and a plaza → class → outcome Sankey.
• AI analytics — per-minute volume anomaly detection (Isolation Forest with z-score fallback), lane demand clustering (KMeans), read-quality correlation, and plain-English auto-insights.
• Downloadable reports — one click for Excel, PDF, PowerPoint, HTML, plus cleaned CSV export.
• Filters — lane and vehicle class, applied across every tab.
• Dark/light theme, responsive layout, cached parsing.

Revenue is modelled. The source extracts carry no monetary field, so revenue and leakage are derived from a configurable tariff table (tollvision/config.py) and labelled "modelled" throughout — never presented as measured.

🧱 Tech stack

Python · Streamlit · Polars · DuckDB-ready · Plotly · scikit-learn · openpyxl · python-pptx · reportlab · Docker · GitHub Actions.

📁 Structure

tollvision-analytics/
├── app.py                     # Streamlit UI (tabs, filters, downloads)
├── tollvision/
│   ├── config.py              # schema specs, vehicle classes, tariff model
│   ├── schema.py              # detection · TS decode · normalisation · validation
│   ├── ingest.py              # CSV/ZIP/multi loading (Polars)
│   ├── clean.py               # cleaning pipeline + quality report
│   ├── kpis.py                # KPI engine + modelled revenue/leakage
│   ├── charts.py              # Plotly chart builders
│   ├── analytics.py           # anomaly / clustering / correlation / insights
│   └── reports.py             # Excel · PDF · PPTX · HTML exports
├── sample_data/               # bundled passage/ivdc/transaction samples
├── tests/                     # pytest suite (runs on sample data)
├── docs/                      # architecture · data dictionary · user & deploy guides
├── .streamlit/config.toml     # theme + upload limits
├── Dockerfile · docker-compose.yml
├── requirements.txt · runtime.txt · .env.example · .gitignore · LICENSE
└── .github/workflows/ci.yml   # tests + headless Streamlit boot + docker build

🚀 Quick start

git clone https://github.com/<you>/tollvision-analytics.git
cd tollvision-analytics
pip install -r requirements.txt
streamlit run app.py            # http://localhost:8501

The app opens on the bundled sample data. Switch the sidebar to Upload files to analyse your own — drop passage, ivdc, transaction CSVs (or a ZIP). See docs/user_guide.md.

Docker:

docker compose up --build

🧪 Testing

pytest -q

The suite runs entirely against sample_data/, so it needs no external files. A full-pipeline check against real extracts is available via python dev_e2e.py (point it at your file paths).

🚢 Deploy

One-command deploys for Streamlit Community Cloud, Render, Railway, Azure App Service, and Docker — see docs/deployment.md.

📚 Docs

• Architecture
• Data dictionary — the recognised schemas
• User guide
• Deployment guide

🗺️ Roadmap

• Persist parsed extracts to DuckDB for 10M+ row sessions
• GIS map view (requires plaza coordinates — add to config.py)
• Prophet/Holt-Winters forecasting once multi-day extracts are available
• Cross-dataset reconciliation (passage ↔ transaction ↔ IVDC) report

📄 License

MIT — see LICENSE.

---

A note on scope & honesty

This repository is a real, working analytics platform verified end-to-end on 166k/145k/136k-row extracts — every KPI, chart, insight and report is computed from real columns and runs green in tests/CI. Two deliberate scope decisions: (1) revenue is modelled (no money field exists in the data), and (2) forecasting/GIS are on the roadmap rather than faked, because single-day extracts without plaza coordinates can't support them honestly. Everything advertised above is implemented and tested.