OpenLLM Monitor is a plug-and-play, real-time observability dashboard for monitoring and debugging LLM API calls across OpenAI, Ollama, OpenRouter, and more. Tracks tokens, latency, cost, retries, and lets you replay prompts — fully open-source and self-hostable.

A streamlined and easy-to-use AI performance evaluation / summary template with modern UI in HTML, including correct percentage chart and comparison with other models, precision, recall, F1-score, and confusion matrix. Enables you to create the result chart within 3 minutes.