tokens-per-second

Reproducible on-device LLM benchmarks for Apple Silicon (iPhone 17 Pro, M4 Max): Apple Core AI, MLX, llama.cpp, LiteRT-LM and Core ML on the same model and harness, every number with its quantization and capture session; hybrid Mamba-2 models (Nemotron-3 Nano, Granite-4.0-H, Falcon-H1) included.

Community benchmark database for running LLMs on Apple Silicon Macs

The local-inference observability layer for your terminal: live tokens/sec, VRAM-spill & throttle warnings, the GPU metrics nvidia-smi can't show. Also a gorgeous htop/btop-class system monitor. Rust, zero deps.

Real-time terminal dashboard for Ollama — live output token counting (tokens/sec), CPU & thread usage, Apple Silicon GPU utilization, VRAM and loaded models. htop for your local LLM.

Linux kernel and systems fast path for LLM inference: eBPF tracing, runtime hints, cgroups, NUMA/GPU locality, KV-cache memory policies, TTFT boost, and experimental kernel primitives.

Pi extension that measures and displays model TPS in the status bar

A tiny always-on-top Windows window for ZCode: live output speed (tokens/sec) and prompt-cache hit rate. No console window, no network access.

Manifest V3 Chrome Extension that tracks real-time AI response time, token count & speed (tok/s) across 17+ web AI platforms — with a Side Panel dashboard, session history, and CSV/JSON export.

Live tokens/sec meters for coding-agent CLIs

Real-time tokens-per-second readout for AI coding agents — zero dependencies, local-only

Dynamically benchmarks every model on NVIDIA NIM — type, context window, reasoning/cache, TTFT & token rate — and renders an interactive, dated HTML report.

Hands-on CPU vs GPU benchmarks for Apple Silicon (M-series): PyTorch MPS, TensorFlow-Metal, MLX, and llama.cpp to measure TFLOP/s & tokens/sec and learn why GPUs accelerate training.

一款搭配 DSH 客户端的悬浮窗小工具，实时监测并显示会话的实时与平均 Token 输出速率（tokens/s）

Tokens-per-second in pi's footer + opt-in observation log

OpenCode TUI plugin for tool-wait-free session and per-model TPS metrics

Measuring proxy + throughput dashboard for local LLM engines (llama.cpp, NInfer): live tok/s, cache-hit rate, TTFT, and history charts. Single-file panel, stdlib-only proxy, MIT.

Quiet DSH-style decode TPS for Pi — stable cumulative model generation speed with tool execution excluded

Simple tool for measuring inference engine performance under multi-user load

Transparent, reproducible LLM inference metrology for latency, throughput, GPU telemetry, and serving performance.

Speedtest for AI. Test latency to every major AI provider from your terminal.