Channel: Lobste.rs
Filed at: 2026-05-17T20:04:48Z
Draft
Title: cachebench: prompt-cache observability for Anthropic, OpenAI, and Bedrock
URL to submit: https://pypi.org/user/mukundakatta/ (or the direct GitHub repo if public)
Tags to select on Lobste.rs: python, llm, benchmark, performance
Submission summary (paste into the "Description" field or leave blank — Lobste.rs surfaces the URL title by default; use this as the basis for any comment you add):
cachebench is a Python library for measuring LLM prompt-cache performance across Anthropic, OpenAI, and AWS Bedrock.
Most applications that enable prompt caching do not know whether it is working. The provider dashboard shows aggregate spend, not per-request cache hit rate. cachebench instruments each API call and surfaces the numbers that are otherwise invisible: hit rate per model and endpoint, how many tokens were served from cache versus recomputed, cost savings versus an uncached baseline, and latency distribution for cold versus warm calls.
It works as a standalone benchmark harness or as a thin wrapper around existing API calls. No changes to prompt structure are required. Outputs are structured: you can log them to a file, chart them in a dashboard, or diff them across deploys to catch regressions when you change prompt templates.
The library covers all three major providers in one package so you can compare cache behavior across provider boundaries without writing separate instrumentation for each one.
Available on PyPI: https://pypi.org/user/mukundakatta/
GitHub profile (all packages): https://github.com/MukundaKatta
Where to post
Submit at: https://lobste.rs/stories/new
Steps:
- Log in to your Lobste.rs account (invite-only; if you do not have an account, ask a member for an invitation link)
- Click "Submit story"
- Paste the PyPI or GitHub URL for cachebench into the URL field
- Set the title to:
cachebench: prompt-cache observability for Anthropic, OpenAI, and Bedrock
- Select tags:
python, llm (and performance or benchmark if available)
- Optionally paste the summary above into the description field or as your first comment after submission
Action
Review, edit if needed, then post. Close this issue when shipped.
Channel: Lobste.rs
Filed at: 2026-05-17T20:04:48Z
Draft
Title: cachebench: prompt-cache observability for Anthropic, OpenAI, and Bedrock
URL to submit: https://pypi.org/user/mukundakatta/ (or the direct GitHub repo if public)
Tags to select on Lobste.rs: python, llm, benchmark, performance
Submission summary (paste into the "Description" field or leave blank — Lobste.rs surfaces the URL title by default; use this as the basis for any comment you add):
cachebench is a Python library for measuring LLM prompt-cache performance across Anthropic, OpenAI, and AWS Bedrock.
Most applications that enable prompt caching do not know whether it is working. The provider dashboard shows aggregate spend, not per-request cache hit rate. cachebench instruments each API call and surfaces the numbers that are otherwise invisible: hit rate per model and endpoint, how many tokens were served from cache versus recomputed, cost savings versus an uncached baseline, and latency distribution for cold versus warm calls.
It works as a standalone benchmark harness or as a thin wrapper around existing API calls. No changes to prompt structure are required. Outputs are structured: you can log them to a file, chart them in a dashboard, or diff them across deploys to catch regressions when you change prompt templates.
The library covers all three major providers in one package so you can compare cache behavior across provider boundaries without writing separate instrumentation for each one.
Available on PyPI: https://pypi.org/user/mukundakatta/
GitHub profile (all packages): https://github.com/MukundaKatta
Where to post
Submit at: https://lobste.rs/stories/new
Steps:
cachebench: prompt-cache observability for Anthropic, OpenAI, and Bedrockpython,llm(andperformanceorbenchmarkif available)Action
Review, edit if needed, then post. Close this issue when shipped.