LLM Format Bridge

English | 中文

A lightweight proxy that translates between LLM API formats (OpenAI Chat Completions / Responses, Anthropic). Let downstream agents use their preferred format, bridge auto-converts to the upstream provider.

For full documentation, see docs/guide.md | 完整文档见 docs/guide.zh.md

git clone <repo> && cd llm-format-bridge && npm install
./index.js                 # 交互式菜单
./index.js start           # 启动服务
npm start                  # 同上

Quick Example

Downstream agent sends Anthropic format, bridge converts to OpenAI:

curl -X POST http://localhost:8080/v1/messages \
  -H "Authorization: Bearer my-bridge-key" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-4-20250514","messages":[{"role":"user","content":"Hi"}],"max_tokens":100}'

Features

• Format translation: openai_completions ↔ openai_responses ↔ anthropic (all 6 directions)
• Multi-modal: automatic image format conversion
• Streaming: SSE event format translation
• Model mapping: one-to-one with default fallback
• Auth: per-downstream API key validation
• Language: bilingual (中文 / English) CLI interface. Switch via --lang zh|en, config app_settings.lang, or LANG env var
• Lightweight: only 5 dependencies, 1700+ lines of code

Routing, Failover & Cost

A route can fan out to multiple upstreams. Selection is driven by three layers, in priority order:

1. Failover (availability first) — when an upstream returns 500/429/408/504, the bridge automatically switches to the next available upstream. After circuit_breaker_threshold consecutive failures an upstream is opened (paused) for circuit_breaker_cooldown ms, then recovers. Tunable in app_settings.failover.
2. Routing strategy (app_settings.route_strategy)
   • round_robin (default): spread load across upstreams. A model observed to support caching is pinned to one upstream to preserve cache locality.
   • lowest_price: among available upstreams, pick the cheapest by prices. Cached models are estimated with a blended cache price (cache_read + cache_write)/2, so the most cost-effective upstream that also reuses the cache wins.
3. Cache affinity — models that have actually used prompt caching are fixed to a single upstream (under both strategies) so the cache is not fragmented by rotation.

Per-model pricing is configured on each upstream via prices (USD per 1M tokens):

{
  "name": "openai",
  "prices": {
    "gpt-4o": { "input": 2.5, "output": 10, "cache_read": 0.625, "cache_write": 5 }
  }
}

All of the above — failover, route_strategy, per-upstream prices, round_robin — are editable directly from the web console (/dashboard → 配置 → 设置 / 各 Upstream 编辑), no manual JSON editing required.

Note: model cache capability is detected from live traffic (cache tokens in responses), not from /v1/models metadata, since most providers omit it.

---

致谢 / Acknowledgments

If you distribute or use this project, please consider mentioning ant-cave in your project or its documentation. This is not a license requirement — it would just greatly encourage me to keep contributing to the open-source community.

如果你在项目中分发或使用了本项目，请求你在项目或文档中提一下 ant-cave。这不是协议强制要求， 但这会让我备受鼓舞，继续为开源社区贡献更多。谢谢！