Support custom cache_control blocks with TTL configuration for Anthropic models

[jainaayush01]

Problem: Current implementation always strips existing cache_control and applies default ephemeral caching
Proposed Solution: Check for existing cache_control blocks and preserve them, with optional TTL config
Use Case: Advanced users want control over cache duration and which messages are cached
Compatibility: Maintains backward compatibility - defaults to current behavior

[jainaayush01]

@danny-avila i'm open to working on this issue,

Anthropic directly supports prompt caching with 5m and 1hr ttl configs, on production it makes more sense to have 1hr ttl for system prompts and tools and helps reduce the cost drastically.

I would also like to know the contributing guidelines for this repo or should i follow similar approach to that of LibreChat

Thank you for the work that you have been doing!

[danny-avila]

Adding context from a LibreChat/agents investigation on 2026-05-31 so this is easier to pick back up later.

Related LibreChat issue: LibreChat-AI/LibreChat#12965

Goal

Support configurable prompt-cache TTLs in @librechat/agents, then let LibreChat expose the setting as a model parameter. The practical requested behavior is:

• Keep promptCache: true as the enable switch.
• Add a TTL option, proposed shape promptCacheTTL?: '5m' | '1h'.
• Default LibreChat's Anthropic/Bedrock Claude prompt caching to 1h when prompt caching is enabled.
• Let users opt back to 5m for the existing/default-provider behavior.
• Preserve current wire behavior when TTL is omitted or explicitly 5m by not emitting a ttl field unless it is needed.

Provider payloads

Anthropic Messages API supports TTL directly on cache-control breakpoints:

cache_control: { type: 'ephemeral', ttl: '1h' }

The local agents dependency already types this in @anthropic-ai/sdk@0.92.0:

type CacheControlEphemeral = {
  type: 'ephemeral';
  ttl?: '5m' | '1h';
};

Bedrock Converse uses a different marker shape:

cachePoint: { type: 'default', ttl: '1h' }

The local agents dependency already types this in @aws-sdk/client-bedrock-runtime@3.1041.0:

type CacheTTL = '5m' | '1h';
type CachePointBlock = {
  type: 'default';
  ttl?: CacheTTL;
};

Docs checked during the investigation:

• Anthropic prompt caching: https://platform.claude.com/docs/en/build-with-claude/prompt-caching
• Bedrock prompt caching: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html

Important Bedrock caveat: AWS docs describe 1h TTL support as model-dependent. We should either guard by known-supported Claude models or live-test the current production Bedrock models before defaulting all Bedrock Claude traffic to 1h.

Current agents code paths

Anthropic-style message markers currently get created in:

• src/messages/cache.ts
  • addCacheControl(...)
  • addCacheControlToStablePrefixMessages(...)
• src/agents/AgentContext.ts
  • system message cache marker
  • summary user-message cache marker
  • stable-prefix dynamic-tail handling
• src/graphs/Graph.ts
  • applies message-level cache control before model invocation when prompt cache is enabled

Anthropic tool-definition markers currently get created in:

• src/messages/anthropicToolCache.ts
  • CACHE_CONTROL = { type: 'ephemeral' }
  • partitionAndMarkAnthropicToolCache(...)

Bedrock message cache points currently get created in:

• src/messages/cache.ts
  • addBedrockCacheControl(...)
  • system prompt path via AgentContext.buildSystemMessage(...)

Bedrock tool cache points currently get created in:

• src/llm/bedrock/toolCache.ts
  • CACHE_POINT = { cachePoint: { type: 'default' } }
  • partitionAndMarkBedrockToolCache(...)
  • insertBedrockToolCachePoint(...)

Bedrock conversion currently strips/converts cache points without preserving TTL:

• src/llm/bedrock/utils/message_inputs.ts
  • isDefaultCachePoint(...)
  • convertLangChainContentBlockToConverseContentBlock(...)
  • convertSystemMessageToConverseMessage(...)
  • message content conversion around cachePoint: { type: 'default' }

Types currently only have a boolean:

• src/types/llm.ts
  • AnthropicClientOptions has promptCache?: boolean
  • BedrockAnthropicInput has promptCache?: boolean

Suggested agents implementation

1. Add shared type and helpers, roughly:

export type PromptCacheTTL = '5m' | '1h';

function createAnthropicCacheControl(ttl?: PromptCacheTTL) {
  return ttl === '1h'
    ? { type: 'ephemeral' as const, ttl }
    : { type: 'ephemeral' as const };
}

function createBedrockCachePoint(ttl?: PromptCacheTTL) {
  return ttl === '1h'
    ? { cachePoint: { type: 'default' as const, ttl } }
    : { cachePoint: { type: 'default' as const } };
}

2. Add promptCacheTTL?: PromptCacheTTL to Anthropic and Bedrock client options.

3. Thread the option through AgentContext and Graph once per provider instead of recomputing in each marker site.

4. Change Anthropic message/tool marker creators to accept the resolved TTL and use the helper.

5. Change Bedrock message/tool marker creators to accept the resolved TTL and use the helper.

6. Update Bedrock message conversion so ttl survives when converting LangChain content blocks/system blocks to Bedrock Converse blocks.

7. Keep OpenRouter on the existing { type: 'ephemeral' } path unless/until OpenRouter support for TTL is separately confirmed. Do not accidentally flow promptCacheTTL into OpenRouter.

8. Consider Vertex separately. The LibreChat issue mentions Anthropic + Vertex, and Vertex appears to rely on Anthropic-style cache_control body markers with beta headers filtered out in LibreChat. If the agents change is generic to Anthropic-style markers, Vertex may inherit it, but it needs a targeted check.

LibreChat follow-up needed

This SDK work is not sufficient by itself. LibreChat should also expose/pass the model parameter, likely next to promptCache:

• packages/data-provider/src/types.ts / assistant model parameter types
• packages/data-provider/src/schemas.ts / Anthropic schema picks
• packages/data-provider/src/bedrock.ts / Bedrock input parser and output parser
• packages/data-provider/src/parameterSettings.ts / UI definition next to promptCache
• packages/api/src/endpoints/anthropic/llm.ts / preserve promptCacheTTL in llmConfig when supported
• packages/api/src/endpoints/bedrock/initialize.ts path via bedrockInputParser / bedrockOutputParser

Default proposal on the LibreChat side:

• Anthropic Claude: promptCacheTTL: '1h' when promptCache is true unless user selects 5m.
• Bedrock Claude: promptCacheTTL: '1h' only for known-supported models or after live validation; otherwise omit TTL to preserve 5m behavior.
• Non-Claude Bedrock / OpenRouter / non-Anthropic endpoints: omit the setting.

Why this matters

Prod Langfuse traces showed large repeated cache-creation costs in long Claude/Bedrock agent sessions. The newer cache placement work improves stable prefix reuse, but 5-minute TTL still lets expensive stable prefixes expire during longer user workflows. A 1h TTL should reduce re-creation for system prompts, static tools, and long stable prefixes when users pause or long tool sequences span more than five minutes.

This does not replace prompt placement/tool-input optimization. Some high-cost traces were driven by huge volatile assistant/tool-call history and large tool inputs, so 1h TTL helps the reuse window but will not fix every cache-creation burst by itself.

Test plan

Unit coverage in agents:

• Anthropic message cache markers include ttl: '1h' when configured.
• Anthropic message cache markers omit ttl when configured as 5m or undefined.
• Anthropic tool cache markers include/omit TTL correctly and preserve existing prototype/extras behavior.
• Bedrock message cache points include ttl: '1h' when configured.
• Bedrock message conversion preserves cachePoint.ttl for normal messages and system messages.
• Bedrock tool cache points include/omit TTL correctly.
• OpenRouter behavior remains unchanged.

Live validation:

• Anthropic live call accepts cache_control.ttl = '1h' and produces normal usage.
• Bedrock live call accepts cachePoint.ttl = '1h' for the target Claude model(s).
• Repeated calls show cache reads after initial write.
• promptCacheTTL: '5m' or omitted produces the same payload shape as today's implementation.

Definition of done:

• @librechat/agents accepts a TTL config for Anthropic and Bedrock prompt caching.
• Existing users see no behavior change unless LibreChat or direct SDK usage passes the new TTL config.
• LibreChat can pass the option through model_parameters without patching built output files.