Add prompt caching for AmazonBedrockGenerator and AmazonBedrockChatGenerator

[deep-rloebbert]

Is your feature request related to a problem? Please describe.
Yes. When using AmazonBedrockGenerator or AmazonBedrockChatGenerator with large, static contexts (such as long system prompts or few-shot examples), the model re-processes the input tokens for every request. This results in unnecessarily high costs and increased latency.

Amazon Bedrock supports Prompt Caching to reuse processed tokens, but currently, there is no mechanism in the Haystack components to pass the required caching parameters to the underlying Bedrock API.

Describe the solution you'd like
I would like the ability to specify prompt caching parameters within the Haystack Bedrock components. Best case a simple bool toggle.

Ideally, this would be handled via metadata in ChatMessage (for Chat Generators) or call parameters. When the component constructs the API payload, it should detect these flags and inject the specific cache control structure required by Bedrock into the request body (e.g., adding cache_control or cachePoint fields to the message blocks).

Describe alternatives you've considered
Spending too much money.

Additional context

• AWS Documentation: [Amazon Bedrock Prompt Caching](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html#prompt-caching-simplified)
• Benefit: caching can significantly reduce inference costs (up to 90% for cached tokens) and latency for supported models.

[sjrl]

@deep-rloebbert we'd probably only do this for the ChatGenerator since the Generator doesn't support multiple messages (ie it only sends one message) so there wouldn't be anything to cache using their caching protocol.

Also the Generator doesn't support the converse API which looks to be required to use this feature.

[anakin87]

@deep-rloebbert I started to investigate how to support this feature.

What I found is that if you are using Claude models, you might already be using simplified prompt caching. Unfortunately, it's not transparent.

I'll work on implementing explicit prompt caching for supported models.

[deep-rloebbert]

@anakin87 Awesome! Looking forward to it.

Simplified Prompt Caching:

Instead of requiring you to specify exact cache checkpoint locations, you can use automatic cache management with a single breakpoint at the end of your static content.

As far as I know I cannot / did not append a breakpoint to any chat message.
My understanding of this documentation is that we have to specify caching in the API call, and that there is (unfortunately) no option for any config.

[anakin87]

Ok docs are not very clear. In any case, I'll work on supporting cache for ChatMessage. Might not feasible or hard to do for Tools.

[deep-rloebbert]

@anakin87 maybe it can be a feature flag/config on the ChatGenerator to enable simplified caching, so that the System Prompt and tools are cached?

Then we could skip explicit cache points for ChatMessage?

[anakin87]

@anakin87 maybe it can be a feature flag/config on the ChatGenerator to enable simplified caching, so that the System Prompt and tools are cached?

Then we could skip explicit cache points for ChatMessage?

Thanks for providing this idea. I'll think about it.