[Task]: unify prompt caching behavior - Anthropic vs Anthropic through OpenRouter

[kantord]

Description

blocked by #892

This feature might have limited usefulness, as it should only impact situations where the "integration" (AI tool) does not support prompt caching through OpenRouter. So far we only identified "Continue" as having this problem. (See more here: https://docs.google.com/document/d/1gEgkIyT1CRcTMAwYiU6ldeeosKKbWMX5HM8AdQx3xV0/edit?tab=t.0#heading=h.9nmo8mvyk6k9)

Nevertheless, CodeGate could technically "normalize"/override the requests in some way that prompt caching is properly supported regardless of how Continue (or any CodeGate integration) behaves out of the box

Additional Context

internal document about prompt caching: https://docs.google.com/document/d/1gEgkIyT1CRcTMAwYiU6ldeeosKKbWMX5HM8AdQx3xV0/edit?tab=t.0#heading=h.9nmo8mvyk6k9

[lukehinds]

should these be grouped under #940 @kantord ? you should be able to add them with add-parent, if not let me know , it might be a permissions thing

[kantord]

should these be grouped under #940 @kantord ? you should be able to add them with add-parent, if not let me know , it might be a permissions thing

I am not sure. #940 seems independent from this issue, as that feature could be relevant even if this issue is not fixed and vice versa.

I was thinking of creating an epic or user story to group all 4 tasks into one, but I chose against this because each have very different ROI and are likely to be prioritized separately.

If you have a specific preference or idea though, I am happy to make changes

[kantord]

@lukehinds

Actually, I think you are right. Grouping it there makes sense because the user story gives a good framing to what "prompt caching" and CodeGate have to do with each other and how this could be visible to the average user.

What I said about different ROI and priority is true, but once we prioritize the user story we can still reject/delay specific subtasks separately. So I will make the change