-
-
Notifications
You must be signed in to change notification settings - Fork 5.1k
Labels
bugSomething isn't workingSomething isn't working
Description
What happened?
If the Gemini API counts part of the input as IMAGE, then these tokens are not considered for the cost calculation. prompt_tokens as a total is correct, but this is not being used for cost calculation.
Example:
'usageMetadata': {'promptTokenCount': 262, 'candidatesTokenCount': 117, 'totalTokenCount': 1080, 'promptTokensDetails': [{'modality': 'TEXT', 'tokenCount': 4}, {'modality': 'IMAGE', 'tokenCount': 258}], 'thoughtsTokenCount': 701}, 'modelVersion': 'gemini-2.5-flash', 'responseId': 'M-C6aMH-Mq-nnsEPyrC6wAs'}
In the example from the screenshot you can see that the cost is calculated as:
0.002046 = 4 * 0.3 * 1e-6 + 818 * 2.5 * 1e-6
instead of
0.002124 = 262 * 0.3 * 1e-6 + 818 * 2.5 * 1e-6
Relevant log output
Are you a ML Ops Team?
No
What LiteLLM version are you on ?
v1.78.5
Twitter / LinkedIn details
No response
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working