[Feature]: Proxy - return OpenAI compatible remaining_requests and remaining_tokens headers

[ishaan-jaff]

The Feature

he was using the headers to track usage and remaining quota. He was very surprised to find out his metric was broken on the migration, as LiteLLM was supposed to be a drop-in replacement.

Action items for this ticket

• Return x-ratelimit-* headers in responses https://platform.openai.com/docs/guides/rate-limits/usage-tiers
• If virtual key / team / user has no rate limit set then return rate limit from litellm model group
• If just one deployment in a model group -> return the headers in OpenAI compatible format
• If multiple deployments in a model group then return the remaining tokens / requests for the model group

Motivation, pitch

Twitter / LinkedIn details

No response

[krrishdholakia]

request comes in -> check remaining key rate limit -> return that

expose a get_remaining_rate_limit(user_api_key_dict: UserAPIKeyAuth, llm_router: Router) -> returns the remaining rate limit based on the checks from here

Test case 1: returns remaining rate limit for key with no limit, openai endpoint

Test case 2: returns remaining rate limit for key with no limit, openai + azure endpoint

Test case 3: returns remaining rate limit for key with limit, openai + azure endpoint

[krrishdholakia]

• Returns remaining rate limit for a given key - 0c859a5
• Return remaining rate limit for model via - .completion() - 184678b
• Return remaining rate limit for model group via - router.completion()

[krrishdholakia]

Need to cover openai + fireworks ai embeddings too

[krrishdholakia]

I think we should guarantee this header even if the provider doesn't support it, so we don't break people's client-side code.

Perhaps we can set it to None, to prevent any client-side errors? cc: @oz-elhassid

[krrishdholakia]

We should add this to docs, so the response is clear