Skip to content

Conversation

@krrishdholakia
Copy link
Contributor

Relevant issues

Addresses user request for enhanced Prometheus metrics.

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🆕 New Feature

Changes

WHY: Users require the model_id label in key Prometheus metrics to accurately calculate success rates and analyze performance at the specific model deployment level, enhancing observability beyond just the requested_model.

WHAT:

  • Added the model_id label to the following Prometheus metrics:
    • litellm_proxy_total_requests_metric
    • litellm_proxy_failed_requests_metric
    • litellm_request_total_latency_metric
    • litellm_llm_api_time_to_first_token_metric (TTFT)
  • Refactored the TTFT metric logging in litellm/integrations/prometheus.py to utilize prometheus_label_factory for consistent label handling.
  • Updated Prometheus documentation (docs/my-website/docs/proxy/prometheus.md) to reflect the new labels.
  • Added and updated unit tests to verify the correct inclusion of model_id in the affected metrics.

Slack Thread

Open in Cursor Open in Web

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
@cursor
Copy link

cursor bot commented Dec 16, 2025

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
Learn more about Cursor Agents

@vercel
Copy link

vercel bot commented Dec 16, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
litellm Ready Ready Preview, Comment Dec 16, 2025 10:33am

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants