Prometheus metrics does not work as expected.

[sywangyi]

System Info

Runtime environment:
Target: x86_64-unknown-linux-gnu
Cargo version: 1.79.0
Commit sha: N/A
Docker label: N/A
nvidia-smi:
N/A
xpu-smi:
N/A

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

I want to use https://github.com/huggingface/text-generation-inference/blob/main/docs/source/basic_tutorials/monitoring.md to monitor TGI case. however. I got blank output.
I use curl http://127.0.0.1:80/metrics -i to get the metrics output, find the content-length is also 0.
see
HTTP/1.1 200 OK
content-type: text/plain; charset=utf-8
vary: origin, access-control-request-method, access-control-request-headers
access-control-allow-origin: *
content-length: 0
date: Thu, 04 Jul 2024 07:46:06 GMT

which could explain why I got blank output in https://github.com/huggingface/text-generation-inference/blob/main/docs/source/basic_tutorials/monitoring.md.

this is my prometheus console. not tgi metric output either.

Expected behavior

could reproduce what's described in https://github.com/huggingface/text-generation-inference/blob/main/docs/source/basic_tutorials/monitoring.md

[bryanSwk]

I faced the same issue using ghcr.io/huggingface/text-generation-inference:2.1 container.

/metrics route works fine using ghcr.io/huggingface/text-generation-inference:2.0.

[sywangyi]

@OlivierDehaene

[birshert]

Same, no metrics for me starting from 2.1.0. Everything was fine in 2.0.4

[sywangyi]

find it is caused by https://github.com/huggingface/text-generation-inference/pull/2129/files metric could be shown up after downgrading metrics-exporter-prometheus to v0.12.1, metric 0.23.0 could with metrics-exporter-prometheus 0.15.1

[Hugoch]

Fixed by #2190 . Thx @sywangyi 🤗

[eero-t]

Missed 2.1.1 release. I guess this will come in 2.1.2?

EDIT: was included to 2.2.0.