Refactor Prometheus and Add Request Level Metrics#2316

Merged

simon-mo merged 52 commits intovllm-project:mainfrom

robertgshaw2-redhat:rs/feature/metrics

Jan 31, 2024

Commits on Dec 30, 2023

added first refactor of metrics
rsnm2
committed
refactored to use counters rather than gauges for monotonically increasing values. computing averages on the client is an anti-pattern for prometheus metrics and should be computed on the prom server
rsnm2
committed
added dev notebook, started running live
rsnm2
committed

Commits on Dec 31, 2023

Commits on Jan 1, 2024

Commits on Jan 4, 2024

Commits on Jan 5, 2024

Commits on Jan 6, 2024

stash to move to other machine
rsnm2
committed
factored per simon-mo request
rsnm2
committed
readded files
rsnm2
committed
fixed bugs in initial version
rsnm2
committed
e2e functional testing complete
rsnm2
committed
readded images
rsnm2
committed
quality
rsnm2
committed

Commits on Jan 7, 2024