Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom metric API example #19

Merged
merged 16 commits into from
Apr 15, 2022
Merged

Custom metric API example #19

merged 16 commits into from
Apr 15, 2022

Conversation

rmccorm4
Copy link
Contributor

@rmccorm4 rmccorm4 commented Apr 13, 2022

Adds an example of adding a custom metric family global to the backend with BackendState, and per-model metrics added to the global metric family by each ModelState via labels:

root@RMCCORMICK:/mnt/jira/3557-metrics-example/server/qa/L0_backend_identity# grep byte_size client.log
# HELP input_byte_size_counter Cumulative input_byte_size across all identity model requests
# TYPE input_byte_size_counter counter
input_byte_size_counter{model="identity_bytes",version="1"} 0.000000
input_byte_size_counter{model="identity_fp32",version="1"} 0.000000
input_byte_size_counter{model="identity_nobatch_int8",version="1"} 0.000000
input_byte_size_counter{model="identity_uint32",version="1"} 64.000000

Updated unit test and documentation here: triton-inference-server/server#4211

src/identity.cc Outdated Show resolved Hide resolved
src/identity.cc Outdated Show resolved Hide resolved
src/identity.cc Outdated Show resolved Hide resolved
src/identity.cc Outdated Show resolved Hide resolved
src/identity.cc Outdated Show resolved Hide resolved
src/identity.cc Outdated Show resolved Hide resolved
@rmccorm4 rmccorm4 merged commit f881381 into main Apr 15, 2022
@rmccorm4 rmccorm4 deleted the rmccormick-metrics-example branch April 15, 2022 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants