Skip to content

Commit

Permalink
Add extension policy documentation limitation (triton-inference-serve…
Browse files Browse the repository at this point in the history
  • Loading branch information
jbkyang-nvi authored Dec 15, 2022
1 parent 95e386b commit dfc6f19
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions docs/protocol/extension_schedule_policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,12 @@ schedule-policy extension allows an inference request to provide
parameters that influence how Triton handles and schedules the
request. Because this extension is supported, Triton reports
“schedule_policy” in the extensions field of its Server Metadata.
Note the policies are specific to [dynamic
batcher](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md#dynamic-batcher)
and not [sequence
batcher](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md#sequence-batcher)
with the [direct](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/architecture.md#direct)
scheduling strategy.

The schedule-policy extension uses request parameters to indicate the
policy. The parameters and their type are:
Expand Down

0 comments on commit dfc6f19

Please sign in to comment.