Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Document HTTP compression feature #3368

Merged
merged 2 commits into from
Sep 15, 2021
Merged

Document HTTP compression feature #3368

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c4f5d05
Document HTTP compression feature
tanmayv25 Sep 14, 2021
cbfafd4
Improve language
tanmayv25 Sep 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Document HTTP compression feature
  • Loading branch information
@tanmayv25
tanmayv25 committed Sep 14, 2021
commit c4f5d05e0592ed992775046cb26ecf2f8772f6d3
9 changes: 8 additions & 1 deletion docs/inference_protocols.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,13 @@ model health, metadata and statistics. Additional endpoints allow
model loading and unloading, and inferencing. See the KFServing and
extension documentation for details.

### HTTP Options
Triton provides following configuring options for server-client network transactions over HTTP protocol.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the following configuration options


#### Compression

Triton allows the on-wire compression of request/response on HTTP through its clients. See [HTTP Compression](https://github.com/triton-inference-server/client/tree/main#compression) for more details.

### GRPC Options
Triton exposes various GRPC parameters for configuring the server-client network transactions. For usage of these options, refer to the output from `tritonserver --help`.

Expand All @@ -70,7 +77,7 @@ Triton allows the on-wire compression of request/response messages by exposing f

* `--grpc-infer-response-compression-level`

For client-side documentation, see [Client-Side GRPC Compression](https://github.com/triton-inference-server/client/tree/main#compression)
For client-side documentation, see [Client-Side GRPC Compression](https://github.com/triton-inference-server/client/tree/main#compression-1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you adding the -1 in the end?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compression without -1 links to http compression and compression-1 links to the second compression sub-heading for grpc.
https://github.com/triton-inference-server/client/tree/main#compression
https://github.com/triton-inference-server/client/tree/main#compression-1

I too was not aware of this way of indexing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL


Compression can be used to reduce the amount of bandwidth used in server-client communication. For more details, see [gRPC Compression](https://grpc.github.io/grpc/core/md_doc_compression.html).

Expand Down