Triton-rust : A gRPC library for Nvidia Triton Inference Server

Triton-rust is a gRPC library to interact with Nvidia Triton Inference Server.

Building

You can compile the library together with all examples with:

make all

Examples

You can find several examples of neural network inference using Triton Inference Server and Rust. These examples could be found here.

Known bugs

• CUDA shared memory is not functionnal yet
• Rust's ndarrays are to be in standard layout

Contact

Boris Albar (b.albar@catie.fr)

Ackowledgements

This work has been done in frame of the Vaniila platform.