Open
Description
openedon Jul 11, 2024
Describe the feature request
Can onnxruntime support directly loading *.engine or *.trt to initialize the session when using TensorRT EP?
Describe scenario use case
Currently using TensorRT EP, in order to ensure the initialization speed, the *.engine is written to the hard disk when using the cache. In my production environment, I want *.engine to be encrypted. And they can be directly loaded in the following way.
//Decrypt model to byte
byte=Decryptor.read("./encrypted_model.engine");
//Initialize to session
session = Ort::Session(env, byte.data(), byte.size(), session_options);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment