❓ [Question] How to save the compiled while using torch.compile

[klin2024]

For the example below, how do I save the compiled model?

backend = "torch_tensorrt"
tp_model = torch.compile(
tp_model,
backend=backend,
options={
"truncate_long_and_double": True,
"enabled_precisions": {torch.float32, torch.float16},
"use_python_runtime": True,
"min_block_size": 1,
},
dynamic=False,
)

[narendasan]

torch.compile is not serializable. There are two options. You can enable the engine cache which create a persistent cache you can use between sessions or you could use the MutableTorchTensorRTModule which has a similar UX but is serializable