Closed
Description
Context
Currently, the torch.compile
backend for Torch-TensorRT requires @fake_tensor_unsupported
due to a few functions which cannot handle fake tensors. This support needs to be updated in order to support fake tensors on the path to full-scale dynamic shape support.
Key Consideration: Is it possible to support fake tensors while preserving the notion of "constant" tensors in TRT?
Proposed Changes
The to_numpy
function will need to be updated to take a scalar:
The create_constant
function will need to be updated to not transform scalars into tensors:
Special considerations must be made for constant tensors with more than one element.