Skip to content

✨[Feature] Fully support FakeTensors in Dynamo compile #1951

Closed
@gs-olive

Description

@gs-olive

Context

Currently, the torch.compile backend for Torch-TensorRT requires @fake_tensor_unsupported due to a few functions which cannot handle fake tensors. This support needs to be updated in order to support fake tensors on the path to full-scale dynamic shape support.

Key Consideration: Is it possible to support fake tensors while preserving the notion of "constant" tensors in TRT?

Proposed Changes

The to_numpy function will need to be updated to take a scalar:

def to_numpy(tensor: Optional[torch.Tensor]) -> Optional[np.ndarray]:

The create_constant function will need to be updated to not transform scalars into tensors:

Special considerations must be made for constant tensors with more than one element.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions