cross compile for windows

[lanluo-nvidia]

Description

The cross compile for windows change has added the following new interface:

Usage 1:
cross compile and save the exported program to the disk in Linux:
torch_tensorrt.cross_compile_for_windows(toch.nn.Module, file_path, inputs, …..)
cross load the exported program from disk in Windows:
torch_tensorrt.load_cross_compiled_exported_program(file_path)

Usage 2:
export the model to exported program
ep = torch.export.export(…)
cross compile the exported program in Linux and return the trt_module:
trt_module = torch_tensorrt.dynamo.cross_compile_for_windows(ExportedProgram, inputs,…..)
save the cross compiled trt module to the disk in Linux:
torch_tensorrt.dynamo.save_cross_compiled_exported_program(trt_module, inputs,…..)
cross load the exported program from disk in Windows:
torch_tensorrt.dynamo.load_cross_compiled_exported_program(file_path)

Type of change

Please delete options that are not relevant and/or add your own.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified

[pytorch-bot]

No ciflow labels are configured for this repo.
For information on how to enable CIFlow bot see this wiki

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-10-30 18:19:43.261631+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-10-30 18:20:03.016868+00:00
@@ -693,13 +693,13 @@
                        module, tuple(arg_inputs), kwargs=kwarg_inputs, strict=False
                    )
                    torch.export.save(exp_program, file_path)


-
def save_cross_compiled_exported_program(
    gm: torch.fx.GraphModule,
    file_path: str,
) -> None:
    from torch_tensorrt.dynamo._exporter import export
+
    exp_program = export(gm, cross_compile_flag=True)
    torch.export.save(exp_program, file_path)
--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/runtime/_utils.py	2024-10-30 18:19:43.281631+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/runtime/_utils.py	2024-10-30 18:20:06.042509+00:00
@@ -146,6 +146,6 @@
    serialized_target_platform: str,
) -> List[torch.Tensor]:

    raise RuntimeError(
        "The saved model is cross compiled for windows in Linux, should only be loadded in Windows via torch_tensorrt.load_cross_compiled_exported_program() api."
-    )
\ No newline at end of file
+    )

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/runtime/_utils.py	2024-10-31 16:35:01.922532+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/runtime/_utils.py	2024-10-31 16:35:26.529213+00:00
@@ -146,6 +146,6 @@
    serialized_target_platform: str,
) -> List[torch.Tensor]:

    raise RuntimeError(
        "The saved model is cross compiled for windows in Linux, should only be loadded in Windows via torch_tensorrt.load_cross_compiled_exported_program() api."
-    )
\ No newline at end of file
+    )

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/runtime/_utils.py	2024-10-31 23:51:37.956329+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/runtime/_utils.py	2024-10-31 23:52:02.447686+00:00
@@ -146,6 +146,6 @@
    serialized_target_platform: str,
) -> List[torch.Tensor]:

    raise RuntimeError(
        "The saved model is cross compiled for windows in Linux, should only be loadded in Windows via torch_tensorrt.load_cross_compiled_exported_program() api."
-    )
\ No newline at end of file
+    )

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/runtime/_utils.py	2024-11-01 17:19:50.118262+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/runtime/_utils.py	2024-11-01 17:20:13.032393+00:00
@@ -146,6 +146,6 @@
    serialized_target_platform: str,
) -> List[torch.Tensor]:

    raise RuntimeError(
        "The saved model is cross compiled for windows in Linux, should only be loadded in Windows via torch_tensorrt.load_cross_compiled_exported_program() api."
-    )
\ No newline at end of file
+    )

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/_compiler.py	2024-11-01 18:10:52.286574+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/_compiler.py	2024-11-01 18:11:12.489684+00:00
@@ -475,11 +475,14 @@
        if make_refittable:
            raise ValueError("Use flag make_refittable only. Flag refit is deprecated.")
        else:
            make_refittable = kwargs["refit"]

-    if "enable_cross_compile_for_windows" in kwargs.keys() and kwargs["enable_cross_compile_for_windows"]:
+    if (
+        "enable_cross_compile_for_windows" in kwargs.keys()
+        and kwargs["enable_cross_compile_for_windows"]
+    ):
        raise ValueError(
            "Please use cross_compile_for_windows() api if you want to cross compile the module in Linux for inferencing in Windows."
        )

    engine_capability = EngineCapability._from(engine_capability)