feat: Python Runtime Windows Builds on TRT 10 #2764

gs-olive · 2024-04-19T00:19:37Z

Description

TRT-10 based implementation of Windows builds

Type of change

New feature (non-breaking change which adds functionality)

Checklist:

[ x ] My code follows the style guidelines of this project (You can use the linters)
[ x ] I have performed a self-review of my own code
[ x ] I have commented my code, particularly in hard-to-understand areas and hacks
[ x ] I have made corresponding changes to the documentation
[ x ] I have added tests to verify my fix or my feature
[ x ] New and existing unit tests pass locally with my changes
[ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

HolyWu · 2024-04-21T12:28:00Z

https://github.com/HolyWu/TensorRT/tree/windows_CI_trt_10_fix
Did some cleanup and fixes. Also changed to install torch and trt according to desired_cuda from build matrix, otherwise torch and trt with cuda 12 would be installed in cu118 matrix.

gs-olive · 2024-04-22T22:22:36Z

Thanks very much @HolyWu - I've cherry-picked in the non-C++ changes (the C++ runtime will be ported in a different PR), plus some test modifications, and will see how the CI tests go.

Add build-test-windows job 9.3.0 Upgrade and version fixing Add bazelisk installation Update installation of Torch-TRT Bazel install fix Python only for initial Windows build Conda install latest cuda toolkit CUDA Driver Update CUDA Driver Update 2 Get windows version info Attempt driver upgrade within system Switch installer script to curl Switch installer script to network Try testing E2E models on Windows Try linux testing action for Windows Update Windows build/test Use local version of yml Use local version of yml 2 Use local version of yml 3 Use local version of yml 4 Remove CONDA_RUN references Update Python source Add torch-trt install script packing Remove extraneous python references Replace pipefail command Replace pipefail command 2 Replace source command Update cat command for Powershell Avoid script overwrite in runner Avoid script overwrite in runner 2 Update path concat to use echo fix: Use forked infra repository Skip things not supported by GitHub-hosted Windows runner Upgrade setuptools and wheel Rework windows test Restore root WORKSPACE to original Attribute naming fix in YML Enable one test in Windows Enable GPU Access Enable GPU Access Attempt 2 Enable GPU Access Attempt 3 - revert Switch off miniconda setup Specify Python 3.9 as default for testing Specify Python 3.9 as default for testing 2 Revert py39 changes, make changes to infra fork, re-enable cu121 installation Failing cuda installation Failing cuda installation 2 Failing cuda installation 3 Differentiate pre-build script during build and test Install trt bindings + lib manually Switch back to pytorch as repo source Specify validation runner Revert "Specify validation runner" This reverts commit 6d664c0. Enable more tests Enable more tests 2 Remove Python runtime specification Clean up and enable additional Windows tests Use .bat for driver upgrade Use .bat for driver upgrade 2 UTF8 encoding for windows testing

Remove serde and extraneous tests

HolyWu · 2024-04-24T14:41:51Z

.github/scripts/install-torch-tensorrt-windows.sh

+source "${BUILD_ENV_FILE}"
+
+# Install test index version of Torch and Torchvision
+${CONDA_RUN} pip install torch torchvision --index-url https://download.pytorch.org/whl/test/${CU_VERSION}


Actually you can restore this line back to ${CONDA_RUN} ${PIP_INSTALL_TORCH} torchvision like before. The reason you had to manually specify test index is that, when you were using trt_10 as base branch, the set-channel action set CHANNEL to nightly and hence PIP_INSTALL_TORCH was set to install nightly build. Now you have changed the base branch to release/2.3, the set-channel action set CHANNEL to test and hence PIP_INSTALL_TORCH was set to install test build.

HolyWu · 2024-04-24T14:48:12Z

packaging/pre_build_script_windows.sh

+python -m pip install -U numpy packaging pyyaml setuptools wheel
+
+# Install Torch from Test Index
+python -m pip install torch==2.3.0 --index-url https://download.pytorch.org/whl/test/${CU_VERSION}


This line can be complete removed for the same reason as above, since the Install torch dependency step already installed the desired version.

narendasan

LGTM

Co-authored-by: HolyWu <holywu@gmail.com>

gs-olive added the WIP Work is in progress, pull request should not be merged yet label Apr 19, 2024

gs-olive self-assigned this Apr 19, 2024

facebook-github-bot added the cla signed label Apr 19, 2024

github-actions bot requested a review from zewenli98 April 19, 2024 00:19

gs-olive removed the request for review from zewenli98 April 19, 2024 01:34

gs-olive force-pushed the windows_CI_trt_10 branch from 120413b to 8db8a0a Compare April 20, 2024 00:08

gs-olive force-pushed the windows_CI_trt_10 branch from 96f129e to 6204ef8 Compare April 22, 2024 22:20

github-actions bot added the component: tests Issues re: Tests label Apr 22, 2024

gs-olive force-pushed the windows_CI_trt_10 branch from 6204ef8 to bc03fbf Compare April 23, 2024 01:15

gs-olive changed the base branch from trt_10 to release/2.3 April 23, 2024 01:16

gs-olive force-pushed the windows_CI_trt_10 branch from bc03fbf to 5fac363 Compare April 23, 2024 01:17

HolyWu and others added 9 commits April 23, 2024 16:35

Switch to TRT 10 in installer

0588052

Installer flags

b66b6f9

Add access to test indices Torch

ace1531

Add access to test indices Torch 2

f642c8a

Add access to test indices Torch 3

43a90ee

Add access to test indices Torch 4

e8edf3a

Add access to test indices Torch 5

20fb155

Add access to test indices Torch 6

2482307

gs-olive and others added 10 commits April 23, 2024 16:35

Missing tensorrt bindings

dc9f0de

Delete usused pre_build_script_driver_upgrade.sh

422423c

Clean up pre_build_script_windows.sh

53638c2

cuDNN is not required by TRT 10

08afe82

Install tensorrt-bindings and tensorrt-libs

bffcfb2

Temporarily install torch in test index

d70e46d

Install torch and tensorrt according to desired_cuda in matrix

c028b67

Disable tests for Python-only runtime + Win unsupported ops

d14833f

Remove serde and extraneous tests

Fix tracing of generator in BERT model on Windows

a39801f

Clean up code changes

f8c300f

gs-olive force-pushed the windows_CI_trt_10 branch from b25d7c2 to f8c300f Compare April 23, 2024 23:43

gs-olive removed the WIP Work is in progress, pull request should not be merged yet label Apr 23, 2024

gs-olive requested a review from narendasan April 23, 2024 23:44

HolyWu reviewed Apr 24, 2024

View reviewed changes

narendasan approved these changes Apr 24, 2024

View reviewed changes

gs-olive added 2 commits April 24, 2024 10:40

Address review comments

0877ca1

Update job prompts for Windows

8ba63ea

gs-olive merged commit 8b9836f into pytorch:release/2.3 Apr 25, 2024
36 checks passed

gs-olive deleted the windows_CI_trt_10 branch April 25, 2024 17:34

gs-olive added a commit that referenced this pull request Apr 25, 2024

feat: Python Runtime Windows Builds on TRT 10 (#2764)

c01fc84

Co-authored-by: HolyWu <holywu@gmail.com>

gs-olive mentioned this pull request Apr 25, 2024

cherry-pick: Python Runtime Windows Builds on TRT 10 (#2764) #2776

Merged

gs-olive added a commit that referenced this pull request Apr 25, 2024

feat: Python Runtime Windows Builds on TRT 10 (#2764)

bd90a99

Co-authored-by: HolyWu <holywu@gmail.com>

gs-olive added a commit that referenced this pull request Apr 25, 2024

feat: Python Runtime Windows Builds on TRT 10 (#2764)

6353d02

Co-authored-by: HolyWu <holywu@gmail.com>

zewenli98 pushed a commit that referenced this pull request Apr 26, 2024

feat: Python Runtime Windows Builds on TRT 10 (#2764)

4734ca4

Co-authored-by: HolyWu <holywu@gmail.com>

gs-olive added a commit that referenced this pull request Apr 26, 2024

feat: Python Runtime Windows Builds on TRT 10 (#2764)

4f2c092

Co-authored-by: HolyWu <holywu@gmail.com>

gs-olive added a commit that referenced this pull request Apr 29, 2024

feat: Python Runtime Windows Builds on TRT 10 (#2764)

a53315f

Co-authored-by: HolyWu <holywu@gmail.com>

gs-olive added a commit that referenced this pull request Apr 30, 2024

feat: Python Runtime Windows Builds on TRT 10 (#2764)

fc97e01

Co-authored-by: HolyWu <holywu@gmail.com>

narendasan pushed a commit that referenced this pull request May 1, 2024

cherry-pick: Python Runtime Windows Builds on TRT 10 (#2764) (#2776)

ff8c872

Co-authored-by: HolyWu <holywu@gmail.com>

laikhtewari pushed a commit that referenced this pull request May 24, 2024

cherry-pick: Python Runtime Windows Builds on TRT 10 (#2764) (#2776)

dbe19dc

Co-authored-by: HolyWu <holywu@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Python Runtime Windows Builds on TRT 10 #2764

feat: Python Runtime Windows Builds on TRT 10 #2764

gs-olive commented Apr 19, 2024 •

edited

Loading

HolyWu commented Apr 21, 2024

gs-olive commented Apr 22, 2024 •

edited

Loading

HolyWu Apr 24, 2024

HolyWu Apr 24, 2024

narendasan left a comment

feat: Python Runtime Windows Builds on TRT 10 #2764

feat: Python Runtime Windows Builds on TRT 10 #2764

Conversation

gs-olive commented Apr 19, 2024 • edited Loading

Description

Type of change

Checklist:

HolyWu commented Apr 21, 2024

gs-olive commented Apr 22, 2024 • edited Loading

HolyWu Apr 24, 2024

Choose a reason for hiding this comment

HolyWu Apr 24, 2024

Choose a reason for hiding this comment

narendasan left a comment

Choose a reason for hiding this comment

gs-olive commented Apr 19, 2024 •

edited

Loading

gs-olive commented Apr 22, 2024 •

edited

Loading