Skip to content

fix: release/2.1 CI Repair #2528

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Dec 20, 2023
Merged

fix: release/2.1 CI Repair #2528

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
d872c20
test: Regular CI
gs-olive Dec 8, 2023
eaa94fb
WORKSPACE
gs-olive Dec 12, 2023
2c0248d
revert workspace changes
gs-olive Dec 14, 2023
6544dd2
CUDA specification
gs-olive Dec 14, 2023
00bfac7
CUDA specification 2
gs-olive Dec 14, 2023
00eac02
CUDA specification 2
gs-olive Dec 14, 2023
009164f
CUDA specification 3
gs-olive Dec 14, 2023
aeacbc5
add debug
gs-olive Dec 14, 2023
acf8c00
torch version fix
gs-olive Dec 15, 2023
f57e277
revert pre-build changes
gs-olive Dec 15, 2023
f0946ab
version constraint fixes
gs-olive Dec 15, 2023
d533852
version constraint fixes 2
gs-olive Dec 15, 2023
7fe3e09
remove junit
gs-olive Dec 15, 2023
25e8810
fix: Torch switch to test
gs-olive Dec 15, 2023
38b6449
fix: isolate bug
gs-olive Dec 16, 2023
4ab933a
fix: narrowing error 2
gs-olive Dec 16, 2023
f2d883a
fix: narrowing error 3
gs-olive Dec 16, 2023
f7481e0
fix: narrowing error 4
gs-olive Dec 16, 2023
1e37315
test fix segfault
gs-olive Dec 18, 2023
ae9ec86
test imports
gs-olive Dec 18, 2023
dee5650
switch to release
gs-olive Dec 18, 2023
fa74dfb
release to main
gs-olive Dec 19, 2023
8842a63
back to test
gs-olive Dec 19, 2023
ab5daed
testing install removal
gs-olive Dec 19, 2023
f400fac
CHANNELS
gs-olive Dec 19, 2023
5b7d785
no channel
gs-olive Dec 19, 2023
64134cd
Return all tests
gs-olive Dec 19, 2023
e2e4999
bump requirements versions
gs-olive Dec 19, 2023
743a5c0
unnecessary modification
gs-olive Dec 19, 2023
64574a7
remove channel from linux test
gs-olive Dec 19, 2023
ca18067
Fix expecttest version
gs-olive Dec 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/scripts/install-torch-tensorrt.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
set -eou pipefail
# Source conda so it's available to the script environment
source ${BUILD_ENV_FILE}
${CONDA_RUN} ${PIP_INSTALL_TORCH} torchvision pyyaml
${CONDA_RUN} ${PIP_INSTALL_TORCH} torch==2.1.2 torchvision==0.16.2 pyyaml
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this do anything? since theres already ${PIP_INSTALL_TORCH}?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is necessary, since in build-test.yml, when building Torch-TRT, we have:

PIP_INSTALL_TORCH='pip install torch==2.1.2 --extra-index-url https://download.pytorch.org/whl/test/cu118'

In linux-test.yml, when testing Torch-TRT using the artifact from build, we have:

PIP_INSTALL_TORCH='pip install torch --extra-index-url https://download.pytorch.org/whl/test/cu118'

The second one installs the 2.2.0.dev test build, which is the most up-to-date test build, but is incompatible with the version on which it was built. Specifying the version explicitly resolves this problem.

export TRT_VERSION=$(${CONDA_RUN} python -c "import versions; versions.tensorrt_version()")
${CONDA_RUN} python -m pip install /opt/torch-tensorrt-builds/torch_tensorrt*+${CU_VERSION}*.whl tensorrt~=${TRT_VERSION} tensorrt-bindings~=${TRT_VERSION} --extra-index-url=https://pypi.ngc.nvidia.com

echo -e "Running test script";
echo -e "Running test script";
39 changes: 20 additions & 19 deletions .github/workflows/build-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,13 @@ on:

jobs:
generate-matrix:
uses: pytorch/test-infra/.github/workflows/generate_binary_build_matrix.yml@main
uses: pytorch/test-infra/.github/workflows/generate_binary_build_matrix.yml@release/2.1
with:
package-type: wheel
os: linux
test-infra-repository: pytorch/test-infra
test-infra-ref: main
test-infra-ref: release/2.1
channel: test
with-rocm: false
with-cpu: false

Expand All @@ -37,12 +38,12 @@ jobs:
smoke-test-script: ""
package-name: torch_tensorrt
name: Build torch-tensorrt whl package
uses: pytorch/test-infra/.github/workflows/build_wheels_linux.yml@main
uses: pytorch/test-infra/.github/workflows/build_wheels_linux.yml@release/2.1
with:
repository: ${{ matrix.repository }}
ref: ""
test-infra-repository: pytorch/test-infra
test-infra-ref: main
test-infra-ref: release/2.1
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
pre-script: ${{ matrix.pre-script }}
env-var-script: ${{ matrix.env-var-script }}
Expand All @@ -64,13 +65,13 @@ jobs:
- repository: pytorch/tensorrt
package-name: torch_tensorrt
pre-script: packaging/pre_build_script.sh
uses: pytorch/tensorrt/.github/workflows/linux-test.yml@main
uses: ./.github/workflows/linux-test.yml
with:
job-name: tests-py-torchscript-fe
repository: "pytorch/tensorrt"
ref: ""
test-infra-repository: pytorch/test-infra
test-infra-ref: main
test-infra-ref: release/2.1
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
pre-script: ${{ matrix.pre-script }}
script: |
Expand All @@ -83,7 +84,7 @@ jobs:
popd
pushd .
cd tests/py/ts
${CONDA_RUN} python -m pip install --pre pytest timm transformers parameterized expecttest --use-deprecated=legacy-resolver
${CONDA_RUN} python -m pip install --pre pytest timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/ts_api_test_results.xml api/
${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/ts_models_test_results.xml models/
${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/ts_integrations_test_results.xml integrations/
Expand All @@ -99,20 +100,20 @@ jobs:
- repository: pytorch/tensorrt
package-name: torch_tensorrt
pre-script: packaging/pre_build_script.sh
uses: pytorch/tensorrt/.github/workflows/linux-test.yml@main
uses: ./.github/workflows/linux-test.yml
with:
job-name: tests-py-dynamo-converters
repository: "pytorch/tensorrt"
ref: ""
test-infra-repository: pytorch/test-infra
test-infra-ref: main
test-infra-ref: release/2.1
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
pre-script: ${{ matrix.pre-script }}
script: |
export USE_HOST_DEPS=1
pushd .
cd tests/py/dynamo
${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest --use-deprecated=legacy-resolver
${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/dynamo_converters_test_results.xml -n 10 conversion/
popd

Expand All @@ -126,20 +127,20 @@ jobs:
- repository: pytorch/tensorrt
package-name: torch_tensorrt
pre-script: packaging/pre_build_script.sh
uses: pytorch/tensorrt/.github/workflows/linux-test.yml@main
uses: ./.github/workflows/linux-test.yml
with:
job-name: tests-py-dynamo-fe
repository: "pytorch/tensorrt"
ref: ""
test-infra-repository: pytorch/test-infra
test-infra-ref: main
test-infra-ref: release/2.1
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
pre-script: ${{ matrix.pre-script }}
script: |
export USE_HOST_DEPS=1
pushd .
cd tests/py/dynamo
${CONDA_RUN} python -m pip install --pre pytest timm transformers parameterized expecttest --use-deprecated=legacy-resolver
${CONDA_RUN} python -m pip install --pre pytest timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/dynamo_fe_test_results.xml --ir dynamo models/test_models_export.py
${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/export_serde_test_results.xml --ir dynamo models/test_export_serde.py
${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/dyn_models_export.xml --ir dynamo models/test_dyn_models.py
Expand All @@ -155,20 +156,20 @@ jobs:
- repository: pytorch/tensorrt
package-name: torch_tensorrt
pre-script: packaging/pre_build_script.sh
uses: pytorch/tensorrt/.github/workflows/linux-test.yml@main
uses: ./.github/workflows/linux-test.yml
with:
job-name: tests-py-torch-compile-be
repository: "pytorch/tensorrt"
ref: ""
test-infra-repository: pytorch/test-infra
test-infra-ref: main
test-infra-ref: release/2.1
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
pre-script: ${{ matrix.pre-script }}
script: |
export USE_HOST_DEPS=1
pushd .
cd tests/py/dynamo
${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest --use-deprecated=legacy-resolver
${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
${CONDA_RUN} python -m pytest -n 10 --junitxml=${RUNNER_TEST_RESULTS_DIR}/torch_compile_be_test_results.xml backend/
${CONDA_RUN} python -m pytest -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/torch_comple_be_e2e_test_results.xml --ir torch_compile models/test_models.py
popd
Expand All @@ -183,20 +184,20 @@ jobs:
- repository: pytorch/tensorrt
package-name: torch_tensorrt
pre-script: packaging/pre_build_script.sh
uses: pytorch/tensorrt/.github/workflows/linux-test.yml@main
uses: ./.github/workflows/linux-test.yml
with:
job-name: tests-py-dynamo-core
repository: "pytorch/tensorrt"
ref: ""
test-infra-repository: pytorch/test-infra
test-infra-ref: main
test-infra-ref: release/2.1
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
pre-script: ${{ matrix.pre-script }}
script: |
export USE_HOST_DEPS=1
pushd .
cd tests/py/dynamo
${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest --use-deprecated=legacy-resolver
${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
${CONDA_RUN} python -m pytest -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_dynamo_core_runtime_test_results.xml runtime/
${CONDA_RUN} python -m pytest -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_dynamo_core_partitioning_test_results.xml partitioning/
${CONDA_RUN} python -m pytest -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_dynamo_core_lowering_test_results.xml lowering/
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/linux-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,4 +191,4 @@ jobs:

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}-${{ inputs.repository }}-${{ github.event_name == 'workflow_dispatch' }}-${{ inputs.job-name }}
cancel-in-progress: true
cancel-in-progress: true
6 changes: 3 additions & 3 deletions py/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
numpy
packaging
pybind11==2.6.2
torch==2.1.1
torchvision==0.16.1
--extra-index-url https://pypi.ngc.nvidia.com
torch==2.1.2
torchvision==0.16.2
--extra-index-url https://pypi.nvidia.com
tensorrt==8.6.1
pyyaml
2 changes: 1 addition & 1 deletion requirements-dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ pytest
transformers
timm
parameterized
expecttest
expecttest==0.1.6
4 changes: 2 additions & 2 deletions toolchains/ci_workspaces/WORKSPACE.x86_64.release.rhel.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,14 @@ http_archive(
name = "libtorch",
build_file = "@//third_party/libtorch:BUILD",
strip_prefix = "libtorch",
urls = ["https://download.pytorch.org/libtorch/test/cu121/libtorch-cxx11-abi-shared-with-deps-latest.zip"],
urls = ["https://download.pytorch.org/libtorch/test/cu121/libtorch-cxx11-abi-shared-with-deps-2.1.2%2Bcu121.zip"],
)

http_archive(
name = "libtorch_pre_cxx11_abi",
build_file = "@//third_party/libtorch:BUILD",
strip_prefix = "libtorch",
urls = ["https://download.pytorch.org/libtorch/test/cu121/libtorch-shared-with-deps-latest.zip"],
urls = ["https://download.pytorch.org/libtorch/test/cu121/libtorch-shared-with-deps-2.1.2%2Bcu121.zip"],
)

####################################################################################
Expand Down