Skip to content

Conversation

pmeier
Copy link
Collaborator

@pmeier pmeier commented Aug 18, 2023

We have an ongoing CI failure for the aarch64 builds after #7834, e.g. https://github.com/pytorch/vision/actions/runs/5892600360/job/15982284257

The jobs fail, because we are building on Linux and thus wheel relocation is enabled

# Third party imports
if sys.platform == "linux":
from auditwheel.lddtree import lddtree

but aarch64 builds no longer install auditwheel:

if [[ "$(uname)" == Darwin || "$OSTYPE" == "msys" || "$ARCH" == "aarch64" ]]; then

else
# Install native CentOS libJPEG, freetype and GnuTLS
yum install -y libjpeg-turbo-devel freetype gnutls
# Download all the dependencies required to compile image and video_reader
# extensions
mkdir -p ext_libraries
pushd ext_libraries
popd
export PATH="$(pwd)/ext_libraries/bin:$PATH"
pip install auditwheel

There are two possible options here:

  1. aarch wheels don't need relocation. This is the assumption I made in this PR, since that might be what Pre-Script Update for Aarch64 #7834 was about. Thus, I fixed our relocation script to only import auditwheel on Linux x86

  2. aarch wheels need relocation. That sounds more likely to me. In this case, we can revert the changes I made to relocate.py and apply something like

    diff --git a/packaging/pre_build_script.sh b/packaging/pre_build_script.sh
    index 7d38f2cb4e..085b90a850 100644
    --- a/packaging/pre_build_script.sh
    +++ b/packaging/pre_build_script.sh
    @@ -33,7 +33,6 @@ else
       pushd ext_libraries
       popd
       export PATH="$(pwd)/ext_libraries/bin:$PATH"
    -  pip install auditwheel
     
       # Point to custom libraries
       export LD_LIBRARY_PATH=$(pwd)/ext_libraries/lib:$LD_LIBRARY_PATH
    @@ -41,5 +40,9 @@ else
       export TORCHVISION_LIBRARY=$(pwd)/ext_libraries/lib
     fi
     
    +if [[ "$(uname)" == Linux ]]; then
    +  pip install auditwheel
    +fi
    +
     pip install numpy pyyaml future ninja
     pip install --upgrade setuptools

cc @seemethere

@pytorch-bot
Copy link

pytorch-bot bot commented Aug 18, 2023

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/7852

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit cb8e8a2 with merge base 2c44eba (image):

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@atalman
Copy link
Contributor

atalman commented Aug 21, 2023

I think we should probably revert #7834 linux aarch64 build should follow almost same logic as linux builds. Let me move the code in this PR to vision:
pytorch/test-infra#4464

if [[ ${{ inputs.architecture }} == 'aarch64' ]]; then
            ${CONDA_RUN} conda install -y libpng jpeg
            ${CONDA_RUN} conda install libpng -yq
            ${CONDA_RUN} conda install -yq ffmpeg=4.2 libjpeg-turbo -c pytorch-nightly
          fi

@pmeier
Copy link
Collaborator Author

pmeier commented Aug 23, 2023

Superseded by #7872.

@pmeier pmeier closed this Aug 23, 2023
@pmeier pmeier deleted the aarch-build branch August 24, 2023 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants