Add stories ci for qnn #4662

cccclai · 2024-08-11T20:49:39Z

As title, add stories end to end ci for qnn. See the success job in https://github.com/pytorch/executorch/actions/runs/10347877541/job/28638999876

pytorch-bot · 2024-08-11T20:49:42Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/4662

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 5174d8f with merge base 99e1ae1 ():

NEW FAILURE - The following job has failed:

docker-builds / docker-build (executorch-ubuntu-22.04-qnn-sdk) (gh)
name unknown: The repository with name 'executorch/executorch-ubuntu-22.04-qnn-sdk' does not exist in the registry

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-08-12T17:29:28Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-08-12T19:51:10Z

As a note, this pr just test fp, and quantized version need to be added later

facebook-github-bot · 2024-08-12T21:48:27Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

kirklandsign · 2024-08-12T22:31:12Z

.ci/docker/ubuntu/Dockerfile

+COPY --chown=ci-user:ci-user ./qualcomm /opt/qualcomm
+# Set up QNN SDK if needed
+RUN if [ -n "${QNN_SDK}" ]; then git config --global user.email "ossci@example.com"; git config --global user.name "OSS CI"; fi
+


ARG QNN_SDK COPY --chown=ci-user:ci-user ./scripts/setup-qnn-deps.sh setup-qnn-deps.sh # Set up QNN SDK if needed RUN if [ -n "${QNN_SDK}" ]; then ./setup-qnn-deps.sh; fi

ARG QNN_SDK
COPY --chown=ci-user:ci-user ./scripts/setup-qnn-deps.sh setup-qnn-deps.sh
RUN mkdir /opt/qualcomm; chown ci-user:ci-user /opt/qualcomm
RUN if [ -n "${QNN_SDK}" ]; then ./setup-qnn-deps.sh; fi

ARG QNN_SDK
COPY --chown=ci-user:ci-user ./scripts/setup-qnn-deps.sh setup-qnn-deps.sh
RUN if [ -n "${QNN_SDK}" ]; then mkdir /opt/qualcomm; chown ci-user:ci-user /opt/qualcomm; ./setup-qnn-deps.sh; fi

kirklandsign · 2024-08-12T22:32:49Z

.ci/scripts/setup-qnn-deps.sh

+
+install_qnn() {
+  echo "Start installing qnn."
+  QNN_INSTALLATION_DIR=/tmp/qnn


Maybe /opt instead of /tmp ?

Yeah will address it in next diff

guangy10 · 2024-08-12T22:53:20Z

.ci/docker/build.sh

@@ -37,6 +37,10 @@ case "${IMAGE_NAME}" in
    ARM_SDK=yes
    CLANG_VERSION=12
    ;;
+  executorch-ubuntu-22.04-qnn-sdk)


Why not adding qnn sdk by extending the executorch-ubuntu-22.04-clang12-android image below? QNN backend would require ndk and cross-compilation, making it a good fit to the android image

this is not android, it's x86

guangy10

@cccclai Thanks for adding the QNN deps to the docker image! We'er thinking of doing the same work for benchmarking infra.

guangy10

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

cccclai · 2024-08-12T23:08:48Z

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

guangy10 · 2024-08-12T23:29:30Z

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

kirklandsign · 2024-08-12T23:31:23Z

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

guangy10 · 2024-08-12T23:34:51Z

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86. The wrapper script .ci/scripts/setup-qnn-deps.sh is reusable.

guangy10

unblocking after clarification

kirklandsign · 2024-08-12T23:37:42Z

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86.

Oh we want a separate docker like executorch-ubuntu-22.04-android which sets QNN as well, plus other android components? If not other components, we can use executorch-ubuntu-22.04-qnn-sdk. Otherwise .ci/scripts/setup-qnn-deps.sh is reusable.

guangy10 · 2024-08-12T23:53:38Z

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86.

Oh we want a separate docker like executorch-ubuntu-22.04-android which sets QNN as well, plus other android components? If not other components, we can use executorch-ubuntu-22.04-qnn-sdk. Otherwise .ci/scripts/setup-qnn-deps.sh is reusable.

If I understand @cccclai 's comment correctly, executorch-ubuntu-22.04-qnn-sdk is for Linux x86 not for android. If we extend executorch-ubuntu-22.04-qnn-sdk by adding additional android deps to build runners and apps, it's kind of duplicate to executorch-ubuntu-22.04-android. A better option seems to be extending executorch-ubuntu-22.04-android with QNN SDK for benchmarking and leave executorch-ubuntu-22.04-qnn-sdk is for Linux x86 w/ emulator only. @cccclai @kirklandsign what do you guys think?

kirklandsign · 2024-08-13T00:02:11Z

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86.

Oh we want a separate docker like executorch-ubuntu-22.04-android which sets QNN as well, plus other android components? If not other components, we can use executorch-ubuntu-22.04-qnn-sdk. Otherwise .ci/scripts/setup-qnn-deps.sh is reusable.

If I understand @cccclai 's comment correctly, executorch-ubuntu-22.04-qnn-sdk is for Linux x86 not for android. If we extend executorch-ubuntu-22.04-qnn-sdk by adding additional android deps to build runners and apps, it's kind of duplicate to executorch-ubuntu-22.04-android. A better option seems to be extending executorch-ubuntu-22.04-android with QNN SDK for benchmarking and leave executorch-ubuntu-22.04-qnn-sdk is for Linux x86 w/ emulator only. @cccclai @kirklandsign what do you guys think?

Oh I guess @cccclai means the CI is for linux x86. The docker is pulling in QNN SDK so it can be used for everything which requires QNN SDK (including Android)

guangy10 · 2024-08-13T00:29:53Z

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86.

Oh we want a separate docker like executorch-ubuntu-22.04-android which sets QNN as well, plus other android components? If not other components, we can use executorch-ubuntu-22.04-qnn-sdk. Otherwise .ci/scripts/setup-qnn-deps.sh is reusable.

If I understand @cccclai 's comment correctly, executorch-ubuntu-22.04-qnn-sdk is for Linux x86 not for android. If we extend executorch-ubuntu-22.04-qnn-sdk by adding additional android deps to build runners and apps, it's kind of duplicate to executorch-ubuntu-22.04-android. A better option seems to be extending executorch-ubuntu-22.04-android with QNN SDK for benchmarking and leave executorch-ubuntu-22.04-qnn-sdk is for Linux x86 w/ emulator only. @cccclai @kirklandsign what do you guys think?

Oh I guess @cccclai means the CI is for linux x86. The docker is pulling in QNN SDK so it can be used for everything which requires QNN SDK (including Android)

I understand that part, and @kirklandsign what's your actionable suggestion? Like expending executorch-ubuntu-22.04-qnn-sdk for android benchmarking and delete executorch-ubuntu-22.04-android, or something else?

kirklandsign · 2024-08-13T00:53:12Z

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86.

Oh we want a separate docker like executorch-ubuntu-22.04-android which sets QNN as well, plus other android components? If not other components, we can use executorch-ubuntu-22.04-qnn-sdk. Otherwise .ci/scripts/setup-qnn-deps.sh is reusable.

If I understand @cccclai 's comment correctly, executorch-ubuntu-22.04-qnn-sdk is for Linux x86 not for android. If we extend executorch-ubuntu-22.04-qnn-sdk by adding additional android deps to build runners and apps, it's kind of duplicate to executorch-ubuntu-22.04-android. A better option seems to be extending executorch-ubuntu-22.04-android with QNN SDK for benchmarking and leave executorch-ubuntu-22.04-qnn-sdk is for Linux x86 w/ emulator only. @cccclai @kirklandsign what do you guys think?

Oh I guess @cccclai means the CI is for linux x86. The docker is pulling in QNN SDK so it can be used for everything which requires QNN SDK (including Android)

I understand that part, and @kirklandsign what's your actionable suggestion? Like expending executorch-ubuntu-22.04-qnn-sdk for android benchmarking and delete executorch-ubuntu-22.04-android, or something else?

I see. I would say we can keep two docker images for now. Just in case -qnn-sdk and -android diverged further next.

A quick patch for -android is in https://github.com/pytorch/executorch/pull/4662/files#diff-e6d9e0419b8aac6f9b7664fe9d2398da07ab195b26f0dce338e13968511daba5R46, add QNN_SDK=yes We get QNN SDK set up for android image for free

cccclai · 2024-08-13T01:45:34Z

backends/qualcomm/scripts/build.sh

@@ -25,9 +25,9 @@ usage() {
 [ "$1" = -h ] && usage

 BUILD_X86_64="true"
-CMAKE_X86_64="cmake-out"
+CMAKE_X86_64="build-x86"


@chiwwang I rename it to the similar name as before, because the ci job will clean up cmake-out for every run. We can make build.sh takes configurable name as next step and change the default name to cmake-out and cmake-out-android

facebook-github-bot · 2024-08-13T03:20:25Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

update qnn sdk

61c57ca

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 11, 2024

cccclai marked this pull request as draft August 11, 2024 20:49

cccclai added 26 commits August 11, 2024 16:40

add try 3

488e98b

more fix to sdk download

b16a17b

fix the qnn sdk path

0e4cd80

fix qnn sdk path

b25acc8

fix qnn sdk path

0cd8823

fix qnn sdk path

34fce2a

fix qnn path

ebaf390

fix next

f912c76

fix qnn path

03919c8

fix conda run

00024a9

set env

12902f8

install

6565dbd

fix build script

16992fa

fix build

8b2aa83

skip aarch64

7f56881

change job number

ccae714

change job numbers

4a76821

change job numbers

5f8975c

fix qnn build

822df7b

add export file

61e2b1d

fix python import

14e5aa8

copy .fbs file

35101b7

fix unbound var

6d6ad5e

fix unbound var

f7755c9

copy .so

2270a79

fix copy so

e451533

move to trunk

757c62e

cccclai added 3 commits August 12, 2024 14:56

move to trunk

be000bd

use qnn docker

fa09ace

fix docker

5a9fe4d

kirklandsign reviewed Aug 12, 2024

View reviewed changes

guangy10 reviewed Aug 12, 2024

View reviewed changes

guangy10 requested changes Aug 12, 2024

View reviewed changes

guangy10 approved these changes Aug 12, 2024

View reviewed changes

fix mac llama

67062b0

fix mac llama

6da04ae

cccclai commented Aug 13, 2024

View reviewed changes

update qnn build output place

5174d8f

facebook-github-bot merged commit e71fa03 into main Aug 13, 2024
70 of 72 checks passed

cccclai mentioned this pull request Aug 13, 2024

fix qnn docker build #4683

Closed

guangy10 mentioned this pull request Aug 15, 2024

Fix test spec for scheduled jobs #4734

Closed

Add stories ci for qnn #4662

Add stories ci for qnn #4662

Uh oh!

Conversation

cccclai commented Aug 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Aug 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/4662

❌ 1 New Failure

Uh oh!

facebook-github-bot commented Aug 12, 2024

Uh oh!

cccclai commented Aug 12, 2024

Uh oh!

facebook-github-bot commented Aug 12, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

guangy10 left a comment

Choose a reason for hiding this comment

Uh oh!

guangy10 left a comment

Choose a reason for hiding this comment

Uh oh!

cccclai commented Aug 12, 2024

Uh oh!

guangy10 commented Aug 12, 2024

Uh oh!

kirklandsign commented Aug 12, 2024

Uh oh!

guangy10 commented Aug 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

guangy10 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kirklandsign commented Aug 12, 2024

Uh oh!

guangy10 commented Aug 12, 2024

Uh oh!

kirklandsign commented Aug 13, 2024

Uh oh!

guangy10 commented Aug 13, 2024

Uh oh!

kirklandsign commented Aug 13, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Aug 13, 2024

Uh oh!

Uh oh!

Uh oh!

cccclai commented Aug 11, 2024 •

edited

Loading

pytorch-bot bot commented Aug 11, 2024 •

edited

Loading

guangy10 commented Aug 12, 2024 •

edited

Loading

guangy10 left a comment •

edited

Loading