Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Qualcomm AI Engine Direct - add cli tool for QNN artifacts #4731

Merged
merged 1 commit into from
Aug 19, 2024
Merged

Qualcomm AI Engine Direct - add cli tool for QNN artifacts #4731

merged 1 commit into from
Aug 19, 2024

Conversation

haowhsu-quic
Copy link
Collaborator

Summary:

  • cli tool for deploying precompiled model library / context bin onto executorch runtime
  • refactor & mionr fixes

@haowhsu-quic
Qualcomm AI Engine Direct - add cli tool for QNN artifacts
43371c1 
Summary:
- cli tool for deploying precompiled model library / context bin
  onto executorch runtime
- refactor & mionr fixes
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 15, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/4731

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 43371c1 with merge base 54f8932 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 15, 2024
@haowhsu-quic
Copy link
Collaborator Author

Hi @cccclai, this PR is a CLI tool for helping convert pre-generated QNN artifacts (*.so / *.bin) into .pte program. We think this will make user engage with Qualcomm AIHUB more smoothly.
Please have a look, thank you!

@manuelcandales manuelcandales added the module: qnn Related to Qualcomm's QNN delegate label Aug 15, 2024
cccclai
cccclai approved these changes Aug 17, 2024
View reviewed changes
Copy link
Contributor

@cccclai cccclai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. Just one question related to the ai hub models

examples/qualcomm/qaihub_scripts/utils/README.md
@@ -0,0 +1,102 @@
# CLI Tool for Compile / Deploy Pre-Built QNN Artifacts

An easy-to-use tool for generating / executing .pte program from pre-built model libraries / context binaries from Qualcomm AI Engine Direct. Tool is verified with [host environement](../../../../docs/source/build-run-qualcomm-ai-engine-direct-backend.md#host-os).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it generic for all models from ai hub?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, artifacts from AIHUB related to QNN are delivered with .so format. Only large generative AI models are shipped with context binaries.
Both of them could be transformed into .pte program with this tool.

@kirklandsign kirklandsign merged commit 4c06907 into pytorch:main Aug 19, 2024
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cccclai cccclai cccclai approved these changes

Assignees

@cccclai cccclai

Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: qnn Related to Qualcomm's QNN delegate
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@haowhsu-quic @cccclai @facebook-github-bot @manuelcandales @kirklandsign