Qualcomm AI Engine Direct - Backend awareness quantizer

[shewu-quic]

Summary:

• Add a file backend_opinfo_adapter.py which adapts BackendOpInfo from the QNN SDK for use with ExecuTorch
  • The BackendOpInfo API is supported starting from QNN SDK 2.41 and above.
  • The BackendOpInfo which is a pybind library contains a list of quantization constraints for each operator. These quantization constraints refer to the operator definitions in QNN documents.
• Refactor QnnQuantizer
  • Add support for backend-specific annotation by implementing lazy loading of xxx_rules.py in registry_loader.py.
  • Enable validation for quantization annotation with BackendOpInfo
    • It will bypass validation for QNN SDK versions 2.41 and earlier because the BackendOpInfo API is not supported in those versions..
    • Add the backend and soc_model parameters to allow configuration of BackendOpInfo.
    • Introduce a strict parameter. By default, it is enabled, causing the validation stage to raise ValueError if quantization constraints are not met. In this mode, all quantization constraints must be satisfied to fully delegate to the QNN Backend. If disabled, the process will only log warnings instead.
    • Validation items include:
      • Verify htp_arch for LPBQ support such as conv2d op.
      • Verify htp_arch for 16a16w support such as matmul op. - Ensure SharedQuantizationSpec is used for is_math_invariant such as view op. - Check scale and zero_point constraints for certain ops, such as requiring scale = 1 / (q_max - q_min + 1) and zero_point = 0 for sigmoid op.
      • Confirm qscheme meets symmetric constraints.
      • Validate that the dtype of input and output is supported.

Test plan

Successfully tested test_qnn_delegate.py and static llama with QNN version 2.41 and above.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17665

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures, 2 Unrelated Failures

As of commit 78161c0 with merge base a5423eb ():

NEW FAILURES - The following jobs have failed:

• pull / test-mediatek-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 0f3db677bdbb5208c1f441918e6188d48559f10bdb600d75c3a3b09558f8af7e /exec failed with exit code 2
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t d3872b207ff6749b8c6408cbf91a0ab5b184cc54e5ef5b1de516b79ad323c103 /exec failed with exit code 1
• pull / test-samsung-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 7626aacfedce910759d749505ddd0cc87b4d1b51140920666d059b8f04312530 /exec failed with exit code 1
• pull / test-samsung-quantmodels-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 4d1f6fdd4b4faaea7a598da63f720b0b49faa9102a67d129cbd3d67507642d9d /exec failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 61969b8f1ab360dafe95619974165533eba08a458a26741940bbf1bb75e78f37 /exec failed with exit code 3

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest-buck / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[shewu-quic]

Hi @cccclai,
This PR is to introduce BackendOpInfo API used to validate quantization configuration for each operator from QNN SDK.
Could you please take a look?
Thanks