Qualcomm AI Engine Direct - Enable custom operator

[shewu-quic]

Summary:

• Support to register op package in QNN Backend
• Add example script to run torch custom op with QNN Op package
• Allow op package override torch built-in operator
• Add op package example
• Modify the flag of dlopen for QNN library
• Generate custom op based on the meta and _schema.arguments of torch.fx.Node
• Add README for the custom op

Reproduce commands:

# Follow the README to Install qpm
# Follow the README to install hexagon-sdk and hexagon-tool
# install hexagon sdk 5.4.0 for SM8650
# qpm-cli --install hexagonsdk5.x --version 5.4.0.3 --path /path/to/Qualcomm/Hexagon_SDK/hexagon-sdk-5.4.0
# install hexagon sdk 6.0.0 for x86
# qpm-cli --install hexagonsdk6.x --version 6.0.0.2 --path /path/to/Qualcomm/Hexagon_SDK/hexagon-sdk-6.0.0
# install hexagon tool 8.8.02 for x86
# qpm-cli --extract hexagon8.8 --version 8.8.02.1 --path /path/to/Qualcomm/Hexagon_SDK/hexagon-sdk-6.0.0/tools/HEXAGON_Tools/8.8.02

export HEXAGON_SDK_ROOT=/path/to/hexagon-sdk-5.4.0
export ANDROID_NDK_ROOT=/path/to/android-ndk-r26c
# use clang-9.0.0
export X86_CXX=/path/to/clang++
# run custom op with example script
python3 examples/qualcomm/custom_op/custom_ops_1.py --build_folder build-android -s <device_serial> -H <host> -m SM8650 --op_package_dir examples/qualcomm/custom_op/example_op_package_htp/ExampleOpPackage --build_op_package
# run custom op with unit test
python3 backends/qualcomm/tests/test_qnn_delegate.py TestUtilScript.test_custom_op -b build-android -s <device_serial> -H <host> -m SM8650 --op_package_dir examples/qualcomm/custom_op/example_op_package_htp/ExampleOpPackage -r </path/to/executorch> -a </path/to/artifacts>

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8726

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Unrelated Failure

As of commit 064d12f with merge base 5a594a7 ():

NEW FAILURES - The following jobs have failed:

• Check Labels / Check labels (gh)
  RuntimeError: Error checking labels: PR does not have required labels
• pull / unittest-buck / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / test-llava-runner-linux / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[shewu-quic]

Hi @cccclai,

This PR is to support custom kernel in QNN Backend.
Could you please help to take a look?
If you have any problems, please let me know. Thanks :)

[cccclai]

I may not know enough details, but can you tell me how qnn backend knows this custom op can be consumed? Previously I was thinking qnn custom ops can be registered in a specific namespace, but it looks a bit different to me.

[shewu-quic]

Certainly. QNN uses the op package mechanism to create custom ops. Once the op package is prepared, it is registered using qnn_backend_register_op_package. In the op builder, you provide the corresponding op_package_name , qnn_op_type_name and schema to create a QNN node.
In the executorh, through compile_spec, you pass QnnExecuTorchOpPackageOptions which includes the op package info and custom_op_name. Using the custom_op_name, such as my_op_lib.mul3.default, a CustomOp builder is created to consume the custom op.