Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial PR for VSINPU execution provider #20903

Merged
merged 3 commits into from
Jun 29, 2024
Merged

Initial PR for VSINPU execution provider #20903

merged 3 commits into from
Jun 29, 2024

Conversation

chenfeiyue-cfy
Copy link
Contributor

Description

-It is an initial PR for VSINPU execution provider

Motivation and Context

  • For support VeriSilicon hardware
  • TIM-VX(Tensor Interface Module) (https://github.com/VeriSilicon/TIM-VX) is an integrated software solution by Verisilicon for our hardware(A311D/i.MX 8M Plus etc.) design, it is easy to use Verisilicon’s hardware by simply connecting onnxruntime with the TIM-VX API by this VSINPU execution provider.

@chenfeiyue-cfy
Copy link
Contributor Author

@sunshinemyson @thezha Could you please confirm if we can sign this CLA？

github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 3, 2024
View reviewed changes
onnxruntime/core/providers/vsinpu/patches/test_scripts/compare_cosine_sim.py Fixed Show resolved Hide resolved
onnxruntime/core/providers/vsinpu/patches/test_scripts/compare_cosine_sim.py Fixed Show resolved Hide resolved
onnxruntime/core/providers/vsinpu/patches/test_scripts/compare_cosine_sim.py Fixed Show resolved Hide resolved
onnxruntime/core/providers/vsinpu/builders/op_builder_factory.h Fixed Show resolved Hide resolved
onnxruntime/core/providers/vsinpu/builders/impl/reduce_op_builder.h Fixed Show resolved Hide resolved
include/onnxruntime/core/providers/vsinpu/vsinpu_provider_factory.h Fixed Show resolved Hide resolved
onnxruntime/core/providers/vsinpu/builders/impl/clip_op_builder.h Fixed Show resolved Hide resolved
onnxruntime/core/providers/vsinpu/builders/impl/norm_op_builder.h Fixed Show resolved Hide resolved
onnxruntime/core/providers/vsinpu/builders/impl/base_op_builder.cc Fixed Show resolved Hide resolved
onnxruntime/core/providers/vsinpu/builders/impl/dequantize_op_builder.h Fixed Show resolved Hide resolved
@tianleiwu tianleiwu requested a review from jywu-msft June 3, 2024 16:47
@jywu-msft jywu-msft removed request for jywu-msft June 4, 2024 23:50
@chenfeiyue-cfy
Copy link
Contributor Author

@microsoft-github-policy-service agree company="VeriSilicon"

@jywu-msft
Copy link
Member

/azp run Windows ARM64 QNN CI Pipeline,Windows x64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,ONNX Runtime Web CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline

@jywu-msft
Copy link
Member

/azp run Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,onnxruntime-binary-size-checks-ci-pipeline,Big Models,Linux Android Emulator QNN CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 10 pipeline(s).

1 similar comment
@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 10 pipeline(s).

@jywu-msft
Copy link
Member

can you provide some details about the patch files under vsinpu/patches? why are they there?

@chenfeiyue-cfy
Copy link
Contributor Author

chenfeiyue-cfy commented Jun 28, 2024 •
edited
Loading

can you provide some details about the patch files under vsinpu/patches? why are they there?

Sure! There are three patches under this directory.I will introduce the function of each patch file separately.

  • AccuracyCorrection.patch: This patch is used to address the precision discrepancy (with a difference of 1) in the uint8 data type.
  • local_testing_record_res.patch: It is a patch of workaround method. The purpose of this patch is to temporarily and informally record each output in order, making it easier to compare the correctness and similarity between our hardware results and the golden values using other scripts later.
  • mlas_crosscompiling.patch: We need to temporary disable mlas_fp16 under our ARM64 HWs, because there exists conflict of compiliation flag when cross compiling the project. This patch is only needed on our some Specific HW.

@jywu-msft jywu-msft merged commit 56b36a5 into microsoft:main Jun 29, 2024
72 of 74 checks passed
@jiangzhiwei2018
Copy link

hi @chenfeiyue-cfy , I‘m tring to build onnxruntiom from soure code with --use_vsinpu .
What's the minimum version of TIM-VX is required ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@skottmckay skottmckay skottmckay left review comments

@jywu-msft jywu-msft jywu-msft approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@chenfeiyue-cfy @jywu-msft @jiangzhiwei2018 @skottmckay