-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[OpenVINO-EP] Update to latest version: OpenVINO 2019 R3.1 #2308
Conversation
@@ -222,6 +222,15 @@ if (onnxruntime_USE_NGRAPH) | |||
) | |||
endif() | |||
|
|||
if (onnxruntime_USE_OPENVINO) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove this? For openvino, I don't think it's needed. for the others, the binaries get copied to the target location so they can be added to the onnxruntime python package.
for openvino, we expect the binaries to be installed on the host system.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jywu-msft thanks for the feedback. The CPU extensions library, which is now required for the squeeze/unsqueeze operators, is a special case because the library is not installed with the OpenVINO binary and must be built separately. OpenVINO's setupvars.sh/bat file will add the path for all plugins with the exception of the CPU extensions, which is why it is being added to the python package (and the target directory for unit tests on windows).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see squeeze in the list of supported ops in OV EP https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/core/providers/openvino/openvino_execution_provider.cc#L159
Unsqueeze is there. Does this mean it wasn't really supported before? Or was support moved from CPU plugin to the CPU extensions library?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right, I misspoke about the squeeze operator (it's not currently supported). Unsqueeze is the reason that the CPU extensions are required. The ONNX unsqueeze operator was previously supported, but the model optimizer in OpenVINO 2019 R1.1 and 2018 R5 converted it to the reshape operator in the OpenVINO IR, which is a layer handled by MKL-DNN with the main CPU plugin. In OpenVINO 2019 R3.1, the model optimizer maintains the ONNX unsqueeze operator as unsqueeze in the IR, an operator which is handled in the CPU extensions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the explanation. It's unfortunate we have to add this extension lib just for a single operator.
Is this a temporary situation? Will feature OV releases incorporate the extensions?
@@ -452,6 +452,14 @@ if(WIN32) | |||
$<TARGET_FILE_DIR:${test_data_target}> | |||
) | |||
endif() | |||
if (onnxruntime_USE_OPENVINO) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove this for same reason as python.cmake above.
openvino libs should already be in the path so this copy isn't necessary.
dockerfiles/Dockerfile.openvino
Outdated
cd /opt/intel/openvino/deployment_tools/model_optimizer/install_prerequisites && sudo ./install_prerequisites_onnx.sh && \ | ||
pip install onnx>=1.1.2 networkx==2.3 numpy>=1.12.0 test-generator==0.1.1 defusedxml>=0.5.0 | ||
|
||
ENV LD_LIBRARY_PATH=/usr/lib:/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add /opt/miniconda/lib back in the PATH
dockerfiles/Dockerfile.openvino
Outdated
ENV LD_LIBRARY_PATH=/opt/miniconda/lib:/usr/lib:/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH | ||
ENV INTEL_OPENVINO_DIR=/opt/intel/openvino_2019.1.144 | ||
ENV INTEL_CVSDK_DIR=/opt/intel/openvino_2019.1.144 | ||
cd /opt/intel/openvino/install_dependencies && sudo ./install_openvino_dependencies.sh && sudo dpkg -i *.deb && \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert lines 38-40 back. it was already validated and works.
dockerfiles/Dockerfile.openvino
Outdated
RUN cd onnxruntime && ./build.sh --config RelWithDebInfo --update --build --parallel --use_openvino $DEVICE --build_wheel && \ | ||
pip install build/Linux/RelWithDebInfo/dist/*-linux_x86_64.whl && rm -rf /code/onnxruntime /code/cmake-3.14.3-Linux-x86_64 | ||
|
||
RUN cd /onnxruntime && ./build.sh --config RelWithDebInfo --update --build --parallel --use_openvino $DEVICE --build_wheel && \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
line 57 should be reverted back to what was there previously.
@@ -120,6 +120,8 @@ def run(self): | |||
libs.extend(['libngraph.so', 'libcodegen.so', 'libcpu_backend.so', 'libmkldnn.so', 'libtbb_debug.so', 'libtbb_debug.so.2', 'libtbb.so', 'libtbb.so.2']) | |||
# Nuphar Libs | |||
libs.extend(['libtvm.so.0.5.1']) | |||
# Openvino Libs | |||
libs.extend(['libcpu_extension.so']) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we don't want to add this openvino lib to the python package right? besides, there are more dependencies than that single library.
@@ -131,6 +133,8 @@ def run(self): | |||
libs.extend(['ngraph.dll', 'cpu_backend.dll', 'tbb.dll', 'mimalloc-override.dll', 'mimalloc-redirect.dll', 'mimalloc-redirect32.dll']) | |||
# Nuphar Libs | |||
libs.extend(['tvm.dll']) | |||
# Openvino Libs | |||
libs.extend(['cpu_extension.dll']) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same comment as above.
OPENVINO_VERSION=${OPENVINO_VERSION:=2019_R1.1} | ||
git clone https://github.com/opencv/dldt.git /data/dldt/openvino_2019.1.144 | ||
OPENVINO_VERSION=${OPENVINO_VERSION:=2019_R3.1} | ||
git clone https://github.com/opencv/dldt.git /data/dldt/openvino_2019.3.376 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this git clone still needed? it looks like you aren't supporting build openvino from source anymore?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I see. this is for the CI. we support building from source for CI?
But not with the build.py script? is that the distinction?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, for CI we try to build from opensource . But for build.py script we do it with binary to reduce the build time and also the footprint. To build dldt it takes sometime and also it does occupy some more space on the system
apt-get update && apt-get -y install libusb-1.0-0-dev | ||
|
||
cd ${INTEL_CVSDK_DIR}/inference-engine | ||
git checkout tags/$OPENVINO_VERSION -b $OPENVINO_VERSION |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if fetching/building openvino from source isn't supported anymore, can remove all these lines related to git
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we are doing the openvino from source for CI for CPU
dockerfiles/Dockerfile.openvino
Outdated
@@ -36,11 +36,12 @@ RUN tar -xzf l_openvino_toolkit*.tgz && \ | |||
cd - && \ | |||
rm -rf l_openvino_toolkit* && \ | |||
cd /opt/intel/openvino/install_dependencies && ./_install_all_dependencies.sh && dpkg -i *.deb && \ | |||
pip install networkx==2.3 test-generator==0.1.1 defusedxml>=0.5.0 | |||
pip install networkx==2.3 numpy>=1.12.0 test-generator==0.1.1 defusedxml>=0.5.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you revert this line as well? numpy>=1.12.0 isn't required. numpy is already installed as dependency earlier.
@@ -337,8 +337,12 @@ TEST(MathOpTest, Sub) { | |||
test.AddOutput<float>("C", dims, | |||
{2.0f, -2.4f, -433.3f, | |||
0.0f, -2.0f, -164.0f, | |||
0.0f, 0.0f, -20000.0f}); | |||
0.0f, 0.0f, -20'000.0f}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert line 340
remove the single '
we need to comply to c++11
ef672a8
to
5bb67f8
Compare
/azp run |
Azure Pipelines successfully started running 12 pipeline(s). |
/azp run |
Azure Pipelines successfully started running 12 pipeline(s). |
Description: Brings the OpenVINO EP up to date with the latest available version (2019 R3.1)
Motivation and Context
https://software.intel.com/en-us/articles/OpenVINO-RelNotes