Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OpenVINO-EP] Update to latest version: OpenVINO 2019 R3.1 #2308

Merged
merged 5 commits into from
Nov 6, 2019

Conversation

psfoley
Copy link
Contributor

@psfoley psfoley commented Nov 1, 2019

Description: Brings the OpenVINO EP up to date with the latest available version (2019 R3.1)

Motivation and Context

@psfoley psfoley requested a review from a team as a code owner November 1, 2019 22:54
@@ -222,6 +222,15 @@ if (onnxruntime_USE_NGRAPH)
)
endif()

if (onnxruntime_USE_OPENVINO)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this? For openvino, I don't think it's needed. for the others, the binaries get copied to the target location so they can be added to the onnxruntime python package.
for openvino, we expect the binaries to be installed on the host system.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jywu-msft thanks for the feedback. The CPU extensions library, which is now required for the squeeze/unsqueeze operators, is a special case because the library is not installed with the OpenVINO binary and must be built separately. OpenVINO's setupvars.sh/bat file will add the path for all plugins with the exception of the CPU extensions, which is why it is being added to the python package (and the target directory for unit tests on windows).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see squeeze in the list of supported ops in OV EP https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/core/providers/openvino/openvino_execution_provider.cc#L159

Unsqueeze is there. Does this mean it wasn't really supported before? Or was support moved from CPU plugin to the CPU extensions library?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, I misspoke about the squeeze operator (it's not currently supported). Unsqueeze is the reason that the CPU extensions are required. The ONNX unsqueeze operator was previously supported, but the model optimizer in OpenVINO 2019 R1.1 and 2018 R5 converted it to the reshape operator in the OpenVINO IR, which is a layer handled by MKL-DNN with the main CPU plugin. In OpenVINO 2019 R3.1, the model optimizer maintains the ONNX unsqueeze operator as unsqueeze in the IR, an operator which is handled in the CPU extensions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation. It's unfortunate we have to add this extension lib just for a single operator.
Is this a temporary situation? Will feature OV releases incorporate the extensions?

@@ -452,6 +452,14 @@ if(WIN32)
$<TARGET_FILE_DIR:${test_data_target}>
)
endif()
if (onnxruntime_USE_OPENVINO)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this for same reason as python.cmake above.
openvino libs should already be in the path so this copy isn't necessary.

cd /opt/intel/openvino/deployment_tools/model_optimizer/install_prerequisites && sudo ./install_prerequisites_onnx.sh && \
pip install onnx>=1.1.2 networkx==2.3 numpy>=1.12.0 test-generator==0.1.1 defusedxml>=0.5.0

ENV LD_LIBRARY_PATH=/usr/lib:/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add /opt/miniconda/lib back in the PATH

ENV LD_LIBRARY_PATH=/opt/miniconda/lib:/usr/lib:/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH
ENV INTEL_OPENVINO_DIR=/opt/intel/openvino_2019.1.144
ENV INTEL_CVSDK_DIR=/opt/intel/openvino_2019.1.144
cd /opt/intel/openvino/install_dependencies && sudo ./install_openvino_dependencies.sh && sudo dpkg -i *.deb && \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert lines 38-40 back. it was already validated and works.

RUN cd onnxruntime && ./build.sh --config RelWithDebInfo --update --build --parallel --use_openvino $DEVICE --build_wheel && \
pip install build/Linux/RelWithDebInfo/dist/*-linux_x86_64.whl && rm -rf /code/onnxruntime /code/cmake-3.14.3-Linux-x86_64

RUN cd /onnxruntime && ./build.sh --config RelWithDebInfo --update --build --parallel --use_openvino $DEVICE --build_wheel && \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line 57 should be reverted back to what was there previously.

@@ -120,6 +120,8 @@ def run(self):
libs.extend(['libngraph.so', 'libcodegen.so', 'libcpu_backend.so', 'libmkldnn.so', 'libtbb_debug.so', 'libtbb_debug.so.2', 'libtbb.so', 'libtbb.so.2'])
# Nuphar Libs
libs.extend(['libtvm.so.0.5.1'])
# Openvino Libs
libs.extend(['libcpu_extension.so'])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't want to add this openvino lib to the python package right? besides, there are more dependencies than that single library.

@@ -131,6 +133,8 @@ def run(self):
libs.extend(['ngraph.dll', 'cpu_backend.dll', 'tbb.dll', 'mimalloc-override.dll', 'mimalloc-redirect.dll', 'mimalloc-redirect32.dll'])
# Nuphar Libs
libs.extend(['tvm.dll'])
# Openvino Libs
libs.extend(['cpu_extension.dll'])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment as above.

OPENVINO_VERSION=${OPENVINO_VERSION:=2019_R1.1}
git clone https://github.com/opencv/dldt.git /data/dldt/openvino_2019.1.144
OPENVINO_VERSION=${OPENVINO_VERSION:=2019_R3.1}
git clone https://github.com/opencv/dldt.git /data/dldt/openvino_2019.3.376
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this git clone still needed? it looks like you aren't supporting build openvino from source anymore?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I see. this is for the CI. we support building from source for CI?
But not with the build.py script? is that the distinction?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, for CI we try to build from opensource . But for build.py script we do it with binary to reduce the build time and also the footprint. To build dldt it takes sometime and also it does occupy some more space on the system

apt-get update && apt-get -y install libusb-1.0-0-dev

cd ${INTEL_CVSDK_DIR}/inference-engine
git checkout tags/$OPENVINO_VERSION -b $OPENVINO_VERSION
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if fetching/building openvino from source isn't supported anymore, can remove all these lines related to git

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are doing the openvino from source for CI for CPU

@@ -36,11 +36,12 @@ RUN tar -xzf l_openvino_toolkit*.tgz && \
cd - && \
rm -rf l_openvino_toolkit* && \
cd /opt/intel/openvino/install_dependencies && ./_install_all_dependencies.sh && dpkg -i *.deb && \
pip install networkx==2.3 test-generator==0.1.1 defusedxml>=0.5.0
pip install networkx==2.3 numpy>=1.12.0 test-generator==0.1.1 defusedxml>=0.5.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you revert this line as well? numpy>=1.12.0 isn't required. numpy is already installed as dependency earlier.

@@ -337,8 +337,12 @@ TEST(MathOpTest, Sub) {
test.AddOutput<float>("C", dims,
{2.0f, -2.4f, -433.3f,
0.0f, -2.0f, -164.0f,
0.0f, 0.0f, -20000.0f});
0.0f, 0.0f, -20'000.0f});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert line 340
remove the single '
we need to comply to c++11

@jywu-msft
Copy link
Member

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 12 pipeline(s).

@jywu-msft
Copy link
Member

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 12 pipeline(s).

@jywu-msft jywu-msft merged commit 1510757 into microsoft:master Nov 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants