Skip to content

Support lr he grpc under Azure #174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 29 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,8 @@ document section that explains the corresponding details and then guides you to
<p>
<span class="md-plain"><a href="https://cczoo.readthedocs.io/en/main/Solutions/horizontal-federated-learning/hfl.html#aliyun-ecs"><span>Alibaba Cloud</span></a>, <br />
<a href="https://cczoo.readthedocs.io/en/main/Solutions/horizontal-federated-learning/hfl.html#tencent-cloud">Tencent Cloud</a>, <br />
<a href="https://cczoo.readthedocs.io/en/main/Solutions/horizontal-federated-learning/hfl.html" target="_blank"><span>ByteDance Cloud</span></a></span>
<a href="https://cczoo.readthedocs.io/en/main/Solutions/horizontal-federated-learning/hfl.html" target="_blank"><span>ByteDance Cloud</span></a>, <br />
<a href="https://cczoo.readthedocs.io/en/main/Solutions/horizontal-federated-learning/hfl.html#microsoft-azure">Microsoft Azure</a></span>
</p>
</td>
<td>
Expand Down Expand Up @@ -199,7 +200,8 @@ document section that explains the corresponding details and then guides you to
<p>
<span class="md-plain"><span><a href="https://cczoo.readthedocs.io/en/latest/Solutions/vertical-federated-learning/vfl.html" target="_blank">Alibaba Cloud</a></span>, <br />
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/vertical-federated-learning/vfl.html" target="_blank">Tencent Cloud</a>, <br />
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/vertical-federated-learning/vfl.html" target="_blank">ByteDance Cloud</a></span>
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/vertical-federated-learning/vfl.html" target="_blank">ByteDance Cloud</a>, <br />
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/vertical-federated-learning/vfl.html" target="_blank">Microsoft Azure</a></span>
</p>
</td>
<td>
Expand Down Expand Up @@ -347,7 +349,8 @@ Cluster PPML</strong></span></a> <br />
<p>
<span class="md-plain"><a href="https://cczoo.readthedocs.io/en/main/Solutions/tensorflow-serving-cluster/index.html#alibaba-cloud"><span>Alibaba Cloud</span></a>, <br />
<a href="https://cczoo.readthedocs.io/en/main/Solutions/tensorflow-serving-cluster/index.html#tencent-cloud">Tencent Cloud</a>, <br />
<a href="https://cczoo.readthedocs.io/en/main/Solutions/tensorflow-serving-cluster/index.html#bytedance-cloud" target="_blank"><span>ByteDance Cloud</span></a></span>
<a href="https://cczoo.readthedocs.io/en/main/Solutions/tensorflow-serving-cluster/index.html#bytedance-cloud" target="_blank"><span>ByteDance Cloud</span></a>, <br />
<a href="https://cczoo.readthedocs.io/en/main/Solutions/tensorflow-serving-cluster/index.html#microsoft-azure" target="_blank"><span>Microsoft Azure</span></a></span>
</p>
</td>
<td>
Expand All @@ -359,7 +362,7 @@ Cluster PPML</strong></span></a> <br />
<span style="color:#333333;font-family:Arial;"><a href="https://cczoo.readthedocs.io/en/latest/Solutions/logistic-regression-inference-HE-SGX/index.html" target="_blank"><strong>Secure Logistic Logical Regression Inference with HE and SGX</strong></a></span>
</td>
<td>
-
Yes
</td>
<td>
-
Expand Down Expand Up @@ -392,10 +395,15 @@ Cluster PPML</strong></span></a> <br />
-
</td>
<td>
-
<p>
<span class="md-plain"><a href="https://cczoo.readthedocs.io/en/latest/Solutions/logistic-regression-inference-HE-SGX/index.html"><span>Alibaba Cloud</span></a>, <br />
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/logistic-regression-inference-HE-SGX/index.html">Tencent Cloud</a>, <br />
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/logistic-regression-inference-HE-SGX/index.html" target="_blank"><span>ByteDance Cloud</span></a>, <br />
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/logistic-regression-inference-HE-SGX/index.html" target="_blank"><span>Microsoft Azure</span></a></span>
</p>
</td>
<td>
In Progress
Published
</td>
</tr>
<tr>
Expand Down Expand Up @@ -910,6 +918,9 @@ Below table shows solutions and component projects validated in public clouds. A
</li>
</ul>
<ul>
<li>
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/logistic-regression-inference-HE-SGX/index.html" target="_blank">Secure Logistic Logical Regression Inference with HE and SGX</a>
</li>
<li>
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/grpc-ra-tls/index.html">RA-TLS Enhanced gRPC</a>
</li>
Expand Down Expand Up @@ -938,6 +949,9 @@ Below table shows solutions and component projects validated in public clouds. A
<li>
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/vertical-federated-learning/vfl.html" target="_blank">Vertical Federated Learning</a>
</li>
<li>
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/logistic-regression-inference-HE-SGX/index.html" target="_blank">Secure Logistic Logical Regression Inference with HE and SGX</a>
</li>
<li>
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/psi/PSI.html" target="_blank">Private
Set Intersection (PSI)</a><br />
Expand All @@ -955,6 +969,12 @@ Set Intersection (PSI)</a><br />
<li>
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/horizontal-federated-learning/hfl.html">Horizontal Federated Learning</a>
</li>
<li>
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/vertical-federated-learning/vfl.html" target="_blank">Vertical Federated Learning</a>
</li>
<li>
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/logistic-regression-inference-HE-SGX/index.html" target="_blank">Secure Logistic Logical Regression Inference with HE and SGX</a>
</li>
</ul>
<p>
<br />
Expand Down Expand Up @@ -983,6 +1003,9 @@ Set Intersection (PSI)</a><br />
<li>
<span></span><a href="https://cczoo.readthedocs.io/en/latest/Solutions/psi/PSI.html" target="_blank">Private Set Intersection (PSI)</a><br />
</li>
<li>
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/logistic-regression-inference-HE-SGX/index.html" target="_blank">Secure Logistic Logical Regression Inference with HE and SGX</a>
</li>
</ul>
<p>
<br />
Expand Down
29 changes: 27 additions & 2 deletions cczoo/lr_infer_he_sgx/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@

FROM ubuntu:20.04

# Optional build argument to select a build for Azure
ARG AZURE

ENV DEBIAN_FRONTEND=noninteractive
ENV INSTALL_PREFIX=/usr/local
ENV LD_LIBRARY_PATH=${INSTALL_PREFIX}/lib:${INSTALL_PREFIX}/lib/x86_64-linux-gnu:${LD_LIBRARY_PATH}
Expand Down Expand Up @@ -46,8 +49,30 @@ RUN wget -qO - https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.k
# Install SGX-PSW
RUN apt-get install -y libsgx-pce-logic libsgx-ae-qve libsgx-quote-ex libsgx-quote-ex-dev libsgx-qe3-logic sgx-aesm-service

# Install SGX-DCAP
RUN apt-get install -y libsgx-dcap-ql-dev libsgx-dcap-default-qpl libsgx-dcap-quote-verify-dev libsgx-dcap-default-qpl-dev
# Install SGX DCAP
RUN apt-get install -y libsgx-dcap-ql-dev libsgx-dcap-quote-verify-dev

# Install SGX-DCAP quote provider library
RUN if [ -z "$AZURE" ]; then \
# Not a build for Azure, so install the default quote provider library \
apt-get install -y libsgx-dcap-default-qpl; \
else \
# Build for Azure, so install the Azure DCAP Client (Release 1.10.0) \
AZUREDIR=/azure \
&& apt-get install -y libssl-dev libcurl4-openssl-dev pkg-config software-properties-common \
&& add-apt-repository ppa:team-xbmc/ppa -y \
&& apt-get update \
&& apt-get install -y nlohmann-json3-dev \
&& git clone https://github.com/microsoft/Azure-DCAP-Client ${AZUREDIR} \
&& cd ${AZUREDIR} \
&& git checkout 1.10.0 \
&& git submodule update --recursive --init \
&& cd src/Linux \
&& ./configure \
&& make DEBUG=1 \
&& make install \
&& cp libdcap_quoteprov.so /usr/lib/x86_64-linux-gnu/; \
fi

# Gramine
ENV GRAMINEDIR=/gramine
Expand Down
12 changes: 11 additions & 1 deletion cczoo/lr_infer_he_sgx/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,21 @@ After the inference is completed, the encrypted result is sent back to the user
## Build and Run
### Prerequisite
- A server with Intel SGX enabled
- Docker
- Docker Engine. Docker Engine is an open source containerization technology for
building and containerizing your applications.
Please follow [this guide](https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script)
to install Docker engine.
### Build Docker Image
```
git clone https://github.com/intel/confidential-computing-zoo
cd confidential-computing-zoo/cczoo/lr_infer_he_sgx
```
For deployments on Microsoft Azure:
```shell
AZURE=1 ./build_docker_image.sh
```
For other cloud deployments:
```shell
./build_docker_image.sh
```
### Execution
Expand Down
9 changes: 8 additions & 1 deletion cczoo/lr_infer_he_sgx/build_docker_image.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#!/bin/bash
#
# Copyright (c) 2022 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
Expand All @@ -13,9 +14,14 @@
# See the License for the specific language governing permissions and
# limitations under the License.

#!/bin/bash
set -e

if [ -z "$AZURE" ] ; then
azure=
else
azure=1
fi

# You can remove no_proxy and proxy_server if your network doesn't need it
no_proxy="localhost,127.0.0.1"
proxy_server="" # your http proxy server
Expand All @@ -26,6 +32,7 @@ DOCKER_BUILDKIT=0 docker build \
--build-arg no_proxy=${no_proxy} \
--build-arg http_proxy=${proxy_server} \
--build-arg https_proxy=${proxy_server} \
--build-arg AZURE=${azure} \
-f Dockerfile \
-t lr_infer_he_sgx:latest \
.
4 changes: 2 additions & 2 deletions cczoo/lr_infer_he_sgx/start_container.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#!/bin/bash
#
# Copyright (c) 2022 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
Expand All @@ -13,7 +14,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.

#!/bin/bash
set -e

function Usage() {
Expand Down Expand Up @@ -61,7 +61,7 @@ elif [ $1 = "server" ]; then
-e http_proxy=${proxy_server} \
-e https_proxy=${proxy_server} \
lr_infer_he_sgx:latest \
bash
/lr_infer_he_sgx/infer_server
else
Usage
exit 1
Expand Down
6 changes: 5 additions & 1 deletion documents/readthedoc/docs/source/Cloud/cloudDeployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,7 @@ The configuration of the M6ce instance as below:

---


## Microsoft Azure

Microsoft Azure [DCsv3-series](https://docs.microsoft.com/en-us/azure/virtual-machines/dcv3-series) instances support Intel® SGX encrypted computing technology.
Expand Down Expand Up @@ -319,7 +320,7 @@ The following is the configuration of the DCsv3-series instance used:
</td>
<td>
<div>
<span><span>5.13.0-1031-azure</span> </span>
<span><span>5.15.0-1022-azure</span> </span>
</div>
</td>
</tr>
Expand Down Expand Up @@ -365,6 +366,9 @@ Validated Solution&nbsp; </span>
<li>
<span><a href="https://cczoo.readthedocs.io/en/latest/Solutions/horizontal-federated-learning/hfl.html">Horizontal Federated Learning&nbsp;</a></span>
</li>
<li>
<span><a href="https://cczoo.readthedocs.io/en/latest/Solutions/logistic-regression-inference-HE-SGX/index.html">Secure Logistic Regression Inference with HE and Intel SGX&nbsp;</a></span>
</li>
<li>
<a href="https://cczoo.readthedocs.io/en/latest/Solutions/grpc-ra-tls/index.html" target="_blank">RA-TLS Enhanced gRPC</a>
</li>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,21 @@ After the inference is completed, the encrypted result is sent back to the user
## Build and Run
### Prerequisite
- A server with Intel SGX enabled
- Docker
- Docker Engine. Docker Engine is an open source containerization technology for
building and containerizing your applications.
Please follow [this guide](https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script)
to install Docker engine.
### Build Docker Image
```
git clone https://github.com/intel/confidential-computing-zoo
cd confidential-computing-zoo/cczoo/lr_infer_he_sgx
```
For deployments on Microsoft Azure:
```shell
AZURE=1 ./build_docker_image.sh
```
For other cloud deployments:
```shell
./build_docker_image.sh
```
### Execution
Expand Down