Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Linux/Android native deploy #980

Merged
merged 9 commits into from
Mar 29, 2018
Prev Previous commit
Next Next commit
review comment fix and java tutorial updated
  • Loading branch information
PariksheetPinjari909 committed Mar 28, 2018
commit 759772279d7a1dc18ad5d3a735825be72c4aafeb
47 changes: 33 additions & 14 deletions apps/android_deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

This folder contains Android Demo app that allows us to show how to deploy model using TVM runtime api on a Android phone.

You will need JDK, [Android NDK](https://developer.android.com/ndk) and an Android device to use this.
You will need [JDK](http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html), [Android SDK](https://developer.android.com/studio/index.html), [Android NDK](https://developer.android.com/ndk) and an Android device to use this.

## Build and Installation

### <a name="buildapk">Build APK</a>
### Build APK

We use [Gradle](https://gradle.org) to build. Please follow [the installation instruction](https://gradle.org/install) for your operating system.

Expand All @@ -26,6 +26,25 @@ dependencies {
}
```

Application default has CPU version TVM runtime flavor and follow below instruction to setup.
In `app/src/main/jni/make` you will find JNI Makefile config `config.mk` and copy it to `app/src/main/jni` and modify it.

```bash
cd apps/android_deploy/app/src/main/jni
cp make/config.mk .
```

Here's a piece of example for `config.mk`.

```makefile
APP_ABI = arm64-v8a

APP_PLATFORM = android-17

# whether enable OpenCL during compile
USE_OPENCL = 0
```

Now use Gradle to compile JNI, resolve Java dependencies and build the Android application together with tvm4j. Run following script to generate the apk file.

```bash
Expand All @@ -40,12 +59,7 @@ Upload `tvmdemo-release.apk` to your Android device and install it.

### Build with OpenCL

This application does not link any OpenCL library unless you configure it to. In `app/src/main/jni/make` you will find JNI Makefile config `config.mk`. Copy it to `app/src/main/jni` and modify it.

```bash
cd apps/android_deploy/app/src/main/jni
cp make/config.mk .
```
Application does not link with OpenCL library unless you configure it to. Modify JNI Makefile config `app/src/main/jni` with proper target OpenCL configuration.

Here's a piece of example for `config.mk`.

Expand All @@ -66,7 +80,7 @@ ADD_LDLIBS = libOpenCL.so

Note that you should specify the correct GPU development headers for your android device. Run `adb shell dumpsys | grep GLES` to find out what GPU your android device uses. It is very likely the library (libOpenCL.so) is already present on the mobile device. For instance, I found it under `/system/vendor/lib64`. You can do `adb pull /system/vendor/lib64/libOpenCL.so ./` to get the file to your desktop.

After you setup the `config.mk`, follow the instructions in [Build APK](#buildapk) to build the Android package.
After you setup the `config.mk`, follow the instructions in [Build APK](#buildapk) to build the Android package with OpenCL flavor.

## Cross Compile and Run on Android Devices

Expand All @@ -83,15 +97,20 @@ cd /opt/android-ndk/build/tools/

If everything goes well, you will find compile tools in `/opt/android-toolchain-arm64/bin`. For example, `bin/aarch64-linux-android-g++` can be used to compile C++ source codes and create shared libraries for arm64 Android devices.

### Cross Compile model and place on Android application assets folder
### Place compiled model on Android application assets folder

First select model and save compiled deploy_lib.so, deploy_graph.json and deploy_param.params refer to https://github.com/dmlc/nnvm/blob/master/tutorials/define_and_compile_model.py
Follow instruction to get compiled version model for android target [here.](https://github.com/dmlc/tvm/blob/master/docs/how_to/deploy_android.md#build-model-for-android-target)

Copied these compiled model deploy_lib.so, deploy_graph.json and deploy_param.params to apps/android_deploy/app/src/main/assets/ and make changes TVM target on MainActivity.java
Copied these compiled model deploy_lib.so, deploy_graph.json and deploy_param.params to apps/android_deploy/app/src/main/assets/ and modify TVM flavor changes on [java](https://github.com/dmlc/tvm/blob/master/apps/android_deploy/app/src/main/java/ml/dmlc/tvm/android/demo/MainActivity.java#L81)

`CPU Verison flavor`
```
private static final boolean EXE_GPU = false;
```

`OpenCL Verison flavor`
```
// create java tvm context
TVMContext tvmCtx = TVMContext.opencl();
private static final boolean EXE_GPU = true;
```


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ public class MainActivity extends AppCompatActivity {
// included with TVM and must be manually placed in the assets/ directory by the user.
// Graphs and models downloaded from https://github.com/pjreddie/darknet/blob/ may be
// converted e.g. via define_and_compile_model.py.
private static final boolean EXE_GPU = true;
private static final boolean EXE_GPU = false;
private static final int MODEL_INPUT_SIZE = 224;
private static final String MODEL_CL_LIB_FILE = "file:///android_asset/deploy_lib_opencl.so";
private static final String MODEL_CPU_LIB_FILE = "file:///android_asset/deploy_lib_cpu.so";
Expand Down
2 changes: 1 addition & 1 deletion apps/android_deploy/app/src/main/jni/Android.mk
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ MY_PATH := $(LOCAL_PATH)
include $(CLEAR_VARS)

LOCAL_PATH := $(MY_PATH)
ROOT_PATH := $(MY_PATH)/../../../../../../..
ROOT_PATH := $(MY_PATH)/../../../../../..

ifndef config
ifneq ("$(wildcard ./config.mk)","")
Expand Down
2 changes: 1 addition & 1 deletion apps/android_deploy/app/src/main/jni/build.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash
PATH="$PATH:/usr/local/bin"
CURR_DIR=$(cd `dirname $0`; pwd)
ROOT_DIR="$CURR_DIR/../../../../../../../"
ROOT_DIR="$CURR_DIR/../../../../../.."
javah -o $CURR_DIR/ml_dmlc_tvm_native_c_api.h -cp "$ROOT_DIR/jvm/core/target/*" ml.dmlc.tvm.LibInfo || exit -1
cp -f $ROOT_DIR/jvm/native/src/main/native/ml_dmlc_tvm_native_c_api.cc $CURR_DIR/ || exit -1
cp -f $ROOT_DIR/jvm/native/src/main/native/jni_helper_func.h $CURR_DIR/ || exit -1
Expand Down
1 change: 1 addition & 0 deletions apps/android_deploy/app/src/main/jni/tvm_runtime.h
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
#include "../src/runtime/file_util.cc"
#include "../src/runtime/dso_module.cc"
#include "../src/runtime/thread_pool.cc"
#include "../src/runtime/threading_backend.cc"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add #include "../src/runtime/thread_backend.cc" Please rebase against the master and test the workflow

#include "../src/runtime/graph/graph_runtime.cc"

Expand Down
50 changes: 28 additions & 22 deletions docs/how_to/deploy_android.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,17 @@
How to deploy and use compiled model on Android
===============================================
# How to deploy and use compiled model on Android

This tutorial explain below aspects (Unlike the android_rpc approach we already have)

1: Build a model for android target.
2: TVM runtime building for android target.
3: A sample native application to test the model on ADB before integrating into android application (apk).
This tutorial explain below aspects (Unlike the android_rpc approach we already have)

Hope once we have working sample at android native, Android app developers know how to integrate with JNI.
* Build a model for android target
* TVM run on Android using Java API
* TVM run on Android using Native API

As an example here is a reference block diagram.

![](https://github.com/dmlc/tvm/tree/master/docs/how_to/android_deploy.png)
![](http://www.tvmlang.org/images/release/tvm_flexible.png)

Build model for Android Target
------------------------------
## Build model for Android Target

NNVM compilation of model for android target could follow same approach like android_rpc.

Expand All @@ -23,25 +20,36 @@ An reference exampe can be found at [chainer-nnvm-example](https://github.com/tk
Above example will directly run the compiled model on RPC target. Below modification at [rum_mobile.py](https://github.com/tkat0/chainer-nnvm-example/blob/5b97fd4d41aa4dde4b0aceb0be311054fb5de451/run_mobile.py#L64) will save the compilation output which is required on android target.

```
lib.export_library("YOLOv2_tiny-aarch64.so", ndk.create_shared)
with open("YOLOv2_tiny-aarch64.json", "w") as fo:
lib.export_library("deploy_lib.so", ndk.create_shared)
with open("deploy_graph.json", "w") as fo:
fo.write(graph.json())
with open("YOLOv2_tiny-aarch64.params", "wb") as fo:
with open("deploy_param.params", "wb") as fo:
fo.write(nnvm.compiler.save_param_dict(params))
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to use 4-space indent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


YOLOv2_tiny-aarch64.so, YOLOv2_tiny-aarch64.json, YOLOv2_tiny-aarch64.params will go to android target.
deploy_lib.so, deploy_graph.json, deploy_param.params will go to android target.

## TVM run on Android using Java API
### TVM Runtime for android Target

TVM Runtime for android Target
-------------------------------
Refer [here](https://github.com/dmlc/tvm/blob/master/apps/android_deploy/README.md#build-and-installation) to build CPU/OpenCL version flavor TVM runtime for android target.

This is a cross compilation process of libtvm_runtime.so for android with OpenCL support.
### Android Native API Reference

From android java TVM API to load model & execute can be refered at this [java](https://github.com/dmlc/tvm/blob/master/apps/android_deploy/app/src/main/java/ml/dmlc/tvm/android/demo/MainActivity.java) sample source.


## TVM run on Android using Native API

### TVM Runtime for android Target

This is a cross compilation process of libtvm_runtime.so for android with OpenCL support.

Prerequisites:
Android stand alone tool chain : ANDROID_NDK_PATH/ndk/android-toolchain-arm64/bin/aarch64-linux-android-
Pls refer to https://developer.android.com/ndk/guides/standalone_toolchain.html to generate standalone toolchain for your android device.
- Android stand alone tool chain : ANDROID_NDK_PATH/ndk/android-toolchain-arm64/bin/aarch64-linux-android-g++.

Please refer [Android NDK toolchain](https://developer.android.com/ndk/guides/standalone_toolchain.html) to generate standalone toolchain for your android device.

Android OpenCL dependencies as shown below under OPENCL_PATH. These can be picked from your android build which is compatible to the target.

```
Expand Down Expand Up @@ -76,9 +84,7 @@ OPENCL_PATH=<path to OPENCL> CXX=<ANDROID_NDK_PATH>/ndk/android-toolchain-arm64/
Result of this step is the libtvm_runtime.so which is a dependency for Android native while building application.
The same can be deployed under /system/lib64/ as a system library too.


Android Native API Reference
----------------------------
### Android Native API Reference

From android native TVM API to load model & execute can be refered at this [native](https://github.com/dmlc/tvm/tree/master/tutorials/deployment) sample source.

Expand Down