[WIP] Linux/Android native deploy #980

PariksheetPinjari909 · 2018-03-09T13:48:08Z

tqchen · 2018-03-09T17:12:12Z

@PariksheetPinjari909 this example uses the C API and requires JNI. Can you build a version that uses the java API instead of JNI? The Fix for thread runtime is useful and we can bring that in first in a separate PR. Thanks!

Specifically, we can:

Build a library.so with export_library
Build TVM java runtime without RPC support
Use TVM java runtime to run the prediction in java

tqchen · 2018-03-09T17:14:40Z

The native JNI deployment can also be helpful, but it would be great to show a java API example, since that is much easier for the users. Also if we are only running prediction example, we don't need to have extra library to save arrays, simply print out the class id and possible class would do for a demo app

tqchen · 2018-03-09T17:14:44Z

cc @yzhliu

PariksheetPinjari909 · 2018-03-10T06:41:18Z

Ref: "Talk about how to use the java API": Yes, build with JAVA makes it much easier than JNI for android app developers. Working in it

Ref: " we don't need to have extra library to save arrays, simply print out the class id and possible class would do for a demo app"

NPY format for in & out was with an intention of developers first try with Python for initial experiments on X86 then bring it to target.
Input varies like direct JPEG stream, decoded Tensor (resized, converted to YUV domain ...etc.)

So to keep the deploy tool generic I preferred npy format. NPY format dump is a single addition to initial python script to dump input and to feed model output back for processing after prediction.

tqchen · 2018-03-10T19:11:17Z

Usually, app developer wants something that is more stand alone. Like take an image and how could I feed it to the app. We can build an example app that takes an image(from camera or file) do a preprocessing and feed it to java runtime. See https://github.com/dmlc/nnvm/blob/master/tutorials/web/resnet.html#L80 for a javascript version of preprocessing

PariksheetPinjari909 · 2018-03-20T10:31:16Z

Android Standalone demo application tutorial updated.

tqchen · 2018-03-20T19:09:31Z

@yzhliu can you review this?

janboeye · 2018-03-24T04:40:36Z

@PariksheetPinjari909
How to native deploy tvm.build for just one layer? Is it same?

Thanks

yzhliu · 2018-03-25T14:37:20Z

docs/how_to/deploy_android.md

+        fo.write(graph.json())
+with open("YOLOv2_tiny-aarch64.params", "wb") as fo:
+        fo.write(nnvm.compiler.save_param_dict(params))
+```


better to use 4-space indent.

yzhliu · 2018-03-25T14:49:54Z

docs/how_to/deploy_android.md

+
+
+Prerequisites:
+Android stand alone tool chain : ANDROID_NDK_PATH/ndk/android-toolchain-arm64/bin/aarch64-linux-android-


The path looks incomplete? Moreover, I think users need extra steps to generate the standalone toolchains from NDK, e.g., by $ANDROID_NDK_PATH/build/tools/ ./make-standalone-toolchain.sh ?

path is complete only, it's tool chain prefix reference.

About standalone tool chain building: I think it's out of scope for current context. Any way I added android URL reference to build it.

yzhliu · 2018-03-25T14:50:38Z

docs/how_to/deploy_android.md

+OPENCL_PATH=<path to OOPENCL path> CXX=<ANDROID_NDK_PATH>/ndk/android-toolchain-arm64/bin/aarch64-linux-android-g++ make lib/libtvm_runtime.so
+```
+
+Result of this setp is the libtvm_runtime.so which is a dependency for Android native while building application.


typo 'setp'

yzhliu · 2018-03-25T14:51:37Z

docs/how_to/deploy_android.md

+- Below command can build libtvm_runtime.so for android.
+
+```
+OPENCL_PATH=<path to OOPENCL path> CXX=<ANDROID_NDK_PATH>/ndk/android-toolchain-arm64/bin/aarch64-linux-android-g++ make lib/libtvm_runtime.so


OOPENCL -> OPENCL

yzhliu · 2018-03-25T14:53:01Z

docs/how_to/deploy_android.md

+    └── libOpenCL.so
+```
+
+- Enable OPENCL in make/config.mk


Shall we remind users to make sure whether the libOpenCL.so is compatible with their devices?

Done. Added required explanation for this.

yzhliu · 2018-03-25T14:56:25Z

tutorials/deployment/android/README.md

+```
+            // create tvm context
+            TVMContext tvmCtx = TVMContext.opencl();
+```


yzhliu · 2018-03-25T14:59:18Z

tutorials/deployment/native/README.txt

@@ -0,0 +1,62 @@
+/*!


how about README.md instead of .txt?

yzhliu · 2018-03-25T15:01:06Z

tutorials/deployment/native/deploy_native.cc

+    uint64_t num_samples=1;
+    TVMArrayAlloc(in_shape, in_ndim, dtype_code, dtype_bits, dtype_lanes, kDLCPU, device_id, &x);
+
+    // Initlalize the Input


typo 'Initlalize'

PariksheetPinjari909 · 2018-03-26T14:08:07Z

@janboeye yes the procedure is same for one layer too.

tqchen · 2018-03-27T17:12:18Z

@PariksheetPinjari909 Thanks for the changes, let us move tutorials/deployment/android -> apps/android_deploy
because this is less like a conventional tutorial, but more of an example app

tqchen · 2018-03-28T04:17:02Z

apps/android_deploy/app/src/main/jni/tvm_runtime.h

+#include "../src/runtime/file_util.cc"
+#include "../src/runtime/dso_module.cc"
+#include "../src/runtime/thread_pool.cc"
+


add #include "../src/runtime/thread_backend.cc" Please rebase against the master and test the workflow

tqchen · 2018-03-28T04:17:40Z

docs/how_to/deploy_android.md

+
+    1: Build a model for android target.
+    2: TVM runtime building for android target.
+    3: A sample native application to test the model on ADB before integrating into android application (apk).


I think we should first refer to the apps/deploy_android on how to use java API, the native API comes second

tqchen · 2018-03-28T21:27:11Z

@PariksheetPinjari909 Let us remove the native deployment tutorial for now as it is in largely duplicates with the apps/how_to_deploy. Focusing on the java api example and make it clear and good is useful for most users

We can still keep a section on what is needed to be done if user prefers JNI and refers most of example code to apps/how_to_deploy.

tqchen

Thanks for all the changes, let us remove the native example for now and focus on making the java example clear and useful

tqchen · 2018-03-29T16:40:05Z

Thanks for the patient during the reviewing process and keep improving the PR, this is now merged

PariksheetPinjari909 mentioned this pull request Mar 9, 2018

Android Standalone Deployment Tutorial #973

Closed

4 tasks

PariksheetPinjari909 force-pushed the tf branch from f65b51f to 748cf63 Compare March 10, 2018 06:26

PariksheetPinjari909 changed the title ~~Linux/Android native deploy~~ [WIP] Linux/Android native deploy Mar 10, 2018

tqchen requested review from tqchen and yzhliu March 10, 2018 23:36

tqchen added the status: review in progress label Mar 10, 2018

yzhliu reviewed Mar 25, 2018

View reviewed changes

yzhliu approved these changes Mar 26, 2018

View reviewed changes

PariksheetPinjari909 added 5 commits March 27, 2018 09:01

Linux/Android native deploy

a50aee5

Android standalone demo application

b1a4176

Native config makefile is added

0c0a7c5

Android standalone tutorial review comment fixed

a3db99f

Some changes based on review.

5f8e5f8

PariksheetPinjari909 force-pushed the tf branch from 1a9fa69 to 5f8e5f8 Compare March 27, 2018 03:32

PariksheetPinjari909 added 2 commits March 28, 2018 09:36

tvm standalone demo application move to apps/android_deploy/

d6e27b0

Merge branch 'master' into tf

f64b90f

tqchen reviewed Mar 28, 2018

View reviewed changes

tqchen requested changes Mar 28, 2018

View reviewed changes

review comment fix and java tutorial updated

7597722

tqchen requested changes Mar 28, 2018

View reviewed changes

TVM Android standalone JAVA tutorial update and removed Native tutorial

b5c7754

tqchen approved these changes Mar 29, 2018

View reviewed changes

tqchen merged commit 043ed4b into apache:master Mar 29, 2018

tqchen pushed a commit to tqchen/tvm that referenced this pull request Jul 6, 2018

[WIP] Linux/Android native deploy (apache#980)

18db02a

sergei-mironov pushed a commit to sergei-mironov/tvm that referenced this pull request Aug 8, 2018

[WIP] Linux/Android native deploy (apache#980)

c902741

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Linux/Android native deploy #980

[WIP] Linux/Android native deploy #980

PariksheetPinjari909 commented Mar 9, 2018

tqchen commented Mar 9, 2018

tqchen commented Mar 9, 2018

tqchen commented Mar 9, 2018

PariksheetPinjari909 commented Mar 10, 2018

tqchen commented Mar 10, 2018

PariksheetPinjari909 commented Mar 20, 2018

tqchen commented Mar 20, 2018

janboeye commented Mar 24, 2018

yzhliu Mar 25, 2018

PariksheetPinjari909 Mar 26, 2018

yzhliu Mar 25, 2018

PariksheetPinjari909 Mar 26, 2018

yzhliu Mar 25, 2018

PariksheetPinjari909 Mar 26, 2018

yzhliu Mar 25, 2018

yzhliu Mar 25, 2018

PariksheetPinjari909 Mar 26, 2018

yzhliu Mar 25, 2018

yzhliu Mar 25, 2018

PariksheetPinjari909 Mar 26, 2018

yzhliu Mar 25, 2018

PariksheetPinjari909 commented Mar 26, 2018

tqchen commented Mar 27, 2018 •

edited

Loading

tqchen Mar 28, 2018

tqchen Mar 28, 2018

tqchen commented Mar 28, 2018

tqchen left a comment

tqchen commented Mar 29, 2018



		Prerequisites:
		Android stand alone tool chain : ANDROID_NDK_PATH/ndk/android-toolchain-arm64/bin/aarch64-linux-android-

[WIP] Linux/Android native deploy #980

[WIP] Linux/Android native deploy #980

Conversation

PariksheetPinjari909 commented Mar 9, 2018

tqchen commented Mar 9, 2018

tqchen commented Mar 9, 2018

tqchen commented Mar 9, 2018

PariksheetPinjari909 commented Mar 10, 2018

tqchen commented Mar 10, 2018

PariksheetPinjari909 commented Mar 20, 2018

tqchen commented Mar 20, 2018

janboeye commented Mar 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PariksheetPinjari909 commented Mar 26, 2018

tqchen commented Mar 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tqchen commented Mar 28, 2018

tqchen left a comment

Choose a reason for hiding this comment

tqchen commented Mar 29, 2018

tqchen commented Mar 27, 2018 •

edited

Loading