TVM RPC Server

This folder contains a simple recipe to make RPC server in c++.

Usage (Non-Windows)

• Configure the tvm cmake build with config.cmake ensuring that USE_CPP_RPC is set to ON in the config.
• If cross compiling for Android, add the following options to the cmake config or specify them when invoking cmake:

  # Whether to build the C++ RPC server binary
  set(USE_CPP_RPC ON)
  # Path to the Android NDK cmake toolchain
  set(CMAKE_TOOLCHAIN_FILE $ENV{ANDROID_NDK}/build/cmake/android.toolchain.cmake)
  # The Android ABI and platform to target
  set(ANDROID_ABI "arm64-v8a")
  set(ANDROID_PLATFORM android-28)

• Similarly, if cross compiling for embedded Linux add the following options to cmake config:

  # Needed to ensure pthread is linked
  set(OS Linux)
  # Path to the desired C++ cross compiler
  set(CMAKE_CXX_COMPILER /path/to/cross/compiler/executable)

• If linking against a custom device OpenCL library is needed, in the config specify the path to the OpenCL SDK containing the include/CL headers and lib/ or lib64/libOpenCL.so:

  set(USE_OPENCL /path/to/opencl-sdk)

• From within the configured tvm build directory, compile tvm_runtime and the tvm_rpc server:

  cd $TVM_ROOT/build
  make -jN tvm_runtime tvm_rpc

• Use ./tvm_rpc server to start the RPC server

Usage (Windows)

• Configure the tvm cmake build with config.cmake ensuring that USE_CPP_RPC is set to ON in the config.
• Install LLVM pre-build binaries, making sure to select the option to add it to the PATH.
• Verify Python 3.6 or newer is installed and in the PATH.
• Use <tvm_output_dir>\tvm_rpc.exe to start the RPC server

How it works

• The tvm runtime dll is linked along with this executable and when the RPC server starts it will load the tvm runtime library.

Command line usage
 server       - Start the server
--host        - The hostname of the server, Default=0.0.0.0
--port        - The port of the RPC, Default=9090
--port-end    - The end search port of the RPC, Default=9199
--tracker     - The RPC tracker address in host:port format e.g. 10.1.1.2:9190 Default=""
--key         - The key used to identify the device type in tracker. Default=""
--custom-addr - Custom IP Address to Report to RPC Tracker. Default=""
--silent      - Whether to run in silent mode. Default=False
  Example
  ./tvm_rpc server --host=0.0.0.0 --port=9000 --port-end=9090 --tracker=127.0.0.1:9190 --key=rasp

Note

Currently support is only there for Linux / Android / Windows environment and proxy mode isn't supported currently.