Update RK3562/RK3566/RK3568/RK3588/RV1103/RV1106 NPU SDK to 1.5.2

Signed-off-by: Randall Zhuo <randall.zhuo@rock-chips.com>
rockchip-linux · Aug 28, 2023 · f29bfee · f29bfee
1 parent 4e2101a
commit f29bfee
Show file tree

Hide file tree

Showing 101 changed files with 2,786 additions and 474 deletions.
diff --git a/README.md b/README.md
@@ -20,7 +20,15 @@ Note:
 
 ## ReleaseLog
 
-# 1.5.0
+### 1.5.2
+- Improved dynamic shape support
+- Improved matmul api support
+- Add GPU back-end implementations for some operators such as matmul
+- Improve transformer support
+- Reduce rknn_init memory usage
+- Optimize rknn_init time-consuming
+
+### 1.5.0
 
 - Support RK3562
 - Support more NPU operator fuse, such as Conv-Silu/Conv-Swish/Conv-Hardswish/Conv-sigmoid/Conv-HardSwish/Conv-Gelu ..

diff --git a/doc/RKNN_Compiler_Support_Operator_List_v1.5.0.pdf b/doc/RKNN_Compiler_Support_Operator_List_v1.5.0.pdf
diff --git a/doc/RKNN_Compiler_Support_Operator_List_v1.5.2.pdf b/doc/RKNN_Compiler_Support_Operator_List_v1.5.2.pdf
diff --git a/doc/RKNN_Dynamic_Shape_Usage.md b/doc/RKNN_Dynamic_Shape_Usage.md
@@ -48,7 +48,7 @@
 
 
 ## 4.设置输入形状
-加载动态形状输入RKNN模型后，您可以在运行时动态修改输入的形状。通过调用rknn_set_input_shape接口，传入包含形状信息的rknn_tensor_attr指针可以设置当前次推理的形状。例如，使用rknn_query获取的输入形状设置输入时，您可以使用以下代码：
+加载动态形状输入RKNN模型后，您可以在运行时动态修改输入的形状。通过调用rknn_set_input_shapes接口，传入所有输入的rknn_tensor_attr数组,可以设置当前次推理的形状。例如，使用rknn_query获取的输入形状设置输入时，您可以使用以下代码：
 
 ```
     for (int s = 0; s < shape_num; ++s)
@@ -59,17 +59,16 @@
             {
                 input_attrs[i].dims[j] = shape_range[i].dyn_range[s][j];
             }
-
-            ret = rknn_set_input_shape(ctx, &input_attrs[i]);
-            if (ret < 0)
-            {
-                fprintf(stderr, "rknn_set_input_shape error! ret=%d\n", ret);
-                return -1;
-            }
+        }
+        ret = rknn_set_input_shapes(ctx, io_num.n_input, input_attrs);
+        if (ret < 0)
+        {
+            fprintf(stderr, "rknn_set_input_shapes error! ret=%d\n", ret);
+            return -1;
         }
     }
 ```
-其中，shape_num是支持的形状个数，shape_range[i]是第i个输入的rknn_input_range结构体，input_attrs[i]是第i个输入的rknn_tensor_attr结构体。
+其中，shape_num是支持的形状个数，shape_range[i]是第i个输入的rknn_input_range结构体，io_num.n_input是输入数量,input_attrs是模型所有输入的rknn_tensor_attr结构体数组。
 
 在设置输入形状后，可以再次调用rknn_query查询当前次推理成功设置后的输入和输出形状，例如，您可以使用以下代码：
 ```

diff --git a/doc/Rockchip_Quick_Start_RKNN_SDK_V1.5.0_CN.pdf b/doc/Rockchip_Quick_Start_RKNN_SDK_V1.5.0_CN.pdf
diff --git a/doc/Rockchip_Quick_Start_RKNN_SDK_V1.5.2_CN.pdf b/doc/Rockchip_Quick_Start_RKNN_SDK_V1.5.2_CN.pdf
diff --git a/doc/Rockchip_Quick_Start_RKNN_SDK_V1.5.2_EN.pdf b/doc/Rockchip_Quick_Start_RKNN_SDK_V1.5.2_EN.pdf
diff --git a/doc/Rockchip_RKNPU_User_Guide_RKNN_API_V1.5.0_CN.pdf b/doc/Rockchip_RKNPU_User_Guide_RKNN_API_V1.5.0_CN.pdf
diff --git a/doc/Rockchip_RKNPU_User_Guide_RKNN_API_V1.5.0_EN.pdf b/doc/Rockchip_RKNPU_User_Guide_RKNN_API_V1.5.0_EN.pdf
diff --git a/doc/Rockchip_RKNPU_User_Guide_RKNN_API_V1.5.2_CN.pdf b/doc/Rockchip_RKNPU_User_Guide_RKNN_API_V1.5.2_CN.pdf
diff --git a/doc/Rockchip_RKNPU_User_Guide_RKNN_API_V1.5.2_EN.pdf b/doc/Rockchip_RKNPU_User_Guide_RKNN_API_V1.5.2_EN.pdf
diff --git a/doc/Rockchip_RV1106_Quick_Start_RKNN_SDK_V1.5.0_CN.pdf b/doc/Rockchip_RV1106_Quick_Start_RKNN_SDK_V1.5.0_CN.pdf
diff --git a/doc/Rockchip_RV1106_Quick_Start_RKNN_SDK_V1.5.2_CN.pdf b/doc/Rockchip_RV1106_Quick_Start_RKNN_SDK_V1.5.2_CN.pdf
diff --git a/doc/Rockchip_RV1106_Quick_Start_RKNN_SDK_V1.5.2_EN.pdf b/doc/Rockchip_RV1106_Quick_Start_RKNN_SDK_V1.5.2_EN.pdf
diff --git a/examples/README.md b/examples/README.md
@@ -1,7 +1,15 @@
-1、目前RV1106/RV1103仅支持rknn_mobilenet_demo 和 rknn_yolov5_demo两个demo，在examples/RV1106_RV1103目录下，demo中的**build脚本用RV1106指代RV1106/RV1103**
 
-2、RK356X和RK3588支持example目录下，除examples/RV1106_RV1103文件夹外的所有demo
 
-3、**RV1106/RV1103设置LD_LIBRARY_PATH必须为全路径**，例如：export LD_LIBRARY_PATH=/userdata/lib
+1. Currently, there are only two demo supported on the RV1106/1103 platform，located under the diretory of examples/RV1106_RV1103. **The RV1106 build script is referring to RV1106/RV1103 in the demo**.
+
+2. For all demos, except for the one under examples/RV1106_RV1103, they are all supported on the platform of RK356X/RK3588.
+
+3. **When setting LD_LIBRARY_PATH on the RV1106 or RV1103**, it must be the full path. For example,
+
+   ```shell
+   export LD_LIBRARY_PATH=/userdata/lib
+   ```
+
+   For RK356X, RK3588, LD_LIBRARY_PATH can be set as either full path or relative path
+
 
-4、RK356X和RK3588设置LD_LIBRARY_PATH为全路径和相对路径均可
diff --git a/examples/README_CN.md b/examples/README_CN.md
@@ -0,0 +1,7 @@
+1、目前RV1106/RV1103仅支持rknn_mobilenet_demo 和 rknn_yolov5_demo两个demo，在examples/RV1106_RV1103目录下，demo中的**build脚本用RV1106指代RV1106/RV1103**
+
+2、RK356X和RK3588支持example目录下，除examples/RV1106_RV1103文件夹外的所有demo
+
+3、**RV1106/RV1103设置LD_LIBRARY_PATH必须为全路径**，例如：export LD_LIBRARY_PATH=/userdata/lib
+
+4、RK356X和RK3588设置LD_LIBRARY_PATH为全路径和相对路径均可
diff --git a/examples/RV1106_RV1103/rknn_mobilenet_demo/README.md b/examples/RV1106_RV1103/rknn_mobilenet_demo/README.md
@@ -1,6 +1,7 @@
-下述<TARGET_PLATFORM>是RV1106或RV1103
+The following <TARGET_PLATFORM> is RV1106 or RV1103
 
-# arm Linux Demo
+
+# Arm Linux Demo
 
 ## build
 

diff --git a/examples/RV1106_RV1103/rknn_mobilenet_demo/README_CN.md b/examples/RV1106_RV1103/rknn_mobilenet_demo/README_CN.md
@@ -0,0 +1,34 @@
+下述<TARGET_PLATFORM>是RV1106或RV1103
+
+# Arm Linux Demo
+
+## 编译
+
+修改目标平台的`build-linux_<TARGET_PLATFORM>.sh`上的`GCC_COMPILER`，
+
+然后执行
+
+```
+./build-linux_<TARGET_PLATFORM>.sh
+```
+
+## 安装
+
+连接设备并将构建输出推送到“/userdata”
+
+```
+adb push install/rknn_mobilenet_demo_Linux /userdata/
+```
+
+## 运行
+
+```
+adb shell
+cd /userdata/rknn_mobilenet_demo_Linux/
+```
+
+```
+export LD_LIBRARY_PATH=./lib
+./rknn_mobilenet_demo model/<TARGET_PLATFORM>/mobilenet_v1.rknn model/dog_224x224.jpg
+```
+
diff --git a/examples/RV1106_RV1103/rknn_yolov5_demo/README.md b/examples/RV1106_RV1103/rknn_yolov5_demo/README.md
@@ -1,39 +1,40 @@
 # Yolo-v5 demo
 
-# 导出rknn模型
+# Export RKNN Model
 
-1. 进到examples/RV1106_RV1103/rknn_yolov5_demo/convert_rknn_demo/yolov5目录下，执行如下命令，可生成rknn模型:
+Please refer https://github.com/airockchip/rknn_model_zoo/tree/main/models/CV/object_detection/yolo
 
-```sh
-   cd examples/RV1106_RV1103/rknn_yolov5_demo/convert_rknn_demo/yolov5
-   python onnx2rknn.py
-```
 
-## arm Linux Demo
 
-### 编译
+## Arm Linux Demo
+
+### Compiling and Building
+
+The 'build-linux_RV1106.sh' can be used for compiling demo for target including RV1106 and RV1103.
 
-RV1106/RV1103编译脚本均为 `build-linux_RV1106.sh`，设置交叉编译器所在目录的路径 `RK_RV1106_TOOLCHAIN`，例如修改成
+Changing the cross compiler path via the setting the `RK_RV1106_TOOLCHAIN`, shown as below:
 
 ```sh
 export RK_RV1106_TOOLCHAIN=~/opts/toolchain/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf
 ```
 
-然后执行：
+then, run the script：
 
 ```sh
 ./build-linux_RV1106.sh
 ```
 
-### 推送执行文件到板子
+Note: The RV1106 and RV1103 requires this 'arm-rockchip830-linux-uclibcgnueabihf' compiler to build the demo or another applications. 
 
-连接板子的usb口到PC,将整个demo目录到 `/userdata`:
+### Push build output files to the board
+
+Connecting the usb port to the PC, and pushing all demo folder to the directory '/userdata':
 
 ```sh
 adb push install/rknn_yolov5_demo_Linux /userdata/
 ```
 
-### 运行
+### Running 
 
 ```sh
 adb shell
@@ -43,4 +44,8 @@ export LD_LIBRARY_PATH=/userdata/rknn_yolov5_demo_Linux/lib
 ./rknn_yolov5_demo model/RV1106/yolov5s-640-640.rknn model/bus.jpg
 ```
 
-Note: LD_LIBRARY_PATH 必须采用全路径
+Note: 
+
+- LD_LIBRARY_PATH must use the full path
+- For performance reasons, the output fmt of the RKNN model is set to **RKNN_QUERY_NATIVE_NHWC_OUTPUT_ATTR** in the demo to obtain better inference performance. At this time, the model output buf is arranged in the order of NHWC. For example, the original shape of the first output is **1,255,80,80**. At this case, the shape output by RKNN is 1,80,80,255. The post-processing in this demo is also optimized and adjusted according to this order.
+
diff --git a/examples/RV1106_RV1103/rknn_yolov5_demo/README_CN.md b/examples/RV1106_RV1103/rknn_yolov5_demo/README_CN.md
@@ -0,0 +1,46 @@
+# Yolo-v5 demo
+
+# 导出rknn模型
+
+请参考 https://github.com/airockchip/rknn_model_zoo/tree/main/models/CV/object_detection/yolo
+
+
+
+## Arm Linux Demo
+
+### 编译
+
+RV1106/RV1103编译脚本均为 `build-linux_RV1106.sh`，设置交叉编译器所在目录的路径 `RK_RV1106_TOOLCHAIN`，例如修改成
+
+```sh
+export RK_RV1106_TOOLCHAIN=~/opts/toolchain/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf
+```
+
+然后执行：
+
+```sh
+./build-linux_RV1106.sh
+```
+
+### 推送执行文件到板子
+
+连接板子的usb口到PC,将整个demo目录到 `/userdata`:
+
+```sh
+adb push install/rknn_yolov5_demo_Linux /userdata/
+```
+
+### 运行
+
+```sh
+adb shell
+cd /userdata/rknn_yolov5_demo_Linux/
+
+export LD_LIBRARY_PATH=/userdata/rknn_yolov5_demo_Linux/lib
+./rknn_yolov5_demo model/RV1106/yolov5s-640-640.rknn model/bus.jpg
+```
+
+Note:
+
+- LD_LIBRARY_PATH 必须采用全路径
+- 基于性能原因，demo中将 RKNN 模型的输出 fmt 设置为 RKNN_QUERY_NATIVE_NHWC_OUTPUT_ATTR，以获取更好的推理性能。此时模型输出 buf 是以 NHWC 顺序进行排布的，比如第一个输出的原始 shape 是 1,255,80,80，此时RKNN输出的 shape 是1,80,80,255，此demo中的后处理也根据这个顺序做了相应的优化调整。
diff --git a/examples/RV1106_RV1103/rknn_yolov5_demo/convert_rknn_demo/yolov5/bus.jpg b/examples/RV1106_RV1103/rknn_yolov5_demo/convert_rknn_demo/yolov5/bus.jpg
diff --git a/examples/RV1106_RV1103/rknn_yolov5_demo/convert_rknn_demo/yolov5/dataset.txt b/examples/RV1106_RV1103/rknn_yolov5_demo/convert_rknn_demo/yolov5/dataset.txt
diff --git a/examples/RV1106_RV1103/rknn_yolov5_demo/convert_rknn_demo/yolov5/onnx2rknn.py b/examples/RV1106_RV1103/rknn_yolov5_demo/convert_rknn_demo/yolov5/onnx2rknn.py
diff --git a/examples/RV1106_RV1103/rknn_yolov5_demo/convert_rknn_demo/yolov5/onnx_models/yolov5s.onnx b/examples/RV1106_RV1103/rknn_yolov5_demo/convert_rknn_demo/yolov5/onnx_models/yolov5s.onnx
diff --git a/examples/RV1106_RV1103/rknn_yolov5_demo/model/RV1106/yolov5s-640-640.rknn b/examples/RV1106_RV1103/rknn_yolov5_demo/model/RV1106/yolov5s-640-640.rknn