Skip to content

add illustration for annotation format for inference #244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 32 additions & 4 deletions deploy/demo/mindocr_lite/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,20 @@ Download:Please download the [IC15](https://rrc.cvc.uab.es/?ch=4&com=downloads

Conversion:Please refer to the script and format in the [dataset_converters](https://github.com/mindspore-lab/mindocr/tree/main/tools/dataset_converters), convert the test sets of IC5 and SVT into detection and recognition formats.

The annotation format of det_gt.txt for text recognition follows:

```
img_478.jpg [{"transcription": "SPA", "points": [[1136, 36], [1197, 0], [1220, 49], [1145, 96]]}, {...}]
```

The annotation format of rec_gt.txt for text recognition follows:

```
word_421.png UNDER
word_1657.png CANDY
word_1814.png CATHAY
```


### Model Export

Expand Down Expand Up @@ -93,23 +107,37 @@ cd deploy/demo/mindocr_lite
python infer.py --input_images_dir=dataset/ic15/det/test/ch4_test_images --device=Ascend --device_id=3 --parallel_num=1 --precision_mode=fp16 --det_model_path=mindir/dbnet_resnet50.mindir --engine=lite --res_save_dir=det_ic15
```

The results will be saved in det_ic15/det_results.txt.

The results will be saved in det_ic15/det_results.txt, with the following format:
```
img_478.jpg [[[1114, 35], [1200, 0], [1234, 52], [1148, 97]], [...]]
```

2. Text recognition

```shell
python infer.py --input_images_dir=dataset/svt/rec/test/cropped_images --device=Ascend --device_id=3 --parallel_num=1 --precision_mode=fp16 --rec_model_path=mindir/crnn_resnet34.mindir --engine=lite --res_save_dir=rec_svt
```

The results will be saved in rec_svt/rec_results.txt.
The results will be saved in rec_svt/rec_results.txt, with the following format:

```
word_421.png "under"
word_1657.png "candy"
word_1814.png "cathay"
```

3. Text detection + recognition

```shell
python infer.py --input_images_dir=dataset/ic15/det/test/ch4_test_images --device=Ascend --device_id=3 --parallel_num=1 --precision_mode=fp16 --det_model_path=mindir/dbnet_resnet50.mindir --rec_model_path=mindir/crnn_resnet34.mindir --engine=lite --res_save_dir=det_rec_ic15
```

The results will be saved in det_rec_ic15/pipeline_results.txt.
The results will be saved in det_rec_ic15/pipeline_results.txt, with the following format:

```
img_478.jpg [{"transcription": "spa", "points": [[1114, 35], [1200, 0], [1234, 52], [1148, 97]]}, {...}]
```


After the inference is executed, the corresponding inference performance FPS value will be printed.

Expand Down
36 changes: 32 additions & 4 deletions deploy/demo/mindocr_lite/README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,20 @@

格式转换:参考[dataset_converters](https://github.com/mindspore-lab/mindocr/tree/main/tools/dataset_converters) 中的脚本和转换格式,将IC5和SVT的测试集转换为检测和识别的格式。

文本检测的标签文件det_gt.txt格式如下:

```
img_478.jpg [{"transcription": "SPA", "points": [[1136, 36], [1197, 0], [1220, 49], [1145, 96]]}, {...}]
```

文本识别的标签文件rec_gt.txt格式如下:

```
word_421.png UNDER
word_1657.png CANDY
word_1814.png CATHAY
```


### 模型导出

Expand Down Expand Up @@ -94,23 +108,37 @@ cd cd deploy/demo/mindocr_lite
python infer.py --input_images_dir=dataset/ic15/det/test/ch4_test_images --device=Ascend --device_id=3 --parallel_num=1 --precision_mode=fp16 --det_model_path=mindir/dbnet_resnet50.mindir --engine=lite --res_save_dir=det_ic15
```

检测结果保存在det_ic15/det_results.txt.
检测结果保存在det_ic15/det_results.txt, 格式如下:

```
img_478.jpg [[1114, 35], [1200, 0], [1234, 52], [1148, 97]], [...]]
```

2. 识别

```shell
python infer.py --input_images_dir=dataset/svt/rec/test/cropped_images --device=Ascend --device_id=3 --parallel_num=1 --precision_mode=fp16 --rec_model_path=mindir/crnn_resnet34.mindir --engine=lite --res_save_dir=rec_svt
```

识别结果保存在rec_svt/rec_results.txt.
识别结果保存在rec_svt/rec_results.txt,格式如下:

```
word_421.png "under"
word_1657.png "candy"
word_1814.png "cathay"
```

3. 检测+识别

```shell
python infer.py --input_images_dir=dataset/ic15/det/test/ch4_test_images --device=Ascend --device_id=3 --parallel_num=1 --precision_mode=fp16 --det_model_path=mindir/dbnet_resnet50.mindir --rec_model_path=mindir/crnn_resnet34.mindir --engine=lite --res_save_dir=det_rec_ic15
```

端到端的检测识别结果保存在det_rec_ic15/pipeline_results.txt.
端到端的检测识别结果保存在det_rec_ic15/pipeline_results.txt,格式如下:

```
img_478.jpg [{"transcription": "spa", "points": [[1114, 35], [1200, 0], [1234, 52], [1148, 97]]}, {...}]
```

推理执行完毕之后,会打印相应的推理性能FPS值。

Expand All @@ -122,7 +150,7 @@ infer.py推理命令的详细参数列表如下:
| device | 推理设备名称(默认Ascend) | False |
| device_id | 推理设备id(默认0) | False |
| engine | 推理引擎(默认lite) | False |
| parallel_num | 推理流水线中每个节点并行数 | False |
| parallel_num | 推理流水线中每个节点并行数(默认1) | False |
| precision_mode | 推理的精度模式(默认fp32) | False |
| det_algorithm | 文本检测算法名(默认DBNet) | False |
| rec_algorithm | 文字识别算法名(默认CRNN) | False |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,7 @@ def process(self, input_data):
infer_res_list = []
image = input_data.frame
for box in boxes:
points = box.flatten().tolist()
infer_res_list.append(points[:8])
infer_res_list.append(box.tolist())
sub_image = get_rotate_crop_image(image, np.array(box, dtype=np.float32))
sub_image_list.append(sub_image)

Expand Down