Paddle inference raise segmentation fault error for quantized ocr recognition model

[juncaipeng]

Thank you for contributing to PaddlePaddle.
Before submitting the issue, you could search issue in the github in case that th
If there is no solution,please make sure that this is an inference issue including the following details :
System information
-PaddlePaddle version
commit id:72c78e4dbbdf74fe9995d9b0757ca6e5832c79d7
cmake: cmake -DCMAKE_BUILD_TYPE=Release -DWITH_PYTHON=OFF -DWITH_MKL=ON -DWITH_GPU=OFF -DON_INFER=ON ..
-CPU: CPUMKL/MKLDNN
-GPU:
-OS Platform: Ubuntu 16.04.5 LTS gcc (Ubuntu 4.8.5-4ubuntu2) 4.8.5
-Python version: 3.7
Note: You can get most of the information by running summary_env.py.
To Reproduce
Steps to reproduce the behavior
Describe your current behavior
Code to reproduce the issue

Download code and modes(url: https://pan.baidu.com/s/1y8OLGP6WajnSkM_f4_FNKg password: y0qp) and refer to the readme.md.

Other info / logs

[paddle-bot-old]

您好，我们已经收到了您的问题，会安排技术人员在一天之内解答您的疑惑，请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时，您也可以通过查看官网API文档、常见问题、历史Issue、AI社区来寻求解答。祝您生活愉快～

Hi! We've received your issue and please be patient to get responded. The average response time is expected to be with in one day. Please make sure that you have posted enough message to demo your request. You may also check out the API，FAQ，Github Issue and AI community to get the answer.Have a nice day!

[lidanqing-vv]

data and model is downloaded at shanghai-6271-ip:/data/issue_28578

[wojtuss]

@juncaipeng
The issue is caused by the faulty fc_lstm_fuse_pass (some input and output variables are not set properly in the fuse). The pass is executed during transformation of the quant model into int8 model (during saving the int8 model), and is not executed during inference of the fp32, quant or int8 model - that is why the problem occurs only during int8 inference. It is not related to the quantization process nor mkldnn.
I have proposed a fix for the issue: #28709.

[paddle-bot-old]

Are you satisfied with the resolution of your issue?

YES
No

[juncaipeng]

@wojtuss Thank you for solving the segmentation fault.
Compared to the origin model, the results of quantized model is wrong, so please check again.

[wojtuss]

@juncaipeng ,
After disabling quantization but enabling optimization passes (either all or only non-mkldnn) the output I get is:

I1123 08:38:11.626078  7530 model_test.cc:59] input[0]:-0.403922
--
I1123 08:38:11.626096  7530 model_test.cc:60] input[-1]:0
I1123 08:38:11.707707  7530 model_test.cc:79] Threads:1, average cost:15.4ms
I1123 08:38:11.707741  7530 model_test.cc:85] Output save_infer_model/scale_0.tmp_0
I1123 08:38:11.707747  7530 model_test.cc:96] output[0]: 5492
I1123 08:38:11.707753  7530 model_test.cc:96] output[1]: 2696
I1123 08:38:11.707758  7530 model_test.cc:100] Output save_infer_model/scale_1.tmp_0
I1123 08:38:11.709888  7530 model_test.cc:111] output[0]: 1.19614e-09
I1123 08:38:11.709901  7530 model_test.cc:111] output[1]: 2.37613e-10
I1123 08:38:11.709905  7530 model_test.cc:111] output[2]: 3.2994e-11
I1123 08:38:11.709908  7530 model_test.cc:111] output[3]: 1.71248e-12
I1123 08:38:11.709913  7530 model_test.cc:111] output[4]: 9.33774e-13
I1123 08:38:11.709915  7530 model_test.cc:111] output[5]: 8.78324e-10
I1123 08:38:11.709918  7530 model_test.cc:111] output[6]: 1.33439e-08
I1123 08:38:11.709921  7530 model_test.cc:111] output[7]: 3.65979e-11
I1123 08:38:11.709925  7530 model_test.cc:111] output[8]: 8.11972e-10
I1123 08:38:11.709928  7530 model_test.cc:111] output[9]: 4.00257e-11
I1123 08:38:11.716387  7530 model_test.cc:113] max value:1

Is it correct?
It differs from the output of fp32 and quant models, which was:

I1123 08:08:41.678112  6677 model_test.cc:59] input[0]:-0.403922
--
I1123 08:08:41.678129  6677 model_test.cc:60] input[-1]:0
I1123 08:08:42.011580  6677 model_test.cc:79] Threads:1, average cost:127.194ms
I1123 08:08:42.011606  6677 model_test.cc:85] Output save_infer_model/scale_0.tmp_0
I1123 08:08:42.011612  6677 model_test.cc:96] output[0]: 389
I1123 08:08:42.011617  6677 model_test.cc:96] output[1]: 519
I1123 08:08:42.011620  6677 model_test.cc:96] output[2]: 472
I1123 08:08:42.011622  6677 model_test.cc:96] output[3]: 519
I1123 08:08:42.011626  6677 model_test.cc:100] Output save_infer_model/scale_1.tmp_0
I1123 08:08:42.013337  6677 model_test.cc:111] output[0]: 1.87588e-08
I1123 08:08:42.013348  6677 model_test.cc:111] output[1]: 1.32461e-08
I1123 08:08:42.013350  6677 model_test.cc:111] output[2]: 1.21617e-08
I1123 08:08:42.013353  6677 model_test.cc:111] output[3]: 3.32012e-08
I1123 08:08:42.013355  6677 model_test.cc:111] output[4]: 3.81083e-09
I1123 08:08:42.013357  6677 model_test.cc:111] output[5]: 1.23054e-09
I1123 08:08:42.013360  6677 model_test.cc:111] output[6]: 1.43666e-09
I1123 08:08:42.013362  6677 model_test.cc:111] output[7]: 6.18276e-09
I1123 08:08:42.013365  6677 model_test.cc:111] output[8]: 2.0969e-09
I1123 08:08:42.013366  6677 model_test.cc:111] output[9]: 6.70895e-08
I1123 08:08:42.018697  6677 model_test.cc:113] max value:1

[juncaipeng]

@wojtuss Base on your test results, the outputs of opted int8 model is incorrect.
As the outputs of quant model are the same to the fp32 model, I think the MKLDNN inference still has errors.

The OCR model has two outputs, please modify the demo to check more output data.