VIT_OCR performance drop (ERNIE drops also)

[pmajchrzak]

Commit df82fd3 causes performance drop of VIT_OCR model. Especially wisible for bf16 numeric mode.

results:

fp32

bf16

CMAKE parameters:

-DCMAKE_BUILD_TYPE=Release -DWITH_GPU=OFF -DWITH_MKLDNN=ON -DWITH_TESTING=ON -DWITH_PROFILER=OFF -DWITH_STYLE_CHECK=OFF -DON_INFER=ON -DWITH_INFERENCE_API_TEST=ON -DPY_VERSION=3.6

VIT_OCR instalation and run:

VIT-OCR build:

vit-ocr.tar.gz

cd ${VIT_OCR_DIR}
mkdir -p build
cd build
cmake \
  .. \
  -DPADDLE_LIB=${PADDLE_DIR}/build/paddle_inference_install_dir \
  -DWITH_MKL=ON \
  -DDEMO_NAME=trt_test \
  -DWITH_STATIC_LIB=OFF
make -j ${CORES_NUM}

Run command:

FP32

./trt_test --model_file /data/PaddlePaddle/models/vit-ocr/model/inference.pdmodel --params_file /data/PaddlePaddle/models/vit-ocr/model/inference.pdiparams

BF16

./trt_test --model_file /data/PaddlePaddle/models/vit-ocr/model/inference.pdmodel --params_file /data/PaddlePaddle/models/vit-ocr/model/inference.pdiparams --bfloat16

[paddle-bot]

您好，我们已经收到了您的问题，会安排技术人员尽快解答您的问题，请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时，您也可以通过查看官网API文档、常见问题、历史Issue、AI社区来寻求解答。祝您生活愉快～

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the API，FAQ，Github Issue and AI community to get the answer.Have a nice day!

[wangna11BD]

@YuanRisheng

[YuanRisheng]

We will analyze the causes and find ways to reduce the impact on performance. Thank you.

[pmajchrzak]

to information: @jczaja @Silv3S

[pmajchrzak]

The same commit df82fd3 causes also ERNIE's performance drop (Latency rised)