PaddleTRT无法执行PaddleOCR量化后模型

[wanghaoshuang]

No description provided.

[wanghaoshuang]

@yukavio

[LDOUBLEV]

问题

1. TRT对OCR的模型加速不明显

如下图：

2. 开启TRT加速后，python端预测无法加载量化模型

报错信息如下：

W1026 11:07:49.388718 25396 analysis_predictor.cc:195] WARNING: Results may be DIFF! Please use the corresponding version of the model and prediction library, and do not use the develop branch.
I1026 11:07:49.389149 25396 analysis_predictor.cc:449] TensorRT subgraph engine is enabled
--- Running analysis [ir_graph_build_pass]
--- Running analysis [ir_graph_clean_pass]
--- Running analysis [ir_analysis_pass]
--- Running IR pass [conv_affine_channel_fuse_pass]
--- Running IR pass [conv_eltwiseadd_affine_channel_fuse_pass]
--- Running IR pass [shuffle_channel_detect_pass]
--- Running IR pass [quant_conv2d_dequant_fuse_pass]
I1026 11:07:49.451793 25396 graph_pattern_detector.cc:100] ---  detected 57 subgraphs
I1026 11:07:49.469231 25396 graph_pattern_detector.cc:100] ---  detected 11 subgraphs
I1026 11:07:49.479030 25396 graph_pattern_detector.cc:100] ---  detected 5 subgraphs


--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
0   std::unique_ptr<paddle::PaddlePredictor, std::default_delete<paddle::PaddlePredictor> > paddle::CreatePaddlePredictor<paddle::AnalysisConfig>(paddle::AnalysisConfig const&)
1   std::unique_ptr<paddle::PaddlePredictor, std::default_delete<paddle::PaddlePredictor> > paddle::CreatePaddlePredictor<paddle::AnalysisConfig, (paddle::PaddleEngineKind)2>(paddle::AnalysisConfig const&)
2   paddle::AnalysisPredictor::Init(std::shared_ptr<paddle::framework::Scope> const&, std::shared_ptr<paddle::framework::ProgramDesc> const&)
3   paddle::AnalysisPredictor::PrepareProgram(std::shared_ptr<paddle::framework::ProgramDesc> const&)
4   paddle::AnalysisPredictor::OptimizeInferenceProgram()
5   paddle::inference::analysis::Analyzer::RunAnalysis(paddle::inference::analysis::Argument*)
6   paddle::inference::analysis::IrAnalysisPass::RunImpl(paddle::inference::analysis::Argument*)
7   paddle::inference::analysis::IRPassManager::Apply(std::unique_ptr<paddle::framework::ir::Graph, std::default_delete<paddle::framework::ir::Graph> >)
8   paddle::framework::ir::Pass::Apply(paddle::framework::ir::Graph*) const
9   paddle::framework::ir::QuantDequantFusePass::ApplyImpl(paddle::framework::ir::Graph*) const
10  paddle::framework::ir::FuseDequant(paddle::framework::ir::Graph*, paddle::framework::Scope*, std::string const&, std::string const&)
11  paddle::framework::ir::GraphPatternDetector::operator()(paddle::framework::ir::Graph*, std::function<void (std::map<paddle::framework::ir::PDNode*, paddle::framework::ir::Node*, std::less<paddle::framework::ir::PDNode*>, std::allocator<std::pair<paddle::framework::ir::PDNode* const, paddle::framework::ir::Node*> > > const&, paddle::framework::ir::Graph*)>)
12  paddle::framework::SignalHandle(char const*, int)
13  paddle::platform::GetCurrentTraceBackString()

----------------------
Error Message Summary:
----------------------
FatalError: A serious error (Erroneous arithmetic operation) is detected by the operating system. (at /paddle/tensorRT/paddle/paddle/fluid/platform/init.cc:303)
  [TimeInfo: *** Aborted at 1603710469 (unix time) try "date -d @1603710469" if you are using GNU date ***]
  [SignalInfo: *** SIGFPE (@0x7f593dd8d7a7) received by PID 25396 (TID 0x7f599a1cc700) from PID 1037621159 ***]

Floating point exception (core dumped)

环境

• CUDA 10.1

• CUDNN 7.6

• Driver 418.67

• TRT： docker 自带的TensorRT6-cuda10.0-cudnn7.tar.gz 和官网下载的TRT6

• PaddleOCR 模型： 链接中的inference model

• python 端predictor构建过程：

    config = AnalysisConfig(model_file_path, params_file_path)

    if args.use_gpu:
        config.enable_use_gpu(args.gpu_mem, 0)
    else:
        config.disable_gpu()
        config.set_cpu_math_library_num_threads(6)
        if args.enable_mkldnn:
            # cache 10 different shapes for mkldnn to avoid memory leak
            config.set_mkldnn_cache_capacity(10)
            config.enable_mkldnn()

    # config.enable_memory_optim()
    #config.disable_glog_info()
    config.switch_ir_optim(True)
    config.switch_use_feed_fetch_ops(False)
    if args.use_zero_copy_run:
        config.delete_pass("conv_transpose_eltwiseadd_bn_fuse_pass")
        config.switch_use_feed_fetch_ops(False)
    else:
        config.switch_use_feed_fetch_ops(True)
    logger.info("run model !")
    if args.use_tensorrt:
        if mode == "det" or not args.is_quant:
            config.enable_tensorrt_engine(
                precision_mode=AnalysisConfig.Precision.Half
                if args.use_fp16 else AnalysisConfig.Precision.Float32,
                max_batch_size=args.max_batch_size, min_subgraph_size=6)
        elif args.is_quant:
            logger.info("init quant model!")
            config.enable_tensorrt_engine(
                precision_mode=AnalysisConfig.Precision.Int8,
                max_batch_size=args.max_batch_size, min_subgraph_size=100)
    predictor = create_paddle_predictor(config)

[LDOUBLEV]

补充

在release/1.8分支的paddle编译 带TRT的paddle，加载量化模型出现如下报错：

[LDOUBLEV]

问题定位

根据新的报错信息，打印quant_conv2d_dequant_fuse_pass.cc：198行的数据进行观察，发现weight_scale和w_dims[0]不匹配导致报错。

[paddle-bot-old]

Since you haven't replied for more than a year, we have closed this issue/pr.
If the problem is not solved or there is a follow-up one, please reopen it at any time and we will continue to follow up.
由于您超过一年未回复，我们将关闭这个issue/pr。
若问题未解决或有后续问题，请随时重新打开，我们会继续跟进。