Add PPMatting Support (PaddlePaddle#163)

* first commit for yolov7 * pybind for yolov7 * CPP README.md * CPP README.md * modified yolov7.cc * README.md * python file modify * delete license in fastdeploy/ * repush the conflict part * README.md modified * README.md modified * file path modified * file path modified * file path modified * file path modified * file path modified * README modified * README modified * move some helpers to private * add examples for yolov7 * api.md modified * api.md modified * api.md modified * YOLOv7 * yolov7 release link * yolov7 release link * yolov7 release link * copyright * change some helpers to private * change variables to const and fix documents. * gitignore * Transfer some funtions to private member of class * Transfer some funtions to private member of class * Merge from develop (#9) * Fix compile problem in different python version (PaddlePaddle#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> * Add PaddleDetetion/PPYOLOE model support (PaddlePaddle#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (PaddlePaddle#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (PaddlePaddle#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> * first commit for yolor * for merge * Develop (PaddlePaddle#11) * Fix compile problem in different python version (PaddlePaddle#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> * Add PaddleDetetion/PPYOLOE model support (PaddlePaddle#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (PaddlePaddle#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (PaddlePaddle#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> * Yolor (PaddlePaddle#16) * Develop (PaddlePaddle#11) (PaddlePaddle#12) * Fix compile problem in different python version (PaddlePaddle#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> * Add PaddleDetetion/PPYOLOE model support (PaddlePaddle#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (PaddlePaddle#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (PaddlePaddle#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> * Develop (PaddlePaddle#13) * Fix compile problem in different python version (PaddlePaddle#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> * Add PaddleDetetion/PPYOLOE model support (PaddlePaddle#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (PaddlePaddle#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (PaddlePaddle#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * Develop (PaddlePaddle#14) * Fix compile problem in different python version (PaddlePaddle#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> * Add PaddleDetetion/PPYOLOE model support (PaddlePaddle#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (PaddlePaddle#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (PaddlePaddle#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> Co-authored-by: Jason <928090362@qq.com> * add is_dynamic for YOLO series (PaddlePaddle#22) * ppmat * first commit for ppmatting * keep VisMattingAlpha with background funtion * support ppmatting and swapbackground function * modify docs error * add photos for docs * add photos for docs * add photos for docs * modify docs * modify docs * modify docs * change remove_small_connected_area function, modify docs * fix compile error Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: root <root@bjyz-sys-gpu-kongming3.bjyz.baidu.com> Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: huangjianhui <852142024@qq.com> Co-authored-by: Jason <928090362@qq.com>
felixhjh · Aug 31, 2022 · f0dbd13 · f0dbd13
1 parent 4d275b3
commit f0dbd13
Show file tree

Hide file tree

Showing 31 changed files with 1,316 additions and 88 deletions.
diff --git a/.gitignore b/.gitignore
@@ -18,4 +18,4 @@ fastdeploy/ThirdPartyNotices*
 *.so*
 fastdeploy/libs/third_libs
 csrc/fastdeploy/core/config.h
-csrc/fastdeploy/pybind/main.cc
+csrc/fastdeploy/pybind/main.cc
diff --git a/csrc/fastdeploy/vision.h b/csrc/fastdeploy/vision.h
@@ -37,6 +37,7 @@
 #include "fastdeploy/vision/faceid/contrib/partial_fc.h"
 #include "fastdeploy/vision/faceid/contrib/vpl.h"
 #include "fastdeploy/vision/matting/contrib/modnet.h"
+#include "fastdeploy/vision/matting/ppmatting/ppmatting.h"
 #include "fastdeploy/vision/ocr/ppocr/classifier.h"
 #include "fastdeploy/vision/ocr/ppocr/dbdetector.h"
 #include "fastdeploy/vision/ocr/ppocr/ppocr_system_v2.h"

diff --git a/csrc/fastdeploy/vision/common/processors/limit_short.cc b/csrc/fastdeploy/vision/common/processors/limit_short.cc
@@ -0,0 +1,70 @@
+// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "fastdeploy/vision/common/processors/limit_short.h"
+
+namespace fastdeploy {
+namespace vision {
+
+bool LimitShort::CpuRun(Mat* mat) {
+  cv::Mat* im = mat->GetCpuMat();
+  int origin_w = im->cols;
+  int origin_h = im->rows;
+  int im_size_min = std::min(origin_w, origin_h);
+  int target = im_size_min;
+  if (max_short_ > 0 && im_size_min > max_short_) {
+    target = max_short_;
+  } else if (min_short_ > 0 && im_size_min < min_short_) {
+    target = min_short_;
+  }
+  if (target != im_size_min) {
+    double scale =
+        static_cast<double>(target) / static_cast<double>(im_size_min);
+    cv::resize(*im, *im, cv::Size(), scale, scale, interp_);
+    mat->SetWidth(im->cols);
+    mat->SetHeight(im->rows);
+  }
+  return true;
+}
+
+#ifdef ENABLE_OPENCV_CUDA
+bool LimitShort::GpuRun(Mat* mat) {
+  cv::cuda::GpuMat* im = mat->GetGpuMat();
+  int origin_w = im->cols;
+  int origin_h = im->rows;
+  im->convertTo(*im, CV_32FC(im->channels()));
+  int im_size_min = std::min(origin_w, origin_h);
+  int target = im_size_min;
+  if (max_short_ > 0 && im_size_min > max_short_) {
+    target = max_short_;
+  } else if (min_short_ > 0 && im_size_min < min_short_) {
+    target = min_short_;
+  }
+  if (target != im_size_min) {
+    double scale =
+        static_cast<double>(target) / static_cast<double>(im_size_min);
+    cv::cuda::resize(*im, *im, cv::Size(), scale, scale, interp_);
+    mat->SetWidth(im->cols);
+    mat->SetHeight(im->rows);
+  }
+  return true;
+}
+#endif
+
+bool LimitShort::Run(Mat* mat, int max_short, int min_short, ProcLib lib) {
+  auto l = LimitShort(max_short, min_short);
+  return l(mat, lib);
+}
+}  // namespace vision
+}  // namespace fastdeploy
diff --git a/csrc/fastdeploy/vision/common/processors/limit_short.h b/csrc/fastdeploy/vision/common/processors/limit_short.h
@@ -0,0 +1,44 @@
+// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#pragma once
+
+#include "fastdeploy/vision/common/processors/base.h"
+
+namespace fastdeploy {
+namespace vision {
+
+class LimitShort : public Processor {
+ public:
+  explicit LimitShort(int max_short = -1, int min_short = -1, int interp = 1) {
+    max_short_ = max_short;
+    min_short_ = min_short;
+    interp_ = interp;
+  }
+  bool CpuRun(Mat* mat);
+#ifdef ENABLE_OPENCV_CUDA
+  bool GpuRun(Mat* mat);
+#endif
+  std::string Name() { return "LimitShort"; }
+
+  static bool Run(Mat* mat, int max_short = -1, int min_short = -1,
+                  ProcLib lib = ProcLib::OPENCV_CPU);
+
+ private:
+  int max_short_;
+  int min_short_;
+  int interp_;
+};
+}  // namespace vision
+}  // namespace fastdeploy
diff --git a/csrc/fastdeploy/vision/common/processors/resize_to_int_mult.cc b/csrc/fastdeploy/vision/common/processors/resize_to_int_mult.cc
@@ -0,0 +1,56 @@
+// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "fastdeploy/vision/common/processors/resize_to_int_mult.h"
+
+namespace fastdeploy {
+namespace vision {
+
+bool ResizeToIntMult::CpuRun(Mat* mat) {
+  cv::Mat* im = mat->GetCpuMat();
+  int origin_w = im->cols;
+  int origin_h = im->rows;
+  int rw = origin_w - origin_w % mult_int_;
+  int rh = origin_h - origin_h % mult_int_;
+  if (rw != origin_w || rh != origin_w) {
+    cv::resize(*im, *im, cv::Size(rw, rh), 0, 0, interp_);
+    mat->SetWidth(im->cols);
+    mat->SetHeight(im->rows);
+  }
+  return true;
+}
+
+#ifdef ENABLE_OPENCV_CUDA
+bool ResizeToIntMult::GpuRun(Mat* mat) {
+  cv::cuda::GpuMat* im = mat->GetGpuMat();
+  int origin_w = im->cols;
+  int origin_h = im->rows;
+  im->convertTo(*im, CV_32FC(im->channels()));
+  int rw = origin_w - origin_w % mult_int_;
+  int rh = origin_h - origin_h % mult_int_;
+  if (rw != origin_w || rh != origin_w) {
+    cv::cuda::resize(*im, *im, cv::Size(rw, rh), 0, 0, interp_);
+    mat->SetWidth(im->cols);
+    mat->SetHeight(im->rows);
+  }
+  return true;
+}
+#endif
+
+bool ResizeToIntMult::Run(Mat* mat, int mult_int, int interp, ProcLib lib) {
+  auto r = ResizeToIntMult(mult_int, interp);
+  return r(mat, lib);
+}
+}  // namespace vision
+}  // namespace fastdeploy
diff --git a/csrc/fastdeploy/vision/common/processors/resize_to_int_mult.h b/csrc/fastdeploy/vision/common/processors/resize_to_int_mult.h
@@ -0,0 +1,42 @@
+// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#pragma once
+
+#include "fastdeploy/vision/common/processors/base.h"
+
+namespace fastdeploy {
+namespace vision {
+
+class ResizeToIntMult : public Processor {
+ public:
+  explicit ResizeToIntMult(int mult_int = 32, int interp = 1) {
+    mult_int_ = mult_int;
+    interp_ = interp;
+  }
+  bool CpuRun(Mat* mat);
+#ifdef ENABLE_OPENCV_CUDA
+  bool GpuRun(Mat* mat);
+#endif
+  std::string Name() { return "ResizeToIntMult"; }
+
+  static bool Run(Mat* mat, int mult_int = 32, int interp = 1,
+                  ProcLib lib = ProcLib::OPENCV_CPU);
+
+ private:
+  int interp_;
+  int mult_int_;
+};
+}  // namespace vision
+}  // namespace fastdeploy
diff --git a/csrc/fastdeploy/vision/common/processors/transform.h b/csrc/fastdeploy/vision/common/processors/transform.h
@@ -19,9 +19,11 @@
 #include "fastdeploy/vision/common/processors/color_space_convert.h"
 #include "fastdeploy/vision/common/processors/convert.h"
 #include "fastdeploy/vision/common/processors/hwc2chw.h"
+#include "fastdeploy/vision/common/processors/limit_short.h"
 #include "fastdeploy/vision/common/processors/normalize.h"
 #include "fastdeploy/vision/common/processors/pad.h"
 #include "fastdeploy/vision/common/processors/pad_to_size.h"
 #include "fastdeploy/vision/common/processors/resize.h"
 #include "fastdeploy/vision/common/processors/resize_by_short.h"
+#include "fastdeploy/vision/common/processors/resize_to_int_mult.h"
 #include "fastdeploy/vision/common/processors/stride_pad.h"
diff --git a/csrc/fastdeploy/vision/matting/matting_pybind.cc b/csrc/fastdeploy/vision/matting/matting_pybind.cc
@@ -17,10 +17,12 @@
 namespace fastdeploy {
 
 void BindMODNet(pybind11::module& m);
+void BindPPMatting(pybind11::module& m);
 
 void BindMatting(pybind11::module& m) {
   auto matting_module =
       m.def_submodule("matting", "Image object matting models.");
   BindMODNet(matting_module);
+  BindPPMatting(matting_module);
 }
 }  // namespace fastdeploy