[Model] Add Silero VAD example #1107

chenqianhe · 2023-01-10T09:39:34Z

PR types(PR类型)

Model

Description

增加 VAD 模型示例；支持cpp调用。
该模型主要用于检测语音的中无声部分和speak部分，并输出speak的时间段。
模型来源和详细介绍见：https://github.com/snakers4/silero-vad

该模型主要应用场景为端侧使用，且推理速度极快，一个音频窗口推理基本不超过1ms，因此仅实现了cpp推理。

…to develop

DefTruth

感谢您的贡献！可以第一次review的意见修改一下哈，等所有修改意见完成后，我们才能合入

examples/audio/silero-vad/cpp/Vad.cc

examples/audio/silero-vad/cpp/wav.h

examples/audio/silero-vad/cpp/Vad.h

examples/audio/silero-vad/cpp/wav.h

examples/audio/silero-vad/cpp/Vad.cc

examples/audio/silero-vad/cpp/README.md

examples/audio/silero-vad/cpp/README_CN.md

chenqianhe · 2023-01-11T08:28:51Z

@DefTruth 确认一个关于 wav.h 的整体问题哈，这个属于第三方的，也要去同步修改格式问题吗

DefTruth · 2023-01-11T08:36:56Z

意思是这是一个从开源库里引用的是吗？可以注明下出处。如果影响使用的话，格式可以先不改。先修改其他的。

chenqianhe · 2023-01-11T08:38:18Z

Reference in new

有标注的来源的，在对应文件最开头就有声明

DefTruth · 2023-01-11T08:40:50Z

Reference in new

有标注的来源的，在对应文件最开头就有声明

好的

chenqianhe · 2023-01-11T12:31:47Z

model_and_example.zip

chenqianhe · 2023-01-11T13:25:38Z

@DefTruth 大部分已经修改完成，但是还有两个点不太明确。
一是成员变量末尾使用下划线进行标记。这个是所有成员变量都需要这样吗，区分 public 和 private 吗？
二是需要增加函数注释，说明函数的作用。目前来看，成员函数主要是两类，一类是getter，一类是功能实现函数，根据名称其实都比较明显能看出来作用，并且文档也给了所有功能函数的说明和参数解释。因此能否明确一下哪些需要补充函数注释，getter应该都不用了吧。

DefTruth · 2023-01-11T15:29:44Z

@DefTruth 大部分已经修改完成，但是还有两个点不太明确。一是成员变量末尾使用下划线进行标记。这个是所有成员变量都需要这样吗，区分 public 和 private 吗？二是需要增加函数注释，说明函数的作用。目前来看，成员函数主要是两类，一类是getter，一类是功能实现函数，根据名称其实都比较明显能看出来作用，并且文档也给了所有功能函数的说明和参数解释。因此能否明确一下哪些需要补充函数注释，getter应该都不用了吧。

关于一：所有成员都加下滑线吧，pubilc成员通过getter和setter获取或设置。关于二：作用明显的函数不加注释，这样处理也没问题。后续我们继续review下哈，感谢您的贡献~

…to develop

chenqianhe · 2023-01-12T02:21:39Z

@DefTruth 大部分已经修改完成，但是还有两个点不太明确。一是成员变量末尾使用下划线进行标记。这个是所有成员变量都需要这样吗，区分 public 和 private 吗？二是需要增加函数注释，说明函数的作用。目前来看，成员函数主要是两类，一类是getter，一类是功能实现函数，根据名称其实都比较明显能看出来作用，并且文档也给了所有功能函数的说明和参数解释。因此能否明确一下哪些需要补充函数注释，getter应该都不用了吧。

关于一：所有成员都加下滑线吧，pubilc成员通过getter和setter获取或设置。关于二：作用明显的函数不加注释，这样处理也没问题。后续我们继续review下哈，感谢您的贡献~

@DefTruth done

DefTruth · 2023-01-12T03:33:13Z

@chenqianhe 这个wav.h的代码风格的CI一直没过，看看有没有可能在不影响使用的基础上修改？可以直接运行commit-prepare.sh后，再提交commit，应该会自动修正一些代码风格

…to develop

chenqianhe · 2023-01-12T03:50:12Z

@chenqianhe 这个wav.h的代码风格的CI一直没过，看看有没有可能在不影响使用的基础上修改？可以直接运行commit-prepare.sh后，再提交commit，应该会自动修正一些代码风格

@DefTruth 本地已完成fix

DefTruth · 2023-01-12T05:10:12Z

@chenqianhe 这个wav.h的代码风格的CI一直没过，看看有没有可能在不影响使用的基础上修改？可以直接运行commit-prepare.sh后，再提交commit，应该会自动修正一些代码风格

@DefTruth 本地已完成fix

LSTM

DefTruth · 2023-01-12T05:19:17Z

@chenqianhe 模型和测试语音文件已经上传到以下地址：

https://bj.bcebos.com/paddlehub/fastdeploy/silero_vad.tgz
https://bj.bcebos.com/paddlehub/fastdeploy/silero_vad_sample.wav

请测试后，参考examples/vision/detection/yolov5的文档补充VAD的文档（提供可下载的模型链接，以及增加该开源库的开源协议说明）

对应的推理案例，需要修改成从链接下载模型和数据，然后infer_xxx model data 的方式

chenqianhe · 2023-01-12T05:29:23Z

这个我看过，主要是因为这个模型应用场景不会存在这种 infer_xxx model data 来直接获取输出的（比如获取分类结果就结束了），基本是一定需要后接其他处理的，这个模型主要是作为完整程序使用的一部分，因此我的示例是将 model 和 data 编码到 infer_xxx 中的。看看这里有必要改成支持命令行参数的吗

DefTruth · 2023-01-12T05:56:52Z

这个我看过，主要是因为这个模型应用场景不会存在这种 infer_xxx model data 来直接获取输出的（比如获取分类结果就结束了），基本是一定需要后接其他处理的，这个模型主要是作为完整程序使用的一部分，因此我的示例是将 model 和 data 编码到 infer_xxx 中的。看看这里有必要改成支持命令行参数的吗

如果不熟悉gflags的话，可以先参考以下这个案例的方式：

https://github.com/PaddlePaddle/FastDeploy/blob/release/1.0.2/examples/vision/facedet/yolov5face/cpp/infer.cc

但整体上，我们还是保持：用户下载模型和案例文件 -> 编译demo -> 执行demo，并输入模型文件和数据这样统一的方式

examples/audio/silero-vad/cpp/vad.h

chenqianhe · 2023-01-12T06:41:42Z

https://bj.bcebos.com/paddlehub/fastdeploy/silero_vad.tgz
https://bj.bcebos.com/paddlehub/fastdeploy/silero_vad_sample.wav

done

chenqianhe · 2023-01-12T06:51:51Z

@chenqianhe 模型和测试语音文件已经上传到以下地址：
https://bj.bcebos.com/paddlehub/fastdeploy/silero_vad.tgz
https://bj.bcebos.com/paddlehub/fastdeploy/silero_vad_sample.wav
请测试后，参考examples/vision/detection/yolov5的文档补充VAD的文档（提供可下载的模型链接，以及增加该开源库的开源协议说明）

对应的推理案例，需要修改成从链接下载模型和数据，然后infer_xxx model data 的方式

done

DefTruth

还有些小问题需要修改，其他的暂时没有问题了。很赞的FD使用示例，后续我们会考虑将该案例移植到主库或移动端，感谢贡献 ~

examples/audio/silero-vad/cpp/README.md

examples/audio/silero-vad/cpp/README_CN.md

chenqianhe · 2023-01-13T03:43:57Z

还有些小问题需要修改，其他的暂时没有问题了。很赞的FD使用示例，后续我们会考虑将该案例移植到主库或移动端，感谢贡献 ~

@DefTruth done

* [Model] Support Insightface model inferenceing on RKNPU (PaddlePaddle#1113) * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * Update issues.md * Update fastdeploy_init.sh * 更新交叉编译 * 更新insightface系列模型的rknpu2支持 * 更新insightface系列模型的rknpu2支持 * 更新说明文档 * 更新insightface * 尝试解决pybind问题 Co-authored-by: Jason <928090362@qq.com> Co-authored-by: Jason <jiangjiajun@baidu.com> * [Other] Add Function For Aligning Face With Five Points (PaddlePaddle#1124) * 更新5点人脸对齐的代码 * 更新代码格式 * 解决comment * update example * 更新注释 Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> * [Lite] Support PaddleYOLOv8 with Lite Backend (PaddlePaddle#1145) * [Model] Support PaddleYOLOv8 model * [YOLOv8] Add PaddleYOLOv8 pybind * [Other] update from latest develop (#30) * [Backend] Remove all lite options in RuntimeOption (PaddlePaddle#1109) * Remove all lite options in RuntimeOption * Fix code error * move pybind * Fix build error * [Backend] Add TensorRT FP16 support for AdaptivePool2d (PaddlePaddle#1116) * add fp16 cuda kernel * fix code bug * update code * [Doc] Fix KunlunXin doc (PaddlePaddle#1139) fix kunlunxin doc * [Model] Support PaddleYOLOv8 model (PaddlePaddle#1136) Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> * [YOLOv8] add PaddleYOLOv8 pybind11 (PaddlePaddle#1144) (#31) * [Model] Support PaddleYOLOv8 model * [YOLOv8] Add PaddleYOLOv8 pybind * [Other] update from latest develop (#30) * [Backend] Remove all lite options in RuntimeOption (PaddlePaddle#1109) * Remove all lite options in RuntimeOption * Fix code error * move pybind * Fix build error * [Backend] Add TensorRT FP16 support for AdaptivePool2d (PaddlePaddle#1116) * add fp16 cuda kernel * fix code bug * update code * [Doc] Fix KunlunXin doc (PaddlePaddle#1139) fix kunlunxin doc * [Model] Support PaddleYOLOv8 model (PaddlePaddle#1136) Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> * [benchmark] add PaddleYOLOv8 -> benchmark * [benchmark] add PaddleYOLOv8 -> benchmark * [Lite] Support PaddleYOLOv8 with Lite Backend Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> * [Model] Add Silero VAD example (PaddlePaddle#1107) * add vad example * fix typo * fix typo * rename file * remove model and wav * delete Vad.cc * delete Vad.h * rename and format * fix max and min * update readme * rename var * format * add params * update readme * update readme * Update README.md * Update README_CN.md Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: Zheng-Bicheng <58363586+Zheng-Bicheng@users.noreply.github.com> Co-authored-by: Jason <928090362@qq.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> Co-authored-by: Qianhe Chen <54462604+chenqianhe@users.noreply.github.com>

* [Model] Support PaddleYOLOv8 model * [YOLOv8] Add PaddleYOLOv8 pybind * [Other] update from latest develop (#30) * [Backend] Remove all lite options in RuntimeOption (#1109) * Remove all lite options in RuntimeOption * Fix code error * move pybind * Fix build error * [Backend] Add TensorRT FP16 support for AdaptivePool2d (#1116) * add fp16 cuda kernel * fix code bug * update code * [Doc] Fix KunlunXin doc (#1139) fix kunlunxin doc * [Model] Support PaddleYOLOv8 model (#1136) Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> * [YOLOv8] add PaddleYOLOv8 pybind11 (#1144) (#31) * [Model] Support PaddleYOLOv8 model * [YOLOv8] Add PaddleYOLOv8 pybind * [Other] update from latest develop (#30) * [Backend] Remove all lite options in RuntimeOption (#1109) * Remove all lite options in RuntimeOption * Fix code error * move pybind * Fix build error * [Backend] Add TensorRT FP16 support for AdaptivePool2d (#1116) * add fp16 cuda kernel * fix code bug * update code * [Doc] Fix KunlunXin doc (#1139) fix kunlunxin doc * [Model] Support PaddleYOLOv8 model (#1136) Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> * [benchmark] add PaddleYOLOv8 -> benchmark * [benchmark] add PaddleYOLOv8 -> benchmark * [Lite] Support PaddleYOLOv8 with Lite Backend * [Pick] Update from latest develop (#32) * [Model] Support Insightface model inferenceing on RKNPU (#1113) * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * 更新交叉编译 * Update issues.md * Update fastdeploy_init.sh * 更新交叉编译 * 更新insightface系列模型的rknpu2支持 * 更新insightface系列模型的rknpu2支持 * 更新说明文档 * 更新insightface * 尝试解决pybind问题 Co-authored-by: Jason <928090362@qq.com> Co-authored-by: Jason <jiangjiajun@baidu.com> * [Other] Add Function For Aligning Face With Five Points (#1124) * 更新5点人脸对齐的代码 * 更新代码格式 * 解决comment * update example * 更新注释 Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> * [Lite] Support PaddleYOLOv8 with Lite Backend (#1145) * [Model] Support PaddleYOLOv8 model * [YOLOv8] Add PaddleYOLOv8 pybind * [Other] update from latest develop (#30) * [Backend] Remove all lite options in RuntimeOption (#1109) * Remove all lite options in RuntimeOption * Fix code error * move pybind * Fix build error * [Backend] Add TensorRT FP16 support for AdaptivePool2d (#1116) * add fp16 cuda kernel * fix code bug * update code * [Doc] Fix KunlunXin doc (#1139) fix kunlunxin doc * [Model] Support PaddleYOLOv8 model (#1136) Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> * [YOLOv8] add PaddleYOLOv8 pybind11 (#1144) (#31) * [Model] Support PaddleYOLOv8 model * [YOLOv8] Add PaddleYOLOv8 pybind * [Other] update from latest develop (#30) * [Backend] Remove all lite options in RuntimeOption (#1109) * Remove all lite options in RuntimeOption * Fix code error * move pybind * Fix build error * [Backend] Add TensorRT FP16 support for AdaptivePool2d (#1116) * add fp16 cuda kernel * fix code bug * update code * [Doc] Fix KunlunXin doc (#1139) fix kunlunxin doc * [Model] Support PaddleYOLOv8 model (#1136) Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> * [benchmark] add PaddleYOLOv8 -> benchmark * [benchmark] add PaddleYOLOv8 -> benchmark * [Lite] Support PaddleYOLOv8 with Lite Backend Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> * [Model] Add Silero VAD example (#1107) * add vad example * fix typo * fix typo * rename file * remove model and wav * delete Vad.cc * delete Vad.h * rename and format * fix max and min * update readme * rename var * format * add params * update readme * update readme * Update README.md * Update README_CN.md Co-authored-by: DefTruth <31974251+DefTruth@users.noreply.github.com> Co-authored-by: Zheng-Bicheng <58363586+Zheng-Bicheng@users.noreply.github.com> Co-authored-by: Jason <928090362@qq.com> Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> Co-authored-by: Qianhe Chen <54462604+chenqianhe@users.noreply.github.com> * [YOLOv8] Support PaddleYOLOv8 on Kunlunxin&Ascend * [YOLOv8] Add PaddleYOLOv8 model download links * [YOLOv8] Add PaddleYOLOv8 Box AP Co-authored-by: Jason <jiangjiajun@baidu.com> Co-authored-by: yeliang2258 <30516196+yeliang2258@users.noreply.github.com> Co-authored-by: Zheng-Bicheng <58363586+Zheng-Bicheng@users.noreply.github.com> Co-authored-by: Jason <928090362@qq.com> Co-authored-by: Qianhe Chen <54462604+chenqianhe@users.noreply.github.com>

chenqianhe added 3 commits January 10, 2023 17:22

add vad example

47d34b8

Merge branch 'develop' of https://github.com/chenqianhe/FastDeploy in…

180a088

…to develop

fix typo

b9d6b25

chenqianhe force-pushed the develop branch from 5ea3c1d to b9d6b25 Compare January 10, 2023 09:40

jiangjiajun requested a review from DefTruth January 10, 2023 11:18

chenqianhe and others added 2 commits January 10, 2023 22:33

Merge branch 'develop' into develop

cf482b6

Merge branch 'develop' into develop

82f4268

DefTruth requested changes Jan 11, 2023

View reviewed changes

fix typo

af0d4f1

chenqianhe and others added 2 commits January 11, 2023 17:21

rename file

356bbf4

Merge branch 'PaddlePaddle:develop' into develop

8071ed3

chenqianhe and others added 6 commits January 11, 2023 20:32

remove model and wav

62a9c1b

delete Vad.cc

35bde94

delete Vad.h

ae0ed89

rename and format

92dfd56

fix max and min

7d733a1

update readme

fdee845

Merge branch 'develop' into develop

230caa3

chenqianhe added 2 commits January 12, 2023 10:20

rename var

3201b1f

Merge branch 'develop' of https://github.com/chenqianhe/FastDeploy in…

fb2d028

…to develop

Merge branch 'develop' into develop

5d9e145

chenqianhe added 2 commits January 12, 2023 11:48

format

326ed7e

Merge branch 'develop' of https://github.com/chenqianhe/FastDeploy in…

253aa3b

…to develop

chenqianhe requested a review from DefTruth January 12, 2023 05:00

DefTruth reviewed Jan 12, 2023

View reviewed changes

examples/audio/silero-vad/cpp/vad.h Show resolved Hide resolved

chenqianhe added 2 commits January 12, 2023 14:38

add params

ca99e77

update readme

60cfb3e

update readme

8cfdbf0

chenqianhe requested a review from DefTruth January 12, 2023 06:52

chenqianhe added 3 commits January 12, 2023 17:17

Merge branch 'develop' into develop

23f6391

Merge branch 'develop' into develop

a4883a3

Merge branch 'develop' into develop

d52d66c

DefTruth requested changes Jan 13, 2023

View reviewed changes

chenqianhe added 2 commits January 13, 2023 11:42

Update README.md

592905d

Update README_CN.md

3d123f3

Merge branch 'develop' into develop

64f5a86

DefTruth requested a review from jiangjiajun January 14, 2023 04:08

DefTruth approved these changes Jan 14, 2023

View reviewed changes

jiangjiajun approved these changes Jan 15, 2023

View reviewed changes

jiangjiajun merged commit a4b94b2 into PaddlePaddle:develop Jan 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Model] Add Silero VAD example #1107

[Model] Add Silero VAD example #1107

chenqianhe commented Jan 10, 2023

DefTruth left a comment

chenqianhe commented Jan 11, 2023

DefTruth commented Jan 11, 2023 •

edited

Loading

chenqianhe commented Jan 11, 2023

DefTruth commented Jan 11, 2023

chenqianhe commented Jan 11, 2023

chenqianhe commented Jan 11, 2023

DefTruth commented Jan 11, 2023

chenqianhe commented Jan 12, 2023

DefTruth commented Jan 12, 2023

chenqianhe commented Jan 12, 2023 •

edited

Loading

DefTruth commented Jan 12, 2023

DefTruth commented Jan 12, 2023

chenqianhe commented Jan 12, 2023

DefTruth commented Jan 12, 2023

chenqianhe commented Jan 12, 2023

chenqianhe commented Jan 12, 2023

DefTruth left a comment

chenqianhe commented Jan 13, 2023

[Model] Add Silero VAD example #1107

[Model] Add Silero VAD example #1107

Conversation

chenqianhe commented Jan 10, 2023

PR types(PR类型)

Description

DefTruth left a comment

Choose a reason for hiding this comment

chenqianhe commented Jan 11, 2023

DefTruth commented Jan 11, 2023 • edited Loading

chenqianhe commented Jan 11, 2023

DefTruth commented Jan 11, 2023

chenqianhe commented Jan 11, 2023

chenqianhe commented Jan 11, 2023

DefTruth commented Jan 11, 2023

chenqianhe commented Jan 12, 2023

DefTruth commented Jan 12, 2023

chenqianhe commented Jan 12, 2023 • edited Loading

DefTruth commented Jan 12, 2023

DefTruth commented Jan 12, 2023

chenqianhe commented Jan 12, 2023

DefTruth commented Jan 12, 2023

chenqianhe commented Jan 12, 2023

chenqianhe commented Jan 12, 2023

DefTruth left a comment

Choose a reason for hiding this comment

chenqianhe commented Jan 13, 2023

DefTruth commented Jan 11, 2023 •

edited

Loading

chenqianhe commented Jan 12, 2023 •

edited

Loading