To cross-compile this for browsers with emscripten:
- Make sure you have Emscripten installed. On a Mac with Homebrew installed, you can simply run:
brew install emscripten
- Go to the source folder:
cd src
- Run the compiler. To include in plain JS for use in a simple static HTML page:
em++ -O3 -s WASM=1 -msimd128 -s USE_PTHREADS=1 -s PTHREAD_POOL_SIZE=4 -s TOTAL_MEMORY=128MB ./facedetectcnn.cpp ./facedetectcnn-data.cpp ./facedetectcnn-model.cpp -o facedetection.js -s EXPORTED_FUNCTIONS=_malloc,_free,_facedetect_cnn
If you want to include in a React/Vite/Typescript project, the steps are more complicated:
em++ -O3 -s WASM=1 -msimd128 -s USE_PTHREADS=1 -s PTHREAD_POOL_SIZE=4 -s TOTAL_MEMORY=128MB -s MODULARIZE=1 -s NO_DISABLE_EXCEPTION_CATCHING ./facedetectcnn.cpp ./facedetectcnn-data.cpp ./facedetectcnn-model.cpp -o facedetection.js -s EXPORTED_FUNCTIONS=_malloc,_free,_facedetect_cnn
Delete facedetection.worker.js - this is a legacy unused file and it's just a comment anyway.
Add line to top of facedetection.js output:
/ *eslint-disable */
Replace the path to play nice with Vite in facedetection.js:
Search for "facedetection.wasm" and replace with "/wasm/facedetection.wasm"
Delete the exports
if (typeof exports === 'object' && typeof module === 'object')
module.exports = Module;
else if (typeof define === 'function' && define['amd'])
define([], () => Module);
Add this to the bottom of the file:
export default Module;
If that is all very scary to you, go in module folder and take the files already set up to do this for you.
- Take the facedetection.js and facedetection.wasm and put in your project. You can see a sample implementation in https://github.com/yembo/libfacedetection-wasm
Original README.md follows.
This is an open source library for CNN-based face detection in images. The CNN model has been converted to static variables in C source files. The source code does not depend on any other libraries. What you need is just a C++ compiler. You can compile the source code under Windows, Linux, ARM and any platform with a C++ compiler.
SIMD instructions are used to speed up the detection. You can enable AVX2 if you use Intel CPU or NEON for ARM.
The model files are provided in src/facedetectcnn-data.cpp (C++ arrays) & the model (ONNX) from OpenCV Zoo. You can try our scripts (C++ & Python) in opencv_dnn/ with the ONNX model. View the network architecture here.
Please note that OpenCV DNN does not support the latest version of YuNet with dynamic input shape. Please ensure you have the exact same input shape as the one in the ONNX model to run latest YuNet with OpenCV DNN.
examples/detect-image.cpp and examples/detect-camera.cpp show how to use the library.
The library was trained by libfacedetection.train.
You can copy the files in directory src/ into your project, and compile them as the other files in your project. The source code is written in standard C/C++. It should be compiled at any platform which supports C/C++.
Some tips:
- Please add facedetection_export.h file in the position where you copy your facedetectcnn.h files, add #define FACEDETECTION_EXPORT to facedetection_export.h file. See: issues #222
- Please add -O3 to turn on optimizations when you compile the source code using g++.
- Please choose 'Maximize Speed/-O2' when you compile the source code using Microsoft Visual Studio.
- You can enable OpenMP to speedup. But the best solution is to call the detection function in different threads.
You can also compile the source code to a static or dynamic library, and then use it in your project.
Using AVX2 instructions
| Method | Time | FPS | Time | FPS |
|---|---|---|---|---|
| X64 | X64 | X64 | X64 | |
| Single-thread | Single-thread | Multi-thread | Multi-thread | |
| cnn (CPU, 640x480) | 50.02ms | 19.99 | 6.55ms | 152.65 |
| cnn (CPU, 320x240) | 13.09ms | 76.39 | 1.82ms | 550.54 |
| cnn (CPU, 160x120) | 3.61ms | 277.37 | 0.57ms | 1745.13 |
| cnn (CPU, 128x96) | 2.11ms | 474.60 | 0.33ms | 2994.23 |
Using AVX512 instructions
| Method | Time | FPS | Time | FPS |
|---|---|---|---|---|
| X64 | X64 | X64 | X64 | |
| Single-thread | Single-thread | Multi-thread | Multi-thread | |
| cnn (CPU, 640x480) | 46.47ms | 21.52 | 6.39ms | 156.47 |
| cnn (CPU, 320x240) | 12.10ms | 82.67 | 1.67ms | 599.31 |
| cnn (CPU, 160x120) | 3.37ms | 296.47 | 0.46ms | 2155.80 |
| cnn (CPU, 128x96) | 1.98ms | 504.72 | 0.31ms | 3198.63 |
- Minimal face size ~10x10
- Intel(R) Core(TM) i7-7820X CPU @ 3.60GHz
- Multi-thread in 16 threads and 16 processors.
| Method | Time | FPS | Time | FPS |
|---|---|---|---|---|
| Single-thread | Single-thread | Multi-thread | Multi-thread | |
| cnn (CPU, 640x480) | 404.63ms | 2.47 | 125.47ms | 7.97 |
| cnn (CPU, 320x240) | 105.73ms | 9.46 | 32.98ms | 30.32 |
| cnn (CPU, 160x120) | 26.05ms | 38.38 | 7.91ms | 126.49 |
| cnn (CPU, 128x96) | 15.06ms | 66.38 | 4.50ms | 222.28 |
- Minimal face size ~10x10
- Raspberry Pi 4 B, Broadcom BCM2835, Cortex-A72 (ARMv8) 64-bit SoC @ 1.5GHz
- Multi-thread in 4 threads and 4 processors.
Run on default settings: scales=[1.], confidence_threshold=0.02, floating point:
AP_easy=0.887, AP_medium=0.871, AP_hard=0.768
- Shiqi Yu, shiqi.yu@gmail.com
All contributors who contribute at GitHub.com are listed here.
The contributors who were not listed at GitHub.com:
- Jia Wu (吴佳)
- Dong Xu (徐栋)
- Shengyin Wu (伍圣寅)
The work was partly supported by the Science Foundation of Shenzhen (Grant No. 20170504160426188).
The master thesis of Mr. Wei Wu. All details of the algorithm are in the thesis. The thesis can be downloaded at 吴伟硕士毕业论文
@thesis{wu2023thesisyunet,
author = {吴伟},
title = {面向边缘设备的高精度毫秒级人脸检测技术研究},
type = {硕士学位论文},
institution = {南方科技大学},
year = {2023},
}
The paper for the main idea of this repository https://link.springer.com/article/10.1007/s11633-023-1423-y.
@article{wu2023miryunet,
title = {YuNet: A Tiny Millisecond-level Face Detector},
author = {Wu, Wei and Peng, Hanyang and Yu, Shiqi},
journal = {Machine Intelligence Research},
pages = {1--10},
year = {2023},
doi = {10.1007/s11633-023-1423-y},
publisher = {Springer}
}
The survey paper on face detection to evaluate different methods. It can be open-accessed at https://ieeexplore.ieee.org/document/9580485
@article{feng2022face,
author = {Feng, Yuantao and Yu, Shiqi and Peng, Hanyang and Li, Yan-Ran and Zhang, Jianguo},
journal = {IEEE Transactions on Biometrics, Behavior, and Identity Science},
title = {Detect Faces Efficiently: A Survey and Evaluations},
year = {2022},
volume = {4},
number = {1},
pages = {1-18},
doi = {10.1109/TBIOM.2021.3120412}
}
The loss used in training is EIoU, a novel extended IoU. The paper can be open-accessed at https://ieeexplore.ieee.org/document/9429909.
@article{peng2021eiou,
author = {Peng, Hanyang and Yu, Shiqi},
journal = {IEEE Transactions on Image Processing},
title = {A Systematic IoU-Related Method: Beyond Simplified Regression for Better Localization},
year = {2021},
volume = {30},
pages = {5032-5044},
doi = {10.1109/TIP.2021.3077144}
}