diff --git a/docs/MODEL_ZOO.md b/docs/MODEL_ZOO.md index dd1b73b9..2f3bee18 100644 --- a/docs/MODEL_ZOO.md +++ b/docs/MODEL_ZOO.md @@ -35,6 +35,27 @@ python scripts/demo_inference.py --cfg configs/halpe_26/resnet/256x192_res50_lr1 - This model is trained based on the first 26 keypoints of Halpe Full-body datatset (without face and hand keypoints). - The speed is tested on COCO val2017 on a single NVIDIA GeForce RTX 3090 gpu, with `batch_size=64` in each iteration and offline yolov3 human detection results. + +## Multi Domain Models **(Strongly Recommended)** +| Model | Backbone | Detector | Input Size | Loss Type | AP | Speed | Download | Config | #keypoints | +|--------------------------|----------|----------|------------|------------|------------|-------|-----------|--------|--------------| +|[Fast Pose](../configs/halpe_coco_wholebody_136/resnet/256x192_res50_lr1e-3_2x-regression.yaml) | ResNet50 | YOLOv3 | 256x192 | Symmetric Integral | 50.1 | 16.28 iter/s | [Google](https://drive.google.com/file/d/1Bb3kPoFFt-M0Y3ceqNO8DTXi1iNDd4gI/view?usp=sharing) [Baidu(code: d0wi)](https://pan.baidu.com/s/1GaHzMHTqYze2rVn7u1sjVg) | [cfg](../configs/halpe_coco_wholebody_136/resnet/256x192_res50_lr1e-3_2x-regression.yaml) | 136 | +|[Fast Pose (DCN)](../configs/halpe_coco_wholebody_136/resnet/256x192_res50_lr1e-3_2x-dcn-combined.yaml) | ResNet50 - dcn | YOLOv3 | 256x192 | Combined (10 hand weight) | 49.8 | 10.35 iter/s | [Google](https://drive.google.com/file/d/1wX1Z2ZOoysgSNovlgiEtJKpbR8tUBWYR/view?usp=sharing) [Baidu(code: app1)](https://pan.baidu.com/s/1bIro0XfYj0FIVf84QzdDoQ) | [cfg](../configs/halpe_coco_wholebody_136/resnet/256x192_res50_lr1e-3_2x-dcn-combined.yaml) | 136 | +|[Fast Pose (DCN)](../configs/halpe_68_noface/resnet/256x192_res50_lr1e-3_2x-dcn-combined.yaml) | ResNet50 - dcn | YOLOv3 | 256x192 | Combined | - | 13.88 iter/s | [Google](https://drive.google.com/file/d/14Qn9gxm-EVzqFi7v25Y5TqKIvrFLy_BR/view?usp=sharing) [Baidu(code: 6kwr)](https://pan.baidu.com/s/1GLNxN3gfekUVY0HZu41fJQ) | [cfg](../configs/halpe_68_noface/resnet/256x192_res50_lr1e-3_2x-dcn-combined.yaml) | 68 (no face) | +|[Fast Pose (DCN)](../configs/single_hand/resnet/256x192_res50_lr1e-3_2x-dcn-regression.yaml) | ResNet50 - dcn | - | 256x192 | Symmetric Integral | - | 30.20 iter/s | [Google](https://drive.google.com/file/d/1MntndimlUP5Hxef1UN9ZDMBVglfA606J/view?usp=sharing) [Baidu(code: nwxx)](https://pan.baidu.com/s/1OR-uH25MFQ7kY8Gt_aJfbw ) | [cfg](../configs/single_hand/resnet/256x192_res50_lr1e-3_2x-dcn-regression.yaml) | 21 (single hand) | + +For the most accurate wholebody pose estimation, you can run with: +``` +python scripts/demo_inference.py --cfg configs/halpe_coco_wholebody_136/resnet/256x192_res50_lr1e-3_2x-dcn-combined.yaml --checkpoint pretrained_models/multi_domain_fast50_dcn_combined_256x192.pth --indir examples/demo/ --save_img +``` +or, you can run with (this version is a little faster and more accurate on body keypoints, but its performance on hand keypoints is worser): +``` +python scripts/demo_inference.py --cfg configs/halpe_coco_wholebody_136/resnet/256x192_res50_lr1e-3_2x-regression.yaml --checkpoint pretrained_models/multi_domain_fast50_regression_256x192.pth --indir examples/demo/ --save_img +``` +#### Notes +- Above models are trained on multiple datasets, thus they can perform well for in-the-wild images. +- 'Combined (10 hand weight)' means that we use different loss for hand and body keypoints + ## [Halpe dataset](https://github.com/Fang-Haoshu/Halpe-FullBody) (136 keypoints) | Model | Backbone | Detector | Input Size | Loss Type | AP | Speed | Download | Config | @@ -73,22 +94,7 @@ python scripts/demo_inference.py --cfg configs/halpe_136/resnet/256x192_res50_lr - The APs are tested under COCO WholeBody's criterion, with flip test on. - The speed is tested on COCO val2017 on a single NVIDIA GeForce RTX 3090 gpu, with `batch_size=64` in each iteration and offline yolov3 human detection results. -## Multi Domain Models **(Strongly Recommended)** -| Model | Backbone | Detector | Input Size | Loss Type | AP | Speed | Download | Config | #keypoints | -|--------------------------|----------|----------|------------|------------|------------|-------|-----------|--------|--------------| -|[Fast Pose](../configs/halpe_coco_wholebody_136/resnet/256x192_res50_lr1e-3_2x-regression.yaml) | ResNet50 | YOLOv3 | 256x192 | Symmetric Integral | 50.1 | 16.28 iter/s | [Google](https://drive.google.com/file/d/1Bb3kPoFFt-M0Y3ceqNO8DTXi1iNDd4gI/view?usp=sharing) [Baidu(code: d0wi)](https://pan.baidu.com/s/1GaHzMHTqYze2rVn7u1sjVg) | [cfg](../configs/halpe_136/resnet/256x192_res50_lr1e-3_2x-regression.yaml) | 136 | -|[Fast Pose (DCN)](../configs/halpe_136/resnet/256x192_res50_lr1e-3_2x-dcn-combined.yaml) | ResNet50 - dcn | YOLOv3 | 256x192 | Combined (10 hand weight) | 49.8 | 10.35 iter/s | [Google](https://drive.google.com/file/d/1wX1Z2ZOoysgSNovlgiEtJKpbR8tUBWYR/view?usp=sharing) [Baidu(code: app1)](https://pan.baidu.com/s/1bIro0XfYj0FIVf84QzdDoQ) | [cfg](../configs/halpe_136/resnet/256x192_res50_lr1e-3_2x-dcn-combined.yaml) | 136 | -|[Fast Pose (DCN)](../configs/halpe_68_noface/resnet/256x192_res50_lr1e-3_2x-dcn-combined.yaml) | ResNet50 - dcn | YOLOv3 | 256x192 | Combined | - | 13.88 iter/s | [Google](https://drive.google.com/file/d/14Qn9gxm-EVzqFi7v25Y5TqKIvrFLy_BR/view?usp=sharing) [Baidu(code: 6kwr)](https://pan.baidu.com/s/1GLNxN3gfekUVY0HZu41fJQ) | [cfg](../configs/halpe_68_noface/resnet/256x192_res50_lr1e-3_2x-dcn-combined.yaml) | 68 (no face) | -|[Fast Pose (DCN)](../configs/single_hand/resnet/256x192_res50_lr1e-3_2x-dcn-regression.yaml) | ResNet50 - dcn | - | 256x192 | Symmetric Integral | - | 30.20 iter/s | [Google](https://drive.google.com/file/d/1MntndimlUP5Hxef1UN9ZDMBVglfA606J/view?usp=sharing) [Baidu(code: nwxx)](https://pan.baidu.com/s/1OR-uH25MFQ7kY8Gt_aJfbw ) | [cfg](../configs/single_hand/resnet/256x192_res50_lr1e-3_2x-dcn-regression.yaml) | 21 (single hand) | -For the most accurate wholebody pose estimation, you can run with: -``` -python scripts/demo_inference.py --cfg configs/halpe_coco_wholebody_136/resnet/256x192_res50_lr1e-3_2x-dcn-combined.yaml --checkpoint pretrained_models/multi_domain_fast50_dcn_combined_256x192.pth --indir examples/demo/ --save_img -``` -or, you can run with (this version is a little faster and more accurate on body keypoints, but its performance on hand keypoints is worser): -``` -python scripts/demo_inference.py --cfg configs/halpe_coco_wholebody_136/resnet/256x192_res50_lr1e-3_2x-regression.yaml --checkpoint pretrained_models/multi_domain_fast50_regression_256x192.pth --indir examples/demo/ --save_img -``` #### Notes - These models are strongly recommended because they are more accurate and flexible.