Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Human pose estimation models update #1718

Merged
merged 58 commits into from
Nov 26, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
4acbc05
add hpe python demo
druzhkov-paul Oct 27, 2020
8844834
add models description
druzhkov-paul Oct 30, 2020
3bd5a5e
fix
druzhkov-paul Oct 30, 2020
1ea0532
fix post-processing
druzhkov-paul Nov 5, 2020
d2553bf
fix net reshape
druzhkov-paul Nov 5, 2020
aa76d7c
add initial AC support and config
druzhkov-paul Nov 5, 2020
cd0563e
fix typo
druzhkov-paul Nov 5, 2020
e62ef4b
update configs
druzhkov-paul Nov 5, 2020
8a43ca8
clean up
druzhkov-paul Nov 5, 2020
af96059
fix padding
druzhkov-paul Nov 5, 2020
629ebc5
rename dataset
druzhkov-paul Nov 6, 2020
fcc343c
remove empty sections in readme
druzhkov-paul Nov 6, 2020
3c6112d
change hpe visualization
druzhkov-paul Nov 6, 2020
18f4dea
style fixes
druzhkov-paul Nov 9, 2020
8ad309b
initial openpose support
druzhkov-paul Nov 9, 2020
396d8d2
optimize grouping
druzhkov-paul Nov 9, 2020
6b5db64
optimize postprocessing
Nov 10, 2020
6e7cb20
add OpenPose AC adapter draft
druzhkov-paul Nov 11, 2020
180ca3c
up
druzhkov-paul Nov 12, 2020
312ec71
add various nms impls to ac adapter
Nov 12, 2020
4384209
add openvino nms to demo model
Nov 12, 2020
7d21e4f
clean up
Nov 13, 2020
da298ff
align decoder in demo and AC
Nov 13, 2020
12e6434
refactor visualization
Nov 13, 2020
5e18824
clean up
Nov 13, 2020
7af051b
add cpp postprocessing support
Nov 13, 2020
59d9bb4
refactor openpose decoder (wip)
Nov 13, 2020
fafd7b2
moved nms out of decoder
Nov 13, 2020
89053cf
refactor openpose related components
Nov 16, 2020
63c56c9
update readme and model.lst for hpe demo
Nov 16, 2020
78fea41
make target size configurable
Nov 16, 2020
c182441
fix trailing spaces
vladimir-dudnik Nov 16, 2020
af6452b
fix trailing spaces
vladimir-dudnik Nov 16, 2020
6f71ddc
revert changes in HumanPoseAdapter
Nov 16, 2020
e009bc0
fix linter comments
Nov 16, 2020
35e3cb2
moved pose estimation adapters to separate modules
Nov 17, 2020
aca1ccc
fixes
druzhkov-paul Nov 17, 2020
cc24fe5
fix GPU inference in demo
Nov 18, 2020
f56b5ed
improve performance of ae decoder
Nov 18, 2020
15bae7b
add license headers
Nov 18, 2020
3f92178
add adapters description to readme
Nov 18, 2020
a37d6e9
handle scipy absence gently
Nov 18, 2020
991a13a
fix models doc pages
Nov 18, 2020
81bc936
fix indent
Nov 18, 2020
2257add
move AC configs
Nov 18, 2020
679b8a4
remove pseudo links
Nov 18, 2020
4946f64
fix typo
Nov 19, 2020
cb1af2e
update arguments description
Nov 19, 2020
86544c2
move image resize to base Model class
Nov 20, 2020
0972357
extend documentation of --tsize arg
Nov 20, 2020
187d39b
support old numpy versions
Nov 20, 2020
230b52e
fix docs
Nov 20, 2020
2116b47
removed cpp postprocessing
Nov 20, 2020
38efe54
add symlinks to AC config files
druzhkov-paul Nov 21, 2020
a980b1e
add demo to the list of all demos in readme
Nov 23, 2020
3d8b5f3
fix readme
Nov 23, 2020
1c0954c
change CLI arg name
Nov 23, 2020
190a85d
fix demo readme...
Nov 23, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion demos/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ The Open Model Zoo includes the following demos:
- [Formula Recognition Python* Demo](./python_demos/formula_recognition_demo/README.md) - The demo demonstrates how to run Im2latex formula recognition models and recognize latex formulas.
- [Gaze Estimation C++ Demo](./gaze_estimation_demo/README.md) - Face detection followed by gaze estimation, head pose estimation and facial landmarks regression.
- [Human Pose Estimation C++ Demo](./human_pose_estimation_demo/README.md) - Human pose estimation demo.
- [Human Pose Estimation Python\* Demo](./python_demos/human_pose_estimation_demo/README.md) - Human pose estimation demo.
- [Image Inpainting Python Demo](./python_demos/image_inpainting_demo/README.md) - Demo application for GMCNN inpainting network.
- [Image Retrieval Python* Demo](./python_demos/image_retrieval_demo/README.md) - The demo demonstrates how to run Image Retrieval models using OpenVINO™.
- [Image Segmentation C++ Demo](./segmentation_demo/README.md) - Inference of image segmentation networks like FCN8 (the demo supports only images as inputs).
Expand Down Expand Up @@ -69,7 +70,10 @@ The table below shows the correlation between models, demos, and supported plugi
| person-vehicle-bike-detection-crossroad-0078 | [Crossroad Camera Demo](./crossroad_camera_demo/README.md) | Supported | Supported | Supported | Supported |
| person-vehicle-bike-detection-crossroad-1016 | [Crossroad Camera Demo](./crossroad_camera_demo/README.md) | Supported | | | |
| person-vehicle-bike-detection-crossroad-yolov3-1020 | [Object Detection for YOLO V3 Python\* Demo](./python_demos/object_detection_demo_yolov3_async/README.md) | Supported | | | |
| human-pose-estimation-0001 | [Human Pose Estimation Demo](./human_pose_estimation_demo/README.md) | Supported | Supported | Supported | Supported |
| human-pose-estimation-0001 | [Human Pose Estimation Demo](./human_pose_estimation_demo/README.md)<br>[Human Pose Estimation Python\* Demo](./python_demos/human_pose_estimation_demo/README.md) | Supported | Supported | Supported | Supported |
| human-pose-estimation-0002 | [Human Pose Estimation Python\* Demo](./python_demos/human_pose_estimation_demo/README.md) | Supported | Supported | | |
| human-pose-estimation-0003 | [Human Pose Estimation Python\* Demo](./python_demos/human_pose_estimation_demo/README.md) | Supported | Supported | | |
| human-pose-estimation-0004 | [Human Pose Estimation Python\* Demo](./python_demos/human_pose_estimation_demo/README.md) | Supported | Supported | | |
| image-retrieval-0001 | [Image Retrieval Python\* Demo](./python_demos/image_retrieval_demo/README.md) | Supported | Supported | Supported | Supported |
| semantic-segmentation-adas-0001 | [Image Segmentation Demo](./segmentation_demo/README.md) | Supported | Supported | | Supported |
| instance-segmentation-security-0010 | [Instance Segmentation Demo](./python_demos/instance_segmentation_demo/README.md) | Supported | | | Supported |
Expand Down
118 changes: 118 additions & 0 deletions demos/python_demos/human_pose_estimation_demo/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Human Pose Estimation Python\* Demo

This demo showcases the work of multi-person 2D pose estimation algorithms. The task is to predict a pose: body skeleton, which consists of a predefined set of keypoints and connections between them, for every person in an input image/video.

Demo application supports inference in both sync and async modes. Please refer to [Optimization Guide](https://docs.openvinotoolkit.org/latest/_docs_optimization_guide_dldt_optimization_guide.html) and [Object Detection SSD, Async API performance showcase](../../object_detection_demo_ssd_async/README.md) demo for more information about Async API and its use.

Other demo objectives are:
* Video as input support via OpenCV\*
* Visualization of the resulting poses
* Demonstration of the Async API in action. For this, the demo features two modes toggled by the **Tab** key:
- "User specified" mode, where you can set the number of Infer Requests, throughput streams and threads.
Inference, starting new requests and displaying the results of completed requests are all performed asynchronously.
The purpose of this mode is to get the higher FPS by fully utilizing all available devices.
- "Min latency" mode, which uses only one Infer Request. The purpose of this mode is to get the lowest latency.

## How It Works

On the start-up, the application reads command-line parameters and loads a network to the Inference
Engine. Upon getting a frame from the OpenCV VideoCapture, it performs inference and displays the results.

> **NOTE**: By default, Open Model Zoo demos expect input with BGR channels order. If you trained your model to work
with RGB order, you need to manually rearrange the default channels order in the demo application or reconvert your
model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about
the argument, refer to **When to Reverse Input Channels** section of
[Converting a Model Using General Conversion Parameters](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Converting_Model_General.html).

## Running

Running the application with the `-h` option yields the following usage message:
```
python3 human_pose_estimation.py -h
```
The command yields the following usage message:
```
usage: human_pose_estimation.py [-h] -i INPUT -m MODEL -at {ae,openpose}
[--tsize TSIZE] [-t PROB_THRESHOLD] [-r]
[-d DEVICE] [-nireq NUM_INFER_REQUESTS]
[-nstreams NUM_STREAMS]
[-nthreads NUM_THREADS] [-loop LOOP]
[-no_show] [-u UTILIZATION_MONITORS]

Options:
-h, --help Show this help message and exit.
-i INPUT, --input INPUT
Required. Path to an image, video file or a numeric
camera ID.
-m MODEL, --model MODEL
Required. Path to an .xml file with a trained model.
-at {ae,openpose}, --architecture_type {ae,openpose}
Required. Type of the network, either "ae" for
Associative Embedding or "openpose" for OpenPose.
--tsize TSIZE Optional. Target input size. This demo implements
image pre-processing pipeline that is common to human
pose estimation approaches. Image is resize first to
some target size and then the network is reshaped to
fit the input image shape. By default target image
size is determined based on the input shape from IR.
Alternatively it can be manually set via this
parameter. Note that for OpenPose-like nets image is
resized to a predefined height, which is the target
size in this case. For Associative Embedding-like nets
target size is the length of a short image side.
-t PROB_THRESHOLD, --prob_threshold PROB_THRESHOLD
Optional. Probability threshold for poses filtering.
-r, --raw_output_message
Optional. Output inference results raw values showing.
-d DEVICE, --device DEVICE
Optional. Specify the target device to infer on; CPU,
GPU, FPGA, HDDL or MYRIAD is acceptable. The sample
will look for a suitable plugin for device specified.
Default value is CPU.
-nireq NUM_INFER_REQUESTS, --num_infer_requests NUM_INFER_REQUESTS
Optional. Number of infer requests
-nstreams NUM_STREAMS, --num_streams NUM_STREAMS
Optional. Number of streams to use for inference on
the CPU or/and GPU in throughput mode (for HETERO and
MULTI device cases use format
<device1>:<nstreams1>,<device2>:<nstreams2> or just
<nstreams>)
-nthreads NUM_THREADS, --num_threads NUM_THREADS
Optional. Number of threads to use for inference on
CPU (including HETERO cases)
-loop LOOP, --loop LOOP
Optional. Number of times to repeat the input.
-no_show, --no_show Optional. Don't show output
-u UTILIZATION_MONITORS, --utilization_monitors UTILIZATION_MONITORS
Optional. List of monitors to show initially.
```

Running the application with the empty list of options yields the short usage message and an error message.
You can use the following command to do inference on CPU with a pre-trained human pose estimation model:
```
python3 human_pose_estimation.py -i <path_to_video>/inputVideo.mp4 -m <path_to_model>/hpe.xml -d CPU
```

To run the demo, you can use public or pre-trained models. You can download the pre-trained models with the OpenVINO
[Model Downloader](../../../tools/downloader/README.md) or from
[https://download.01.org/opencv/](https://download.01.org/opencv/).

> **NOTE**: Before running the demo with a trained model, make sure the model is converted to the Inference Engine
format (\*.xml + \*.bin) using the
[Model Optimizer tool](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html).

The only GUI knob is to use **Tab** to switch between the synchronized execution ("Min latency" mode)
and the asynchronous mode configured with provided command-line parameters ("User specified" mode).

## Demo Output

The demo uses OpenCV to display the resulting frame with estimated poses.
The demo reports
* **FPS**: average rate of video frame processing (frames per second)
* **Latency**: average time required to process one frame (from reading the frame to displaying the results)
You can use both of these metrics to measure application-level performance.

## See Also
* [Using Open Model Zoo demos](../../README.md)
* [Model Optimizer](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
* [Model Downloader](../../../tools/downloader/README.md)
Loading