Human pose estimation models update #1718

druzhkov-paul · 2020-10-30T16:23:09Z

This PR adds three new human-pose-estimation models. These models have architecture different from human-pose-estimation-0001, hence, a new Python demo and AC adapter has been added to support new model.

As for human-pose-estimation-0001, it's support has been added to the new Python demo as well. But I may suppose that a few clarifications are needed here:

OMZ already has two implementations of a postprocessing stage for human-pose-estimation-0001: the one in a C++ demo and another one in a dedicated AC adapter (which is in Python). I assume that these two implementations are equivalent in terms of results. Though due to performance concerns some parameters differ in an AC config vs C++ demo application: for example featuremaps' upscaling factor.
I've added an alternative Python implementation of postprocessing. It has a few advantages:
- It is faster in a demo mode than current Python implementation from AC. C++ postprocessing is still faster, but not much.
- It is more accurate, i.e. gives better metrics on a test dataset. See the table below.
- It is aligned between the demo and AC.
To compare the speed I've added C++ postprocessing with Python bindings (which is also used in human-pose-estimation-3d demo) support to the new demo application. Though I haven't exposed its selection to the demo's CLI. So it should be either made configurable, or removed at all. Let me know, what do you think about it.

AP on the test dataset:

Upscaling factor	1x	4x	8x	original image scale
Current AC postprocessing	23.8	38.3	42.85	42.75
New postprocessing	39.1	41.1	43.2	--

TODOs:

Double check quality number after latest refactoring and clean up steps.
Decide what to do with C++ postprocessing (leave or drop).
AC config for human-pose-estimation-0001 with the new adapter has not been added. IMO, it can be added alongside the current one, rather than replace it, because new adapter is slow in that mode.

demos/python_demos/human_pose_estimation_demo/models.lst

vladimir-dudnik

please adopt existing demo to work with new models and old model (if you not going to deprecate old model)

demos/python_demos/human_pose_estimation_demo/models.lst

vladimir-dudnik

please remove empty sections from model descriptions.

models/intel/human-pose-estimation-0002/description/human-pose-estimation-0002.md

models/intel/human-pose-estimation-0003/description/human-pose-estimation-0003.md

models/intel/human-pose-estimation-0004/description/human-pose-estimation-0004.md

druzhkov-paul · 2020-11-19T13:56:17Z

It is not clear what we should decide "Decide what to do with C++ postprocessing (leave or drop)." There were no C++ postprocessing in this demo. We may consider to add this in future, no need to stop this PR because of that

This PR introduces a new demo, so for sure it had no C++ postprocessing earlier. And now it supports both Python and C++ postprocessing (the same as in human-pose-estimation-3d demo), but results and runtime differ. I'm Ok with dropping C++ postprocessing at all, this is just for your reference if you feel unsatisfied with Python postprocessing performance.

vladimir-dudnik · 2020-11-19T15:58:15Z

@druzhkov-paul I'm ok to have only python implementation of postprocessing for the first release of this demo. we may think about optimization later

demos/python_demos/human_pose_estimation_demo/human_pose_estimation_demo/model.py

druzhkov-paul · 2020-11-20T14:33:37Z

@druzhkov-paul I'm ok to have only python implementation of postprocessing for the first release of this demo. we may think about optimization later

Ok. Removed C++ postprocessing for now.

vladimir-dudnik · 2020-11-20T23:07:34Z

please add soft link to AC config from old configs location

Wovchena

Please add the new demo to the demos root readme. The rest looks fine.

demos/python_demos/human_pose_estimation_demo/README.md

eizamaliev · 2020-11-23T11:37:22Z

demos/python_demos/human_pose_estimation_demo/human_pose_estimation.py

+                      required=True, type=str)
+    args.add_argument('-m', '--model', help='Required. Path to an .xml file with a trained model.',
+                      required=True, type=str)
+    args.add_argument('--type', choices=('ae', 'openpose'), required=True, type=str,


Please, rename this key to -at, --architecture_type. (see here)

eizamaliev · 2020-11-23T11:41:33Z

demos/python_demos/human_pose_estimation_demo/human_pose_estimation.py

+        Modes.USER_SPECIFIED:
+            HPE(ie, args.model, target_size=args.tsize, device=args.device, plugin_config=config_user_specified,
+                results=completed_request_results, max_num_requests=args.num_infer_requests,
+                caught_exceptions=exceptions),
+        Modes.MIN_LATENCY:
+            HPE(ie, args.model, target_size=args.tsize, device=args.device.split(':')[-1].split(',')[0],
+                plugin_config=config_min_latency, results=completed_request_results, max_num_requests=1,
+                caught_exceptions=exceptions)


@Wovchena do we need two modes here?

No, but @druzhkov-paul was pointed at your version when it had this. That means to me it's our responsibility now to rewrite it later (that same is about -at, --architecture_type and other things).

Co-authored-by: Eduard Zamaliev <eduard.zamaliev@intel.com>

IRDonch · 2020-11-25T13:03:59Z

@eizamaliev Ping?

druzhkov-paul · 2020-11-27T08:59:27Z

@Daniil-Osokin FYI

Daniil-Osokin · 2020-12-06T12:51:02Z

@druzhkov-paul Sorry for the delay, it took awhile. Many thanks for making and sharing optimized post-processing! It really works faster in times, looks like you are vectorization jedi 😎 🤠 🦾. Very nice demo overall. And many thanks for the team (@eaidova @eizamaliev @IRDonch @vladimir-dudnik @Wovchena) for maintaining this awesome zoo 👍!

druzhkov-paul added 3 commits October 30, 2020 18:50

add hpe python demo

4acbc05

add models description

8844834

fix

3bd5a5e

eaidova added the model All about model, from enabling to issues label Oct 30, 2020

vladimir-dudnik suggested changes Nov 2, 2020

View reviewed changes

demos/python_demos/human_pose_estimation_demo/models.lst Outdated Show resolved Hide resolved

vladimir-dudnik suggested changes Nov 3, 2020

View reviewed changes

demos/python_demos/human_pose_estimation_demo/models.lst Outdated Show resolved Hide resolved

druzhkov-paul added 6 commits November 5, 2020 13:41

fix post-processing

1ea0532

fix net reshape

d2553bf

add initial AC support and config

aa76d7c

fix typo

cd0563e

update configs

e62ef4b

clean up

8a43ca8

druzhkov-paul force-pushed the dp/hpe_update branch from 1d638a7 to 8a43ca8 Compare November 5, 2020 16:03

druzhkov-paul added 2 commits November 5, 2020 22:52

fix padding

af96059

rename dataset

629ebc5

vladimir-dudnik suggested changes Nov 6, 2020

View reviewed changes

druzhkov-paul and others added 14 commits November 6, 2020 17:31

remove empty sections in readme

fcc343c

change hpe visualization

3c6112d

style fixes

18f4dea

initial openpose support

8ad309b

optimize grouping

396d8d2

optimize postprocessing

6b5db64

add OpenPose AC adapter draft

6e7cb20

up

180ca3c

add various nms impls to ac adapter

312ec71

add openvino nms to demo model

4384209

clean up

7d21e4f

align decoder in demo and AC

da298ff

refactor visualization

12e6434

clean up

5e18824

update arguments description

cb1af2e

Wovchena reviewed Nov 19, 2020

View reviewed changes

demos/python_demos/human_pose_estimation_demo/human_pose_estimation_demo/model.py Outdated Show resolved Hide resolved

vladimir-dudnik approved these changes Nov 19, 2020

View reviewed changes

Pavel Druzhkov added 5 commits November 20, 2020 16:38

move image resize to base Model class

86544c2

extend documentation of --tsize arg

0972357

support old numpy versions

187d39b

fix docs

230b52e

removed cpp postprocessing

2116b47

add symlinks to AC config files

38efe54

druzhkov-paul force-pushed the dp/hpe_update branch from 8aff9d6 to 38efe54 Compare November 21, 2020 22:14

Wovchena reviewed Nov 23, 2020

View reviewed changes

Wovchena assigned IRDonch Nov 23, 2020

add demo to the list of all demos in readme

a980b1e

eizamaliev previously requested changes Nov 23, 2020

View reviewed changes

eizamaliev reviewed Nov 23, 2020

View reviewed changes

Pavel Druzhkov and others added 3 commits November 23, 2020 14:43

fix readme

3d8b5f3

Co-authored-by: Eduard Zamaliev <eduard.zamaliev@intel.com>

change CLI arg name

1c0954c

fix demo readme...

190a85d

Wovchena approved these changes Nov 23, 2020

View reviewed changes

druzhkov-paul requested a review from eizamaliev November 23, 2020 14:56

vladimir-dudnik merged commit d6b6e3c into openvinotoolkit:develop Nov 26, 2020

druzhkov-paul deleted the dp/hpe_update branch November 27, 2020 08:59

Daniil-Osokin mentioned this pull request Dec 6, 2020

Create human_pose_estimation_2d_demo.py #1454

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Human pose estimation models update #1718

Human pose estimation models update #1718

druzhkov-paul commented Oct 30, 2020 •

edited by vladimir-dudnik

Loading

vladimir-dudnik left a comment

vladimir-dudnik left a comment

druzhkov-paul commented Nov 19, 2020

vladimir-dudnik commented Nov 19, 2020

druzhkov-paul commented Nov 20, 2020

vladimir-dudnik commented Nov 20, 2020

Wovchena left a comment

eizamaliev Nov 23, 2020

druzhkov-paul Nov 23, 2020

eizamaliev Nov 23, 2020

Wovchena Nov 23, 2020

IRDonch commented Nov 25, 2020

druzhkov-paul commented Nov 27, 2020

Daniil-Osokin commented Dec 6, 2020

Human pose estimation models update #1718

Human pose estimation models update #1718

Conversation

druzhkov-paul commented Oct 30, 2020 • edited by vladimir-dudnik Loading

vladimir-dudnik left a comment

Choose a reason for hiding this comment

vladimir-dudnik left a comment

Choose a reason for hiding this comment

druzhkov-paul commented Nov 19, 2020

vladimir-dudnik commented Nov 19, 2020

druzhkov-paul commented Nov 20, 2020

vladimir-dudnik commented Nov 20, 2020

Wovchena left a comment

Choose a reason for hiding this comment

eizamaliev Nov 23, 2020

Choose a reason for hiding this comment

druzhkov-paul Nov 23, 2020

Choose a reason for hiding this comment

eizamaliev Nov 23, 2020

Choose a reason for hiding this comment

Wovchena Nov 23, 2020

Choose a reason for hiding this comment

IRDonch commented Nov 25, 2020

druzhkov-paul commented Nov 27, 2020

Daniil-Osokin commented Dec 6, 2020

druzhkov-paul commented Oct 30, 2020 •

edited by vladimir-dudnik

Loading