Skip to content

[WACV 2021] Selective Spatio-Temporal Aggregation based Pose Refinement System: Towards understanding human activities in real-world videos.

License

Notifications You must be signed in to change notification settings

YangDi666/SSTA-PRS

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

40 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Selective Spatio-Temporal Aggregation Based Pose Refinement System

Refined pose data

ad

SST-A toolbox

To obtain refined pose sequence, you need to:

  1. Extract 2D poses of input video from 3 expert estimators (e.g., LCRNet, OpenPose, AlphaPose, ...);
  2. Save 2D pose results into 'xxx-pose1.npz', 'xxx-pose2.npz', 'xxx-pose3.npz', .... Make sure '.npz' has the dimension of nb_frames * nb_joints * 2 and joints indexes are consistent;
  3. Run the script to get refined pose by SST-A. Ouput will be save as 'output.npz'.
python tools/ssta.py --pose1 <xxx-pose1> --pose2 <xxx-pose2> --pose3 <xxx-pose3> --outname <output> (--gt <filename-gt if have>)

An example:

python tools/ssta.py --pose1 demo/WatchTV_p02_r03_v05_c05_Alphapose2d.npz --pose2 demo/WatchTV_p02_r03_v05_c05_LCRNet2d.npz --pose3 demo/WatchTV_p02_r03_v05_c05_OP2d.npz --outname demo/WatchTV_p02_r03_v05_c05_SSTA2d.npz

3D visualization

For 3D visualization, please use VideoPose3D.

Activity Recognition

For pose based activity recognition, please ref to UNIK / 2s-AGCN for smarthome.

Citation

If you find this code useful for your research, please consider citing our paper:

@InProceedings{Yang_2021_WACV,
    author = {Yang, Di and Dai, Rui and Wang, Yaohui and Mallick, Rupayan and Minciullo, Luca and Francesca, Gianpiero and Bremond, Francois},
    title = {Selective Spatio-Temporal Aggregation Based Pose Refinement System: Towards Understanding Human Activities in Real-World Videos},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
    month = {January},
    year = {2021}
}

About

[WACV 2021] Selective Spatio-Temporal Aggregation based Pose Refinement System: Towards understanding human activities in real-world videos.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%