- Skeleton data of Smarthome:
To obtain refined pose sequence, you need to:
- Extract 2D poses of input video from 3 expert estimators (e.g., LCRNet, OpenPose, AlphaPose, ...);
- Save 2D pose results into 'xxx-pose1.npz', 'xxx-pose2.npz', 'xxx-pose3.npz', .... Make sure '.npz' has the dimension of nb_frames * nb_joints * 2 and joints indexes are consistent;
- Run the script to get refined pose by SST-A. Ouput will be save as 'output.npz'.
python tools/ssta.py --pose1 <xxx-pose1> --pose2 <xxx-pose2> --pose3 <xxx-pose3> --outname <output> (--gt <filename-gt if have>)
An example:
python tools/ssta.py --pose1 demo/WatchTV_p02_r03_v05_c05_Alphapose2d.npz --pose2 demo/WatchTV_p02_r03_v05_c05_LCRNet2d.npz --pose3 demo/WatchTV_p02_r03_v05_c05_OP2d.npz --outname demo/WatchTV_p02_r03_v05_c05_SSTA2d.npz
For 3D visualization, please use VideoPose3D.
For pose based activity recognition, please ref to UNIK / 2s-AGCN for smarthome.
If you find this code useful for your research, please consider citing our paper:
@InProceedings{Yang_2021_WACV,
author = {Yang, Di and Dai, Rui and Wang, Yaohui and Mallick, Rupayan and Minciullo, Luca and Francesca, Gianpiero and Bremond, Francois},
title = {Selective Spatio-Temporal Aggregation Based Pose Refinement System: Towards Understanding Human Activities in Real-World Videos},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
month = {January},
year = {2021}
}