Human_Activity_Anticipation_en
EN | 中文
An important aspect of human perception is anticipation, which we use extensively in our day-to-day activities when interacting with other humans as well as with our surroundings. Anticipating which activities will a human do next (and how to do them) can enable an assistive robot to plan ahead for reactive responses in the human environments. Furthermore, anticipation can even improve the detection accuracy of past activities. In this work, we represent each possible future using an anticipatory temporal conditional random field (ATCRF) that models the rich spatial-temporal relations through object affordances. We then consider each ATCRF as a particle and represent the distribution over the potential futures using a set of particles
This package has been compiled successfully in Ubuntu 14.04.
Before run this package,you need to install the required dependencies:
- OpenCV version 2.4 or greater (dev version or install from source)
- PCL 1.7
- Boost version 1.50 or greater
Commands to install the required dependencies and run anticipation code on CAD-120 Person 3 data:
# install pcl
sudo add-apt-repository ppa:v-launchpad-jochen-sprickerhof-de/pcl
sudo apt-get update
sudo apt-get install libpcl-all
# install opencv
sudo apt-get install libopencv-dev
#install boost 1.50
wget http://sourceforge.net/projects/boost/files/boost/1.50.0/boost_1_50_0.tar.bz2
tar --bzip2 -xf boost_1_50_0.tar.bz2
#If you prefer to install boost to a specific directory use the following instead
# ./bootstrap.sh --prefix=path/to/installation/prefix
./bootstrap.sh
./b2
sudo ./b2 install
# download code
git clone https://github.com/nathantsoi/human_activity_anticipation
# compile
cd human_activity_anticipation/build
cmake ..
make
cd ../src/pyobjs
make
#install learning code dependencies
cd ../../
sh install_dependencies.sh
# download data
cd data/
wget http://web3.cs.cornell.edu/pr/CAD-120/data/Subject3_rgbd_rawtext.tar.gz
wget http://pr.cs.cornell.edu/humanactivities/data/Subject3_annotations.tar.gz
tar -xvzf Subject3_annotations.tar.gz
tar -xvzf Subject3_rgbd_rawtext.tar.gz
mv Subject3_rgbd_rawtext/*/*rgbd.txt .
mkdir objects
mkdir objects_tracked
cp Subject3_annotations/*/objects/* objects/
cp Subject3_annotations/*/objects_tracked/* objects_tracked/
cp Subject3_annotations/*/*.bag .
cp Subject3_annotations/*/*.txt .
cat Subject3_annotations/*/activityLabel.txt | grep -v END > activityLabel.txt
cat Subject3_annotations/*/labeling.txt > labeling.txt
# run anticipation code
cd ../build/
./predict_seg ../data/ activityLabel.txt 1
waiting for adding...
Q1. When run the code, you may meet the error, COULD NOT LOAD MODULE "svmstruct_mrf_act_dyn"! perhaps module is not in module search path?
A1. First, you should make sure that you executed the script install_dependencies.sh successfully. It's best to install the libraries with the same version listed in the script. Second, you should modify the path（PYTHONPATH） at the bottoｍ of the script.
Q2. Why do we use person 3 dataset here?
A2. We found that some things lost in the person 1 dataset after debuging. So we use person 3 dataset. The following is the link to dataset. http://pr.cs.cornell.edu/humanactivities/data.php#cad120
Q3. When you meet the error ---- numpy.core._internal.AxisError: axis 1 is out of bounds for array of dimension 1
A3. You can check if the error is raised by numpy.concatenate() .
There is a warming when you run Z = np.concatenate((X,Y),1) in the early version of numpy. X and Y are one-dimensional vectors.
Warning: axis != 0 for ndim == 1; this will raise an error in future versions of numpy.
If you use the latest version of numpy, you will meet the error. So, you should run Z = np.concatenate((X,Y)) when the X and Y are one-dimensional vectors.
Q4. The dataset is too big and my computer is too slow to run it.
A4. After you download the dataset and preprocess it, you can modify the activityLabel.txt by deleting some human activity row by row. Then the computing comsunption will decrease.
Q4. How to understand the code?
A4. You can download the papers listed in the following websites: http://pr.cs.cornell.edu/humanactivities/index.php