Classifying human activities from pose by posenet model
The main aim of this repository is to classify few human activities based on the pose generated by posnet model.
1)Keras 2.0
2)opencv-python
3)posenet-python port :https://github.com/atomicbits/posenet-python
The above posenet-python port was slightly changed and then used for our repository.
so all the pre-requisites of the above repository are also required for this repository like tfjs-totf converter(https://github.com/patlevin/tfjs-to-tf).
Posenet, a deep learning model has shown state of art results in recognising pose. It detects 17 keypoints of human body to estimate the human pose. for more details of posenet...check out this link : https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5
The data which I used in this repository was generated from UCF101_data(Link is provided below) by running through the posenet model.I have taken 20 frames per datapoint and then ran it through the posenet model and obtained the heatmaps.So the data used is of shape (m,20,16,17,21) where m is the training examples you are going to extract from UCF101 dataset ,20 is the number of frames we are considering to create the dataset and (16 ,17 ,21) is the shape of heatmap.
Link for the data: https://www.crcv.ucf.edu/data/UCF101/UCF101.rar
The main idea is to train a model of few convlstm2d layers.The model was trained on focal loss to deal with class imbalance.Dropouts are also used to get rid of overfitting(as the model has high number of parameters ,there is a high chance of overfitting!).
The training was done in google colab(with GPU accelerator).And it took aroung 8hrs to fully get trained.The f1 score is around 0.92!
The confusion matrix is plotted below:
