Research and Development Project HBRS

A Comparative Study of Sparsity Methods in Deep Neural Network for Faster Inference

Code and documentation for research and development project with a topic in Deep Neural Network Compression as partial fulfillmrnt in Masters of Autonomous Systems program.

Repository structure:

• L1 norm pruning: cloned from https://github.com/Eric-mingjie/rethinking-network-pruning/tree/master/cifar/l1-norm-pruning with minor modifications. Based on the implementation of the paper Pruning Filters For Efficient ConvNets
• Weight level pruning : cloned from https://github.com/Eric-mingjie/rethinking-network-pruning/tree/master/cifar/l1-norm-pruning with minor modifications. Based on the implementation of the paper Pruning Filters For Efficient ConvNets
• Knowledge Distillation methods :
  • Cloned from https://github.com/peterliht/knowledge-distillation-pytorch with minor modifications. Based on the implementation of the paper Distilling the Knowledge in a Neural Network
  • FitNets implementation. Cloned from https://github.com/AberHu/Knowledge-Distillation-Zoo with modifications. Based on the implementation of the paper FitNets: Hints for Thin Deep Nets
• Report of research and development project