A PyTorch-based model pruning toolkit for pre-trained language models

XtremeDistil framework for distilling/compressing massive multilingual neural network models to tiny and efficient models for AI at scale

[ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408