Cheetah is an optimization zoo for vision transformer (ViT) that consists of all popular ViT optimization techniques (sparse, pruning and multi-exit). Prior to our implementation, I list all influencing papers about ViT acceleration and develop them as optimization modules for any ViT models. Unlike FasterTransformer and DeepSpeed, Cheetah pays more attention to vision transformer instead of BERT, and provids more developer-friendly code through modular design (users can use their favorite optimizations to accelerate ViT). In the end, I will introduce how to profile your new ViT models and deploy them to NVIDIA Trition server (including how to accelerate them with TensorRT).
- Cheetah
- ViT Models (Pytorch)
- Optimizations (Pytorch and CUDA)
- Model Profiling (Pytorch, PyProf and nsight)
- Model Development (ONNX, TensorRT and Triton)
- End2end Profiling (Triton and DCGM)
- Advanced Optimization (TVM)
- Documentation (Main Feature, Installation and User Guide)