README.md

---

This is a curated list of tutorials, projects, libraries, videos, papers, books and anything related to the incredible PyTorch.

Tutorials

• Official PyTorch Tutorials
  1. Deep Learning with PyTorch: a 60-minute blitz
     • A perfect introduction to PyTorch's torch, autograd, nn and optim APIs
     • If you are a former Torch user, you can check out this instead: Introduction to PyTorch for former Torchies
  2. Custom C extensions
     • Write your own C code that interfaces into PyTorch via FFI
  3. Writing your own neural network module that uses numpy and scipy
  4. Reinforcement (Q-)Learning with PyTorch
• Official PyTorch Examples
  • MNIST Convnets
  • Word level Language Modeling using LSTM RNNs
  • Training Imagenet Classifiers with Residual Networks
  • Generative Adversarial Networks (DCGAN)
  • Variational Auto-Encoders
  • Superresolution using an efficient sub-pixel convolutional neural network
  • Hogwild training of shared ConvNets across multiple processes on MNIST
  • Training a CartPole to balance in OpenAI Gym with actor-critic
  • Natural Language Inference (SNLI) with GloVe vectors, LSTMs, and torchtext
• Practical PyTorch
  • This focuses on using RNNs for NLP
  • Classifying Names with a Character-Level RNN
  • Generating Shakespeare with a Character-Level RNN
  • Generating Names with a Conditional Character-Level RNN
  • Translation with a Sequence to Sequence Network and Attention
• Simple Examples to Introduce PyTorch
• Mini Tutorials in PyTorch
  • Tensor Multiplication, Linear Regresison, Logistic Regression, Neural Network, Modern Neural Network, and Convolutional Neural Network
• Deep Learning for NLP
  1. Introduction to Torch's Tensor Library
  2. Computation Graphs and Automatic Differentiation
  3. Deep Learning Building Blocks: Affine maps, non-linearities, and objectives
  4. Optimization and Training
  5. Creating Network Components in Pytorch * Example: Logistic Regression Bag-of-Words text classifier
  6. Word Embeddings: Encoding Lexical Semantics * Example: N-Gram Language Modeling * Exercise: Continuous Bag-of-Words for learning word embeddings
  7. Sequence modeling and Long-Short Term Memory Networks * Example: An LSTM for Part-of-Speech Tagging * Exercise: Augmenting the LSTM tagger with character-level features
  8. Advanced: Making Dynamic Decisions * Example: Bi-LSTM Conditional Random Field for named-entity recognition * Exercise: A new loss function
• Deep Learning Tutorial for Researchers
  1. PyTorch Basics
  2. Linear Regression
  3. Logistic Regression
  4. Feedforward Neural Network
  5. Convolutional Neural Network
  6. Deep Residual Network
  7. Recurrent Neural Network
  8. Bidirectional Recurrent Neural Network
  9. Language Model (RNNLM)
  10. Image Captioning (CNN-RNN)
  11. Generative Adversarial Network
  12. Deep Q-Network and Q-learning (WIP)
• Fully Convolutional Networks implemented with PyTorch

Papers Originally Implemented with PyTorch

• Wasserstein GAN
• OptNet: Differentiable Optimization as a Layer in Neural Networks
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• Wide ResNet model in PyTorch
• Task-based End-to-end Model Learning
• An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition
• Scaling the Scattering Transform: Deep Hybrid Networks
• Adversarial Generator-Encoder Network
• Conditional Similarity Networks
• Multi-style Generative Network for Real-time Transfer

Papers with Third-Party PyTorch Implementations

• Learning to learn by gradient descent by gradient descent
• Densely Connected Convolutional Networks
• A Neural Algorithm of Artistic Style
• Very Deep Convolutional Networks for Large-Scale Image Recognition
  • VGG model in PyTorch.
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Network In Network
• Deep Residual Learning for Image Recognition
  • ResNet model in PyTorch.
• Training Wide ResNets for CIFAR-10 and CIFAR-100 in PyTorch
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• FlowNet: Learning Optical Flow with Convolutional Networks
• Asynchronous Methods for Deep Reinforcement Learning
• Perceptual Losses for Real-Time Style Transfer and Super-Resolution
• Highway Networks
• Collection of Generative Models with PyTorch
  • Generative Adversarial Nets (GAN)
    1. Vanilla GAN
    2. Conditional GAN
    3. InfoGAN
    4. Wasserstein GAN
    5. Mode Regularized GAN
  • Variational Autoencoder (VAE)
    1. Vanilla VAE
    2. Conditional VAE
    3. Denoising VAE
    4. Adversarial Autoencoder
    5. Adversarial Variational Bayes
• A Recurrent Latent Variable Model for Sequential Data (VRNN)
• Hybrid computing using a neural network with dynamic external memory
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
• Value Iteration Networks
• YOLOv2: Real-Time Object Detection
• Convolutional Neural Fabrics
• Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Deformable Convolutional Network
• Continuous Deep Q-Learning with Model-based Acceleration
• Hierarchical Attention Networks for Document Classification
• Spatial Transformer Networks

Projects Implemented with Pytorch

• Collection of Sequence to Sequence Models with PyTorch
  1. Vanilla Sequence to Sequence models
  2. Attention based Sequence to Sequence models
  3. Faster attention mechanisms using dot products between the final encoder and decoder hidden states
• Reinforcement learning models in ViZDoom environment with PyTorch
• Neuraltalk 2, Image Captioning Model, in PyTorch
• Recurrent Neural Networks for Sentiment Analysis (Aspect-Based) on SemEval 2014
• PyTorch Image Classification with Kaggle Dogs vs Cats Dataset
• CNN Based Text Classification
• Open-source (MIT) Neural Machine Translation (NMT) System
• Pytorch Poetry Generation
• Data Augmentation and Sampling for Pytorch
• CIFAR-10 on Pytorch with VGG, ResNet and DenseNet
• Generate captions from an image with PyTorch
• Generative Adversarial Networks, focusing on anime face drawing
• Simple Generative Adversarial Networks
• Fast Neural Style Transfer
• Pixel-wise Segmentation on VOC2012 Dataset using PyTorch
• Draw like Bob Ross
• Reinforcement learning models using Gym and Pytorch
• Open-Source Neural Machine Translation in PyTorch
• Deep Video Analytics
• Adversarial Auto-encoders

Useful PyTorch Extensions

• Weight initialization schemes for PyTorch nn.Modules Added to PyTorch main branch
• Spatial transformer implemented in PyTorch

Community

• PyTorch Discussion Forum
  • This is actively maintained by Adam Paszke
• StackOverflow PyTorch Tags
• Gloqo
  • Add, discover and discuss paper implementations in PyTorch and other frameworks.

Contributions

Do feel free to contribute!

You can raise an issue or submit a pull request, whichever is more convenient for you. The guideline is simple: just follow the format of the previous bullet point.