Deep Transfer Learning

---

When Transfer Learning Meets Deep Learning

Survey

• Domain Adaptation for Visual Applications: A Comprehensive Survey

---

Target Data: labelled, Source Data: labelled

• DAN/JAN (Deep Adaptation Network/Joint Adaptation Network, ICML-15,17)
  • Caffe
• Learning Multiple Tasks with Multilinear Relationship Networks (NIPS-17)
  • Pytorch
• Multi-Adversarial Domain Adaptation (AAAI-18)
  • Caffe
• Partial Transfer Learning with Selective Adversarial Networks (arXiv-17)
  • Caffe
• Gradient Episodic Memory for Continual Learning (NIPS-17)
  • Pytorch
• Unified deep supervised domain adaptation and generalization (ICCV-17)
  • Keras
• Semi-supervised learning knowledge transfer for deep learning from private training data (ICLR-17)
  • Tensorflow
• Net2Net: Accelerating Learning via Knowledge Transfer(ICLR-16)
  • Tensorflow|Torch|Numpy

Evolution based

• Overcoming Catastrophic Forgetting in Neural Networks (PNAS-17)
  • Tensorflow
• Progressive Neural Networks (arXiv-16)
  • Tensorflow
• Evolution Channels Gradient Descent in Super Neural Networks (arXiv-17)
  • Tensorflow
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks (arxiv-17)
  • Tensorflow|Pytorch

One-shot learning

• One-shot Learning with Memory-Augmented Neural Networks (arXiv-16)
  • Theano
• Siamese Neural Networks for One-Shot Image Recognition (ICML-15)
  • Keras
• Learning to Compare: Relation Network for Few-Shot Learning (arXiv-17)
  • Pytorch

---

Target Data: labelled, Source Data: unlabelled

Self Taught Learning

---

Target Data: unlabelled, Source Data: labelled

• RTN (Unsupervised Domain Adaptation with Residual Transfer Networks, NIPS-16)
  • Caffe
• Associative Domain Adaptation (ICCV-17)
  • Tensorflow
• Deep CORAL: Correlation Alignment for Deep Domain Adaptation (ECCV-16)
  • PyTorch|Matlab
• Domain Separation Networks (NIPS-16)
  • Tensorflow
• Deep Hashing Network for Unsupervised Domain Adaptation (CVPR-17)
  • Matlab
• Deep Transfer Network: Unsupervised Domain Adaptation (arXiv-16)
  • Theano
• Joint distribution optimal transportation for domain adaptation (NIPS-17)
  • Sklearn
• When Unsupervised Domain Adaptation Meets Tensor Representations (ICCV-17)
  • Matlab
• Self-ensembling for visual domain adaptation (ICLR-18)
  • Theano
• AutoDIAL: Automatic DomaIn Alignment Layers (ICCV-17)
  • Caffe
• Minimal-Entropy Correlation Alignment for Unsupervised Deep Domain Adaptation (ICLR-18)
  • Tensorflow
• Asymmetric Tri-training for Unsupervised Domain Adaptation (arXiv-17)
  • Pytorch

GAN based

• Unsupervised Domain Adaptation by Backpropagation (ICML-15)
  • Caffe(from author)|Tensorflow(third party)
• Domain-Adversarial Training of Neural Networks (JMLR-16)
  • Tensorflow
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks (CVPR-17)
  • Tensorflow
• ADDA (Adversarial Discriminative Domain Adaptation, arXiv-17)
  • Tensorflow
• Coupled Generative Adversarial Networks (NIPS-16)
  • Tensorflow|caffe|Pytorch
• Wasserstein Distance Guided Representation Learning for Domain Adaptation (arXiv-17)
  • Pytorch

RNN based

• Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks (ICLR-17)
  • Theano

Zero-shot learning

Target Data: unlabelled, Source Data: unlabelled

Self Taught Clustering

• Self-Taught Convolutional Neural Networks for Short Text Clustering (Neural Networks-17)
  • Matlab

---

Concept Drift

• Dynamic Weighted Majority for Incremental Learning of Imbalanced Data Streams with Concept Drift (IJCAI-17)
  • Matlab

Transferability Analysis

• How transferable are features in deep neural networks?

---

Appliactions

• TransNets: Learning to Transform for Recommendation (RecSys-17)
  • Tensorflow
• Empower Sequence Labeling with Task-Aware Neural Language Model (AAAI-18)
  • Pytorch
• Adversarial Learning For Semi-Supervised Semantic Segmentation (ICLR 2018)
  • Pytorch
• Cross-Domain Sentiment Classification via Topic-Related TrAdaBoost (AAAI-17)
  • Keras