Lecturer

Hoa T. Le

Contact me at <first_name>.<last_name>@loria.fr or at my offfice B213 (Loria) (please make an appointment first).

Final project

Link. Note: extend deadline to midnight 27/4

Overview

The aim of this course is to introduce computational, numerical and distributed memories from a theoretical and epistemological standpoint as well as neural networks and their use in cognitive science. Concerning machine learning, the course will focus on various model learners such as Markov Chains, Reinforcement Learning and Neural Networks.

Target audience

This course is for Master 1 Science Cognitive and Applications (University of Lorraine). This is an introduction course, assuming no prior knowledge of Machine Learning.

Course Organization

• 30 hours = 10 work sessions of 3 hours/week
• Courses = half lectures / half exercises or practicals
• Evaluation: individual project
• The last 2 (maybe 3) work sessions will be saved to work on the project

20% projects

You can choose one of these books, read (entirely or at least 5 chapters) and write a resume in one page.

John Tabak´s series:

• Probability and Statistics: The Science of Uncertainty (History of Mathematics)
• Algebra: Sets, Symbols, and the Language of Thought (History of Mathematics)
• Geometry: The Language of Space and Form (History of Mathematics)
• Beyond Geometry: A New Mathematics of Space and Form (History of Mathematics)
• Numbers: Computers, Philosophers, and the Search for Meaning (History of Mathematics)
• Mathematics and the Laws of Nature: Developing the Language of Science (History of Mathematics)

Michael Guillen:

• Five Equations That Changed the World: The Power and Poetry of Mathematics

Ian Stewart:

• In Pursuit of the Unknown: 17 Equations That Changed the World

Book References

• Reinforcement Learning: An Introduction. Richard S. Sutton and Andrew G. Barto (1998).
• Numerical Optimization. Jorge Nocedal and Stephen J. Wright (1999).
• The Elements of Statistical Learning. H. Friedman, Robert Tibshirani and Trevor Hastie (2001).
• Inference in Hidden Markov Models. Olivier Cappé, Eric Moulines and Tobias Rydén (2005).
• Pattern Recognition and Machine Learning. Christopher M. Bishop (2006).
• Deep Learning. Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016).

Syllabus

Lecture 1. Introduction about Artificial Intelligence (Slides)

Reading

• Deep learning. Yann LeCun, Yoshua Bengio & Geoffrey Hinton. Nature 2015.
• Human-level control through Deep Reinforcement Learning. Mnih et al., Nature 2015.
• Mastering the game of Go with deep neural networks and tree search. Silver et al., Nature 2016.
• Hybrid computing using a neural network with dynamic external memory. Graves et al., Nature 2016.
• Neuroscience-Inspired Artificial Intelligence. Hassabis et al., Neuron 2017.

More Reading

• The Rise of Computer-Aided Explanation. Nielsen. QuantaMagazine 2015.
• Will Computers Redefine the Roots of Math ? Hartnett. QuantaMagazine 2015.
• Mapping the Brain to Build Better Machines. Singer, QuantaMagazine 2016.

Practical: Learning basic PyTorch (open tutorial)

• What is PyTorch ?
• Initialization and matrix computation
• Conversion between PyTorch <-> Numpy
• Autograd: automatic differentiation package

Installation instructions:

• How to Install Ubuntu 16.10/16.04 Alongside With Windows 10 or 8 in Dual-Boot
• Install conda environment (if it is not yet installed)
• Install PyTorch
• If Linux and PyTorch is properly installed, to run code, just open Linux terminal and call 'jupyter notebook'

Lecture 2. Baseline models and Loss functions (Slides)

• A classification’s pipeline
• K-Nearest Neighbors (KNN)
• Linear Classifier
• Loss function
• Regularization

Reading

• Chapters 5, 7 of Deep Learning Book.

Practical: Training an Image Classifier on CIFAR10 data from scratch (TP 1)

• Define the network
• Loss function
• Backprop
• Update the weights

Prerequisite: Linear Algebra

• Chapter 2 of Deep Learning Book.
• Linear algebra book (a good book on this subject)

Lecture 3-4. Optimization (Slides) (Revision)

• Linear Least Squares Optimization
  • Cholesky decomposition
  • QR decomposition
• Iterative methods
  • Steepest gradient descent
  • Momentum, Nesterov
  • Adaptive learning rates (Adagrad, Adadelta, Adam)

Reading

• Chapters 4, 8 of Deep Learning Book.

Practical: Neural Networks for Text (TP 2)

• Text Classification with Logistic Regression on BOW Sentence representation
• Text Classification with Word Embeddings
• N-Gram Language Modeling and Continuous BOW

Prerequisite:

• Numerical optimization book (Nocedal and Wright)
• Bag-of-words Sentence Representation
• Word Embeddings

Lecture 5. Neural Network (Slides)

• Feed Forward Neural Network
• Backpropagation
• Recurrent Neural Network

Reading

• Chapters 6, 10 of Deep Learning Book.

Practical: (TP 3)

• RNN for Part-of-Speech Tagging
• RNN for Language Modeling (optional)

More Reading

• Deep Neural Networks for Acoustic Modeling in Speech Recognition. IEEE Signal Processing Magazine 2012
• Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. Arxiv 2016

Lecture 6. Long-Short Term Memory Networks (Slides)

• Vanishing gradient problem of RNN
• Training recurrent networks (activation functions, gradient clipping, initialization,...)
• LSTM (Stacked LSTMs, BiLSTM)
• Sequence-to-Sequence model for Machine Translation

Reading

• Chapter 10 of Deep Learning Book.

Practical: (TP 4)

• Translation with a Sequence to Sequence Network and Attention (from scratch)

More Reading

• Google Neural Machine Translation tutorial (in tensorflow)

Lecture 7. Training Neural Networks (Slides)

• Activation functions
• Data preprocessing
• Weight initialization
• Batch normalization
• Regularization: Dropout

Reading

• Chapter 7 of Deep Learning Book.

Practical:

• OpenNMT PyTorch 1 (Data Loaders)
• OpenNMT PyTorch 2 (Framework and Modules)

Lecture 8. Autoencoders (Slides) (Final project link above)

• Undercomplete Autoencoders
• Denoising Autoencoder (DAE)
• Variational Autoencoder (VAE)
  • Information Theory
  • Shannon Entropy
  • Kullback-Leibler Divergence (Relative Entropy)
  • Approximate Inference
  • Variational Inference

Reading

• Chapter 14 of Deep Learning Book.

More Reading

• (CNN-DCNN) Autoencoder (AE): Yizhe Zhang, Dinghan Shen, Guoyin Wang, Zhe Gan, Ricardo Henao, Lawrence Carin. Deconvolutional Paragraph Representation Learning. NIPS 2017
• (Sequential) Denoising Autoencoder (DAE): Felix Hill, Kyunghyun Cho, Anna Korhonen. Learning Distributed Representations of Sentences from Unlabelled Data. NAACL-HLT 2016
• Variational Autoencoder (VAE): Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Jozefowicz, Samy Bengio. Generating Sentences from a Continuous Space. CoNLL 2016
• Adversarial Autoencoder (AAE): Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow. Adversarial Autoencoders. ICLR 2016

Lecture 9. Reinforcement Learning (Slides)

• Reinforcement Learning problem
• Inside an RL agent
  • Policy
  • Value function
  • Model
• Markov Decision Process
  • Markov Process
  • Markov Reward Process
    • Value function
    • Bellman equation for MRP

Reading

• Chapter 1 and 3 of Reinforcement Learning book.

Lecture 10. Solving Reinforcement Learning problems (1) (Slides)

• Markov Decision Process
  • Policy
  • Action-value function (Q-function)
  • Bellman equation for MDP
  • Optimal Value function
  • Optimal Policy
• Dynamic Programming (Model-based)
  • Policy Evaluation - Policy Iteration
  • Value Iteration

Reading

• Chapter 4 of Reinforcement Learning book.

Lecture 11. Solving Reinforcement Learning problems (2)

• Model-free
  • Prediction
    • Monte-Carlo Learning
  • Control
    • On-policy Monte-Carlo Control
    • Off-policy Learning (Q-learning)
• Value function approximation (ex: Atari games)

Reading

• Chapter 5 and 9, 10, 11 of Reinforcement Learning book.

More Reading

• General approximate solution Policy Gradient method in Chapter 13 and Integrating Learning and Planning in Chapter 8 of Reinforcement Learning book. (ex: game Go)
• Human-level control through Deep Reinforcement Learning. Mnih et al., Nature 2015.
• Mastering the game of Go with deep neural networks and tree search. Silver et al., Nature 2016.