Paper-Implementation-Overview-Gradient-Descent-Optimization-Algorithms

"An Overview of Gradient Descent Optimization Algorithms"

- Sebastian Ruder

Python 2.7

Links to original paper published on arXiv.org>cs>arXiv:1609.04747 : [1], [2]

Implemented following Gradient Desent Optimization Algorithms from Scratch :

1. Vanilla Batch/Stochastic Gradient Descent
2. Momentum
3. Nesterov Accelarated Gradient
4. Adagrad
5. Adadelta
6. RMS Prop