|
720 | 720 | "- updating the weights of the network\n",
|
721 | 721 | "\n",
|
722 | 722 | "The simplest update rule used in practice is the Stochastic Gradient Descent (SGD):\n",
|
723 |
| - "> `weight = weight = learning_rate * gradient`\n", |
| 723 | + "> `weight = weight - learning_rate * gradient`\n", |
724 | 724 | "\n",
|
725 | 725 | "We can implement this using simple python code:\n",
|
726 | 726 | "\n",
|
727 | 727 | "```python\n",
|
728 | 728 | "learning_rate = 0.01\n",
|
729 | 729 | "for f in net.parameters():\n",
|
730 |
| - " f.data.add_(f.grad * learning_rate)\n", |
| 730 | + " f.data.sub_(f.grad * learning_rate)\n", |
731 | 731 | "```\n",
|
732 | 732 | "\n",
|
733 | 733 | "However, as you use neural networks, you want to use various different update rules such as SGD, Nesterov-SGD, Adam, RMSProp, etc.\n",
|
|
1199 | 1199 | "source": [
|
1200 | 1200 | "## Where do I go next?\n",
|
1201 | 1201 | "\n",
|
1202 |
| - "- [Train neural nets to play video games](https://github.com/pytorch/tutorials)\n", |
1203 |
| - "- [Train a state-of-the-art ResNet network on imagenet](https://github.com/pytorch/examples)\n", |
1204 |
| - "- [Train an face generator using Generative Adversarial Networks](https://github.com/pytorch/examples)\n", |
| 1202 | + "- [Train neural nets to play video games](https://github.com/pytorch/tutorials/blob/master/Reinforcement%20(Q-)Learning%20with%20PyTorch.ipynb)\n", |
| 1203 | + "- [Train a state-of-the-art ResNet network on imagenet](https://github.com/pytorch/examples/tree/master/imagenet)\n", |
| 1204 | + "- [Train an face generator using Generative Adversarial Networks](https://github.com/pytorch/examples/tree/master/dcgan)\n", |
1205 | 1205 | "- [Train a word-level language model using Recurrent LSTM networks](https://github.com/pytorch/examples/tree/master/word_language_model)\n",
|
1206 | 1206 | "- [More examples](https://github.com/pytorch/examples)\n",
|
1207 | 1207 | "- [More tutorials](https://github.com/pytorch/tutorials)\n",
|
|
0 commit comments