update exercise k

Author: nicklashansen

5_Recurrent/5.1-EXE-Recurrent-Neural-Networks.ipynb

@@ -473,7 +473,18 @@
         "\n",
         "When we are doing language modelling using a cross-entropy loss, we additionally apply the softmax function to the output $o_{t}$:\n",
         "\n",
-        "- $\\hat{y}_t = \\mathrm{softmax}(o_{t})$"
+        "- $\\hat{y}_t = \\mathrm{softmax}(o_{t})$\n",
+        "\n",
+        "\n",
+        "### Backpropagation through time\n",
+        "\n",
+        "We define a loss function\n",
+        "\n",
+        "- $E = \\sum_t E_t  = \\sum_t E_t(y_t ,\\hat{y}_t ) \\ , $\n",
+        "\n",
+        "where $E_t(y_t ,\\hat{y}_t )$ is the cross-entropy function.\n",
+        "\n",
+        "Backpropagation through time amounts to computing the gradients of the loss using the same type of clever bookkeeping we applied to the feed-forward network in week 1. This you will do in Exercise D."
       ]
     },
     {
@@ -2045,10 +2056,7 @@
         "        \n",
         "        # Recurrent layer\n",
         "        # YOUR CODE HERE!\n",
-        "        self.lstm = nn.LSTM(input_size=vocab_size,\n",
-        "                         hidden_size=50,\n",
-        "                         num_layers=1,\n",
-        "                         bidirectional=False)\n",
+        "        self.lstm = \n",
         "        \n",
         "        # Output layer\n",
         "        self.l_out = nn.Linear(in_features=50,\n",