Skip to content

Commit

Permalink
Corrected definition of intermediate gradient value
Browse files Browse the repository at this point in the history
  • Loading branch information
gitty-up committed Nov 1, 2015
1 parent 9b2b45f commit 310c74f
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion optimization-2.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ invden = 1.0 / den #(7)
f = num * invden # done! #(8)
```

Phew, by the end of the expression we have computed the forward pass. Notice that we have structured the code in such way that it contains multiple intermediate variables, each of which are only simple expressions for which we already know the local gradients. Therefore, computing the backprop pass is easy: We'll go backwards and for every variable along the way in the forward pass (`sigy, num, sigx, xpy, xpysqr, den, invden`) we will have the same variable, but one that begins with a `d`, which will hold the gradient of that variable with respect to the output of the circuit. Additionally, note that every single piece in our backprop will involve computing the local gradient of that expression, and chaining it with the gradient on that expression with a multiplication. For each row, we also highlight which part of the forward pass it refers to:
Phew, by the end of the expression we have computed the forward pass. Notice that we have structured the code in such way that it contains multiple intermediate variables, each of which are only simple expressions for which we already know the local gradients. Therefore, computing the backprop pass is easy: We'll go backwards and for every variable along the way in the forward pass (`sigy, num, sigx, xpy, xpysqr, den, invden`) we will have the same variable, but one that begins with a `d`, which will hold the gradient of the output of the circuit with respect to that variable. Additionally, note that every single piece in our backprop will involve computing the local gradient of that expression, and chaining it with the gradient on that expression with a multiplication. For each row, we also highlight which part of the forward pass it refers to:

```python
# backprop f = num * invden
Expand Down

0 comments on commit 310c74f

Please sign in to comment.