Skip to content

Commit e86b9b6

Browse files
author
chenjunwen
committed
update
1 parent 7325933 commit e86b9b6

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

notes/intro_note_07.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -41,15 +41,15 @@
4141
- 使用所有**叶子节点**的动作价值函数去更新动作价值函数。
4242
- 回报:$G_{t:t+n} = R_{t+1} + \gamma \sum_{a\neq A_{t+1}}\pi(a|S_{t+1})Q_{t+n-1}(S_{t+1}, a) + \gamma\pi(A_{t+1}|S_{t+1})G_{t+1:t+n}$
4343

44-
![](../res/3-step-tree.png)
44+
![3-step-tree](../res/3-step-tree.png)
4545

46-
![](../res/n-step-tree.png)
46+
![n-step-tree](../res/n-step-tree.png)
4747

4848
## n-step $Q(\sigma)$
4949

5050
- $\sigma$代表是否使用全采样。
5151
- 回报:$G_{t:h} = R_{t+1} + \gamma(\sigma_{t+1}\rho_{t+1}+(1-\sigma_{t+1})\pi(A_{t+1}|S_{t+1}))(G_{t+1:h}-Q_{h-1}(S_{t+1}, A_{t+1})) + \gamma \bar V_{h-1}(S_{t+1})$
5252

53-
![](../res/4-step-q.png)
53+
![4-step-q](../res/4-step-q.png)
5454

55-
![](../res/n-step-q.png)
55+
![n-step-q](../res/n-step-q.png)

0 commit comments

Comments
 (0)