Skip to content

Commit

Permalink
Update accelerate.md
Browse files Browse the repository at this point in the history
  • Loading branch information
huangjun12 authored Jan 15, 2021
1 parent 6682958 commit 650490a
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/zh-CN/tutorials/accelerate.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ num_workers=4
基于以上思想,FAIR在实验的基础上提出了Multigrid训练策略: 固定`N*C*T*H*W`的值,降低`T*H*W`时增大`N`的值,增大`T*H*W`时减小`N`的值。具体的有两种策略,如示意图所示:

Long cycle:
设完整训练需要N个epoch,将整个训练过程分4个阶段,每个阶段分别训练`[N/8, N/4,N/2, N]/(1/8+2/8+4/8+8/8) = [N/15, 2N/15, 4N/15, 8N/15]`个epoch数,每个阶段对应的输入tensor形状为:
设完整训练需要N个epoch,将整个训练过程分4个阶段,每个阶段对应的输入tensor形状为:
`[8N, T/4, H/sqrt(2), W/sqrt(2)], [4N, T/2, H/sqrt(2), W/sqrt(2)], [2N, T/2, H, W], [N, T, H, W]`

Short cycle:
Expand Down

0 comments on commit 650490a

Please sign in to comment.