Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Grad Acc V3 #8088

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open

Grad Acc V3 #8088

wants to merge 20 commits into from

Conversation

chengtbf
Copy link
Contributor

@chengtbf chengtbf commented Apr 24, 2022 •
edited
Loading

重新实现 GradAcc

TODO:

  • exec_interval in RegstDesc； 重新实现 infer 逻辑
  • TrainStep 支持 exec_interval = acc num
  • LossInstanceNum 从 time shape -> acc num
  • Pipeline 、 nccl use compute stream 等支持新版的 acc

@chengtbf chengtbf added the WIP work in progress label Apr 24, 2022
@chengtbf

This comment was marked as duplicate.

Sign in to view
@chengtbf chengtbf marked this pull request as ready for review April 29, 2022 12:10
@chengtbf chengtbf added enhancement bottleneck blocking another feature/PR graph graph mode and removed WIP work in progress labels Apr 29, 2022
@chengtbf chengtbf changed the title [WIP] Grad Acc V3 Grad Acc V3 Apr 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hjchen2 hjchen2 Awaiting requested review from hjchen2 hjchen2 is a code owner

@BBuf BBuf Awaiting requested review from BBuf

@jackalcooper jackalcooper Awaiting requested review from jackalcooper jackalcooper is a code owner

@strint strint Awaiting requested review from strint strint is a code owner

@daquexian daquexian Awaiting requested review from daquexian

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
bottleneck blocking another feature/PR enhancement graph graph mode
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@chengtbf