Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Lookahead and ModelAverage Optimizer #30004

Merged
merged 16 commits into from
Jan 6, 2021
Merged

Add Lookahead and ModelAverage Optimizer #30004

merged 16 commits into from
Jan 6, 2021

Conversation

123malin
Copy link
Contributor

@123malin 123malin commented Dec 29, 2020 •
edited
Loading

PR types

New features

PR changes

APIs

Describe

Add LookAhead and ModelAverage Optimizer
image
image
image
image
image
image
image
image
image
image
image
image
image
image
image
image

@paddle-bot-old
Copy link

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@123malin 123malin changed the title Lookahead Add Lookahead and ModelAverage Optimizer Dec 30, 2020
jzhang533
jzhang533 reviewed Dec 31, 2020
View reviewed changes
Copy link
Contributor

@jzhang533 jzhang533 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lookahead and ModelAverage 建议先放到incubate下
正式的API的设计，需要考虑更多的使用场景，动静一致，易用性等问题。
我们需要做更多的调研和收集反馈，才能确定。

jzhang533
jzhang533 approved these changes Jan 6, 2021
View reviewed changes
Copy link
Contributor

@jzhang533 jzhang533 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

zhiqiu
zhiqiu approved these changes Jan 6, 2021
View reviewed changes
Copy link
Contributor

@zhiqiu zhiqiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for op_function_generator.cc

raindrops2sea
raindrops2sea approved these changes Jan 6, 2021
View reviewed changes
Copy link
Collaborator

@raindrops2sea raindrops2sea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

XiaoguangHu01
XiaoguangHu01 approved these changes Jan 6, 2021
View reviewed changes
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@123malin 123malin merged commit 198fbdf into PaddlePaddle:develop Jan 6, 2021
123malin added a commit to 123malin/Paddle that referenced this pull request Jan 7, 2021
@123malin
Add Lookahead and ModelAverage Optimizer (PaddlePaddle#30004)
92dad2e 
* test=develop, add model_average and lookahead
fuyinno4 pushed a commit that referenced this pull request Jan 8, 2021
@123malin
【2.0API CherryPick】LookAhead, ModelAverage, IndexSelect (#30205)
3ce4d34 
* Add Lookahead and ModelAverage Optimizer (#30004)

* test=develop, add model_average and lookahead

* Improve Index select cuda kernel (#30139)

* test=develop, add index_select_cuda kernel
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jzhang533 jzhang533 jzhang533 approved these changes

@seiriosPlus seiriosPlus seiriosPlus approved these changes

@zhiqiu zhiqiu zhiqiu approved these changes

@raindrops2sea raindrops2sea raindrops2sea approved these changes

@XiaoguangHu01 XiaoguangHu01 XiaoguangHu01 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@123malin @jzhang533 @seiriosPlus @zhiqiu @raindrops2sea @XiaoguangHu01