Skip to content

callbacks revision: allow group lr and correct the logging for resuming training #386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 23, 2023

Conversation

wtomin
Copy link
Collaborator

@wtomin wtomin commented Jun 13, 2023

Thank you for your contribution to the MindOCR repo.
Before submitting this PR, please make sure:

Motivation

When resuming training, the correct logging should be [cur_epoch/(training_epochs+start_epochs)] to avoid confusion.

It is possible that opt.get_lr() is not a single scalar but a list of scalars. When users pass a list of dictionaries to the optimizer, for example:

[{  'params': group_base_params , 'lr': base_lr },   { 'params': group_small_params, 'lr': small_lr}]

The visualization should print each distinct lr.

Test Plan

  1. Try to resume training when start_epoch >1
  2. In mindocr/optim/param_grouping.py, create a new grouping function that returns a list of dictionaries containing different lrs, for example:
def grouping_params(params, filter_name, learning_rate):
    base_params, small_params = [],  []
    for param in params:
        if filter_name in param.name:
            base_params.append(param)
        else:
            small_params.append(param)
    return [{  'params': base_params, 'lr': learning_rate}, 
                { 'params': small_params, 'lr': learning_rate*0.1}]

Related Issues and PRs

NA

@wtomin wtomin force-pushed the callback branch 2 times, most recently from ab50f17 to 0525cd3 Compare June 23, 2023 08:28
@wtomin wtomin merged commit b193f84 into mindspore-lab:main Jun 23, 2023
@wtomin wtomin deleted the callback branch July 12, 2023 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants