Is it good to Add Feature "cosine annealing with warmup restart each cycle"?

[sihyeong671]

❓ Questions/Help/Support

I asked this question on discord channel

Do you think it is good to add "cosine annealing with warmup restart each cycle" feature?
If you have plan to add this feature, can I contribute this?

[vfdev-5]

@sihyeong671 thanks for opening this issue, indeed we discussed about this feature and eventually a possibility to add it to ignite.

Do you think it is good to add "cosine annealing with warmup restart each cycle" feature?

Yes, let's add it.

If you have plan to add this feature, can I contribute this?

RIght now we haven't thought about how exactly provide such lr scheduler:

• as an option to some existing class
• write a new class with the code inspired from https://github.com/katsura-jp/pytorch-cosine-annealing-with-warmup/blob/master/cosine_annealing_warmup/scheduler.py

If you have a suggestion we would be happy to try to elaborate it and you can definitely contribute it

[github-actions]

Hey 👋, I've just created a thread for this issue on PyTorch-Ignite Discord where you can quickly talk to the community on the topic.

^{🤖 This comment was automatically posted by Discuss on Discord}

[vfdev-5]

Fixed in ##3064