Skip to content

Conversation

@YgorCastor
Copy link

This pull request introduces a new cosine decay schedule with linear warmup to the Polaris.Schedules module and adds corresponding tests to ensure its functionality.

New Feature: Cosine Decay Schedule with Linear Warmup

  • lib/polaris/schedules.ex: Added the warmup_cosine_decay function, which implements a learning rate schedule that first increases linearly during a warmup period and then follows a cosine decay pattern. This function includes detailed documentation and options for customization.

Tests for New Feature

  • test/polaris/schedules_test.exs: Added a series of tests for the warmup_cosine_decay function to verify its behavior, including tests for correct function arity, linear warmup phase, cosine decay phase, and respect for the alpha parameter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant