Add example for correct usage of Optax lookahead optimizer #1542
+131
−4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
|
Thanks for the contribution, as is this is not a very helpful example, are you interested reviewing / expanding |
Author
Collaborator
|
I'm unclear on the direction of thi |
Collaborator
|
Sorry, closed by mistake. I'm unsure about the direction of this PR. As is, this PR does not meaningfully expand on the existing lookahead optimizer example and the contributed We already have a lookahead optimizer example notebook: Would you like to take a look at meaningfully expanding |
- Add comprehensive explanation section about lookahead optimizer - Include step-by-step initialization guide with comments - Enhance training loop with detailed docstrings and annotations - Add usage summary section with common mistakes and best practices - Clarify when to use params.fast vs params.slow This addresses reviewer feedback to expand the existing lookahead example notebook rather than adding a new .py file.
Solventerritory
force-pushed
the
lookahead-usage-example
branch
from
7cd25b8 to
8ecd70e
Compare
Solventerritory
force-pushed
the
lookahead-usage-example
branch
from
941810f to
1bbdad7
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a new example script,
examples/lookahead_usage.py, demonstrating the correct and intuitive usage of the Optax lookahead optimizer. The example clarifies the required parameter wrapping and update pattern, addressing confusion raised in issue #1429.LookaheadParamsThis should help users adopt the lookahead optimizer as a drop-in replacement, following the Optax API conventions.