Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add MFU Monitoring to LLM #56

Merged
merged 32 commits into from
Dec 30, 2022
Merged

Add MFU Monitoring to LLM #56

merged 32 commits into from
Dec 30, 2022

Conversation

vchiley
Copy link
Contributor

@vchiley vchiley commented Dec 21, 2022

Adds MFU to SpeedMonitor
A generalized variant of this feature should eventually be integrated into Composer's SpeedMonitor callback, but this will work well for MosaicGPT in examples/llm.

Screenshot 2022-12-21 at 1 54 21 PM

Screenshot 2022-12-21 at 1 55 49 PM

@vchiley vchiley self-assigned this Dec 21, 2022
abhi-mosaic
abhi-mosaic reviewed Dec 21, 2022
View reviewed changes
llm/main.py Outdated Show resolved Hide resolved
abhi-mosaic
abhi-mosaic approved these changes Dec 30, 2022
View reviewed changes
Copy link
Contributor

@abhi-mosaic abhi-mosaic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving to unblock, but I think it's important that we don't just hardcode the 312e12 number, otherwise when people run this repo on alternative hardware they will get incorrect MFU values in their reporting.

llm/src/speed_monitor_w_mfu.py Outdated Show resolved Hide resolved
llm/src/speed_monitor_w_mfu.py Outdated Show resolved Hide resolved
abhi-mosaic
abhi-mosaic approved these changes Dec 30, 2022
View reviewed changes
Copy link
Contributor

@abhi-mosaic abhi-mosaic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Excited to have all this info auto generated.

@abhi-mosaic abhi-mosaic merged commit fd08771 into mosaicml:main Dec 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@abhi-mosaic abhi-mosaic abhi-mosaic approved these changes

@bmosaicml bmosaicml Awaiting requested review from bmosaicml

@dakinggg dakinggg Awaiting requested review from dakinggg

@dblalock dblalock Awaiting requested review from dblalock

Assignees

@vchiley vchiley

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vchiley @abhi-mosaic