-
Notifications
You must be signed in to change notification settings - Fork 2.8k
[misc] fix: Use AutoModelForImageTextToText instead of AutoModelForVision2Seq
#2475
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request updates the checkpoint saving logic to handle cases where generation_config.json is missing and adds support for glm4v in the FLOPs counter.
My main feedback is on the fsdp_checkpoint_manager.py file. The current implementation for checking the existence of generation_config.json introduces a potential race condition. I've suggested a more robust try-except pattern to prevent potential crashes.
The changes in flops_counter.py look good.
I noticed the PR title mentions replacing AutoModelForVision2Seq with AutoModelFotImageTextToText, but this change doesn't seem to be included in the diffs. You might want to double-check if that was intended for this PR.
AutoModelFotImageTextToText instead of AutoModelForVision2SeqAutoModelForImageTextToText instead of AutoModelForVision2Seq
|
@eric-haibin-lin Hi eric. Could you review this PR? Thanks a lot. |
Removed 'glm4v' from the list of supported models.
What does this PR do?
AutoModelForVision2Seqwill be deprecated soon. So verl need to useAutoModelForImageTextToTextinstead.