-
Notifications
You must be signed in to change notification settings - Fork 414
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Assert LLama Vision Image size divides by 14 #1745
base: main
Are you sure you want to change the base?
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/1745
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 2ac9100 with merge base 7cf656b (): This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Add a test. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1745 +/- ##
==========================================
+ Coverage 69.33% 69.34% +0.01%
==========================================
Files 305 305
Lines 15892 15914 +22
==========================================
+ Hits 11018 11036 +18
- Misses 4874 4878 +4 ☔ View full report in Codecov by Sentry. |
@@ -159,6 +160,7 @@ def lora_llama3_2_vision_11b( | |||
a subset of the attention projections in each layer. | |||
|
|||
""" | |||
assert image_size % 14 == 0, "image_size must be divisible by 14, the encode patch size" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you make this a ValueError instead?
@@ -41,6 +41,7 @@ def llama3_2_vision_11b( | |||
Returns: | |||
DeepFusionModel: Instantiation of the Llama 3.2 Vision 11B model | |||
""" | |||
assert image_size % 14 == 0, "image_size must be divisible by 14, the encode patch size" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
did you mean encoder or encode?
Also, should it be ValueError, or assert? I think that normally we prefer to raise ValueError
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just saw Rafi's comment :P
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just need to address comments and it's good to go
Image size must be divisible by ViT patches in the CLIP encoder, 14.