Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Idefics2 #30253

Merged
merged 131 commits into from
Apr 15, 2024
Merged

Add Idefics2 #30253

merged 131 commits into from
Apr 15, 2024

Commits on Mar 4, 2024

  1. Merge pull request #9 from huggingface/update

    Update
    molbap authored Mar 4, 2024
    Configuration menu
    Copy the full SHA
    e536f6a View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2024

  1. Configuration menu
    Copy the full SHA
    ef8c0fb View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2024

  1. Configuration menu
    Copy the full SHA
    79277fe View commit details
    Browse the repository at this point in the history
  2. Test

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    6c89a99 View commit details
    Browse the repository at this point in the history
  3. All weights loading

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    2e1155b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    661b794 View commit details
    Browse the repository at this point in the history
  5. Local and remote the same

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    dd0a3d2 View commit details
    Browse the repository at this point in the history
  6. Matching local and remote

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    d124863 View commit details
    Browse the repository at this point in the history
  7. Fixup

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    7aff0b7 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    799dc71 View commit details
    Browse the repository at this point in the history
  9. Don't skip by default

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    dbebbb1 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    465e3ed View commit details
    Browse the repository at this point in the history
  11. Remove self.config

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    ae5b94d View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    7983e93 View commit details
    Browse the repository at this point in the history
  13. Tidy up

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    0a00064 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    6e4ff1b View commit details
    Browse the repository at this point in the history
  15. Most tests passing

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    1aa8f7a View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    ea4bf34 View commit details
    Browse the repository at this point in the history
  17. Add image processor

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    b6a92da View commit details
    Browse the repository at this point in the history
  18. Update doc pages

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    0d09b95 View commit details
    Browse the repository at this point in the history
  19. Update conversion script

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    3c11158 View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    c6d4559 View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    c6275e9 View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    5dd0071 View commit details
    Browse the repository at this point in the history
  23. Fix up

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    015356b View commit details
    Browse the repository at this point in the history
  24. Image processor tests

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    8c50169 View commit details
    Browse the repository at this point in the history
  25. Update tests

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    da389b8 View commit details
    Browse the repository at this point in the history
  26. Add a processor

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    e8b131d View commit details
    Browse the repository at this point in the history
  27. Add a processor

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    2fc3ff3 View commit details
    Browse the repository at this point in the history
  28. Update convert script

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    e06740c View commit details
    Browse the repository at this point in the history
  29. Configuration menu
    Copy the full SHA
    083e82b View commit details
    Browse the repository at this point in the history
  30. Bug fix

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    256fa30 View commit details
    Browse the repository at this point in the history
  31. Add processing test

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    0fd5400 View commit details
    Browse the repository at this point in the history
  32. Use processor

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    f537f27 View commit details
    Browse the repository at this point in the history
  33. Fix up

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    d14485a View commit details
    Browse the repository at this point in the history
  34. Update src/transformers/models/idefics2/modeling_idefics2.py

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    02371e9 View commit details
    Browse the repository at this point in the history
  35. Update src/transformers/models/idefics2/modeling_idefics2.py

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    7fba70a View commit details
    Browse the repository at this point in the history
  36. Fix test

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    0987d15 View commit details
    Browse the repository at this point in the history
  37. Configuration menu
    Copy the full SHA
    78ba577 View commit details
    Browse the repository at this point in the history
  38. Reviewer comments

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    971dd72 View commit details
    Browse the repository at this point in the history
  39. Configuration menu
    Copy the full SHA
    d7dfec9 View commit details
    Browse the repository at this point in the history
  40. Update src/transformers/models/idefics2/modeling_idefics2.py

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    097f402 View commit details
    Browse the repository at this point in the history
  41. Apply suggestions from code review

    Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
    amyeroberts and ArthurZucker committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    1370836 View commit details
    Browse the repository at this point in the history
  42. Configuration menu
    Copy the full SHA
    9dff742 View commit details
    Browse the repository at this point in the history
  43. Fix

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    0e1be29 View commit details
    Browse the repository at this point in the history
  44. Configuration menu
    Copy the full SHA
    c334307 View commit details
    Browse the repository at this point in the history
  45. Sync with upstream main

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    e5b5bc4 View commit details
    Browse the repository at this point in the history
  46. Configuration menu
    Copy the full SHA
    ec867d8 View commit details
    Browse the repository at this point in the history
  47. Configuration menu
    Copy the full SHA
    0019bf1 View commit details
    Browse the repository at this point in the history
  48. Configuration menu
    Copy the full SHA
    b0e4081 View commit details
    Browse the repository at this point in the history
  49. Configuration menu
    Copy the full SHA
    863b2ee View commit details
    Browse the repository at this point in the history
  50. Configuration menu
    Copy the full SHA
    68990f8 View commit details
    Browse the repository at this point in the history
  51. Address review comments

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    e1456a0 View commit details
    Browse the repository at this point in the history
  52. Fix and fix up

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    f4b45d3 View commit details
    Browse the repository at this point in the history
  53. Configuration menu
    Copy the full SHA
    ffb2de3 View commit details
    Browse the repository at this point in the history
  54. Fix copies

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    700119d View commit details
    Browse the repository at this point in the history
  55. Fix

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    cefdd1d View commit details
    Browse the repository at this point in the history
  56. Add layer norms to FA2

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    4823ecd View commit details
    Browse the repository at this point in the history
  57. Fix tests

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    2de1098 View commit details
    Browse the repository at this point in the history
  58. Apply suggestions from code review

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    5205bba View commit details
    Browse the repository at this point in the history
  59. Fix

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    7edaff5 View commit details
    Browse the repository at this point in the history
  60. Review comments

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    a7a0a2c View commit details
    Browse the repository at this point in the history
  61. Update src/transformers/models/idefics2/modeling_idefics2.py

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    16f7666 View commit details
    Browse the repository at this point in the history
  62. Update inputs merger

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    e3a22e4 View commit details
    Browse the repository at this point in the history
  63. Configuration menu
    Copy the full SHA
    1c397b1 View commit details
    Browse the repository at this point in the history
  64. Update convert script

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    182ea5f View commit details
    Browse the repository at this point in the history
  65. Update src/transformers/models/idefics2/processing_idefics2.py

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    0ba4cc4 View commit details
    Browse the repository at this point in the history
  66. Update template

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    65bf223 View commit details
    Browse the repository at this point in the history
  67. Configuration menu
    Copy the full SHA
    84ea6e8 View commit details
    Browse the repository at this point in the history
  68. More review comments

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    ee548af View commit details
    Browse the repository at this point in the history
  69. Tidy up

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    649563b View commit details
    Browse the repository at this point in the history
  70. Update processing

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    4c4f315 View commit details
    Browse the repository at this point in the history
  71. Configuration menu
    Copy the full SHA
    f95e76b View commit details
    Browse the repository at this point in the history
  72. Configuration menu
    Copy the full SHA
    eae3f08 View commit details
    Browse the repository at this point in the history
  73. Vectorize inputs_merger

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    3043e40 View commit details
    Browse the repository at this point in the history
  74. Update src/transformers/models/idefics2/__init__.py

    Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
    amyeroberts and ArthurZucker committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    914fa74 View commit details
    Browse the repository at this point in the history
  75. Configuration menu
    Copy the full SHA
    877109a View commit details
    Browse the repository at this point in the history
  76. Review comments

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    9cde5c2 View commit details
    Browse the repository at this point in the history
  77. saying bye to the qk_layer_norms

    VictorSanh authored and amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    3307e6b View commit details
    Browse the repository at this point in the history
  78. Simplify

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    ecaac39 View commit details
    Browse the repository at this point in the history
  79. Update latents

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    366d21d View commit details
    Browse the repository at this point in the history
  80. Configuration menu
    Copy the full SHA
    5312a80 View commit details
    Browse the repository at this point in the history
  81. Configuration menu
    Copy the full SHA
    9d1078b View commit details
    Browse the repository at this point in the history
  82. Configuration menu
    Copy the full SHA
    b1e2f42 View commit details
    Browse the repository at this point in the history
  83. Configuration menu
    Copy the full SHA
    09796a3 View commit details
    Browse the repository at this point in the history
  84. image splitting

    VictorSanh authored and amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    eaff6e6 View commit details
    Browse the repository at this point in the history
  85. fix test

    VictorSanh authored and amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    0034f84 View commit details
    Browse the repository at this point in the history
  86. some more comment

    VictorSanh authored and amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    e2845b1 View commit details
    Browse the repository at this point in the history
  87. some comment

    VictorSanh authored and amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    3ae2a1b View commit details
    Browse the repository at this point in the history
  88. Apply suggestions from code review

    Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
    VictorSanh and amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    833a802 View commit details
    Browse the repository at this point in the history
  89. Update src/transformers/models/idefics2/image_processing_idefics2.py

    Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
    VictorSanh and amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    502c3dc View commit details
    Browse the repository at this point in the history
  90. Update processor

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    e8ca7b3 View commit details
    Browse the repository at this point in the history
  91. Update model tests

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    4bde406 View commit details
    Browse the repository at this point in the history
  92. Update src/transformers/models/idefics2/processing_idefics2.py

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    fea200e View commit details
    Browse the repository at this point in the history
  93. Update src/transformers/models/idefics2/processing_idefics2.py

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    33e51a6 View commit details
    Browse the repository at this point in the history
  94. Don't add BOS in template

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    fcad4e4 View commit details
    Browse the repository at this point in the history
  95. Update src/transformers/models/idefics2/processing_idefics2.py

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    1dc90f0 View commit details
    Browse the repository at this point in the history
  96. Remove index in examples

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    0e16d4a View commit details
    Browse the repository at this point in the history
  97. Update tests to reflect #13

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    107693a View commit details
    Browse the repository at this point in the history
  98. Update src/transformers/models/idefics2/processing_idefics2.py

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    cd4f76a View commit details
    Browse the repository at this point in the history
  99. Configuration menu
    Copy the full SHA
    31945ee View commit details
    Browse the repository at this point in the history
  100. Configuration menu
    Copy the full SHA
    4ab5e1d View commit details
    Browse the repository at this point in the history
  101. Update docs

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    d8c5045 View commit details
    Browse the repository at this point in the history
  102. Configuration menu
    Copy the full SHA
    e8b9751 View commit details
    Browse the repository at this point in the history
  103. Update examples

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    b5a7622 View commit details
    Browse the repository at this point in the history
  104. Fix and update tests

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    7ee8681 View commit details
    Browse the repository at this point in the history
  105. Small addition

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    31c6634 View commit details
    Browse the repository at this point in the history
  106. Configuration menu
    Copy the full SHA
    75f59ef View commit details
    Browse the repository at this point in the history
  107. Update example

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    b5ad135 View commit details
    Browse the repository at this point in the history
  108. small fixes

    VictorSanh authored and amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    419fba2 View commit details
    Browse the repository at this point in the history
  109. Update docs/source/en/model_doc/idefics2.md

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    ea3838e View commit details
    Browse the repository at this point in the history
  110. Update docs/source/en/model_doc/idefics2.md

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    301e1c5 View commit details
    Browse the repository at this point in the history
  111. Update README.md

    Co-authored-by: Victor SANH <victorsanh@gmail.com>
    amyeroberts and VictorSanh committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    7b1c4dc View commit details
    Browse the repository at this point in the history
  112. Connector model as bridge

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    5be2feb View commit details
    Browse the repository at this point in the history
  113. Fix up

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    34eb76b View commit details
    Browse the repository at this point in the history
  114. Fix up

    amyeroberts committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    7c73ede View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. Configuration menu
    Copy the full SHA
    455dccf View commit details
    Browse the repository at this point in the history
  2. IDEFICS-2 -> Idefics2

    VictorSanh committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    3bbd272 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #18 from huggingface/vs/name-change

    Name change
    VictorSanh authored Apr 15, 2024
    Configuration menu
    Copy the full SHA
    b122cb2 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    5414a02 View commit details
    Browse the repository at this point in the history
  5. IDEFICS-2 -> Idefics2

    amyeroberts committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    3d84654 View commit details
    Browse the repository at this point in the history
  6. Add back llava-next

    amyeroberts committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    8739092 View commit details
    Browse the repository at this point in the history
  7. Update readmes

    amyeroberts committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    779a8f8 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    f8c5301 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    661b93b View commit details
    Browse the repository at this point in the history
  10. Fix doc example

    amyeroberts committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    0efb5e8 View commit details
    Browse the repository at this point in the history
  11. Fix doc example

    amyeroberts committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    fed24d1 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    541ce14 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    2a563b2 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    26c8a55 View commit details
    Browse the repository at this point in the history
  15. Fix config doctest

    amyeroberts committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    16c8317 View commit details
    Browse the repository at this point in the history