Bump the patch-update group with 10 updates

[dependabot]

Bumps the patch-update group with 10 updates:

Package	From	To
tomli	2.0.1	2.0.2
awscli	1.35.16	1.35.19
boto3	1.35.50	1.35.53
botocore	1.35.50	1.35.53
buildkite-test-collector	0.1.8	0.1.9
lm-eval[api]	0.4.4	0.4.5
rpds-py	0.20.0	0.20.1
torchvision	0.20	0.20.1
yarl	1.17.0	1.17.1
lm-format-enforcer	0.10.6	0.10.9

Updates tomli from 2.0.1 to 2.0.2

Changelog

Sourced from tomli's changelog.

2.0.2

• Removed
  • Python 3.7 support
• Improved
  • Make loads raise TypeError not AttributeError on bad input types that do not have the replace attribute. Improve error message when bytes is received.
• Type annotations
  • Type annotate load input as typing.IO[bytes] (previously typing.BinaryIO).

Commits

• 3ec6775 Bump version: 2.0.1 → 2.0.2
• 1dcd317 Add v2.0.2 changelog
• c94ee69 Fix GitHub Actions badge
• 4e245a4 tomli.loads: Raise TypeError not AttributeError. Improve message (#229)
• facdab0 Update pre-commit. Remove docformatter
• a613867 Use sys.version_info in compatibility layer (#220)
• 39eff9b Add support for Python 3.12, drop EOL 3.7 (#224)
• 0054e60 [pre-commit.ci] pre-commit autoupdate (#208)
• 1bd3345 Test against Python 3.12-dev
• 5646e69 Type annotate as IO[bytes], not BinaryIO
• Additional commits viewable in compare view

Updates awscli from 1.35.16 to 1.35.19

Commits

• 1d0eedf Merge branch 'release-1.35.19'
• a670144 Bumping version to 1.35.19
• c4b2b3e Update changelog based on model updates
• a0dba13 Merge branch 'release-1.35.18'
• 8a78a2e Merge branch 'release-1.35.18' into develop
• ba899b0 Bumping version to 1.35.18
• 3bd5551 Update changelog based on model updates
• 31327fc Merge branch 'release-1.35.17'
• cc622a3 Merge branch 'release-1.35.17' into develop
• 5587ab9 Bumping version to 1.35.17
• Additional commits viewable in compare view

Updates boto3 from 1.35.50 to 1.35.53

Commits

• 7fda69a Merge branch 'release-1.35.53'
• d6e1a82 Bumping version to 1.35.53
• 88bbcd4 Add changelog entries from botocore
• 93b71d0 Merge branch 'release-1.35.52'
• e980e72 Merge branch 'release-1.35.52' into develop
• d61432d Bumping version to 1.35.52
• cba8e22 Add changelog entries from botocore
• 62dc1fd Merge branch 'release-1.35.51'
• d557327 Merge branch 'release-1.35.51' into develop
• 174ce87 Bumping version to 1.35.51
• Additional commits viewable in compare view

Updates botocore from 1.35.50 to 1.35.53

Commits

• dc3f272 Merge branch 'release-1.35.53'
• efd711b Bumping version to 1.35.53
• 11d2dc5 Update endpoints model
• 9f96d16 Update to latest models
• 43ce8ed Bucket name doc updates (#3292)
• 0b82f04 Merge branch 'release-1.35.52'
• d2dd935 Merge branch 'release-1.35.52' into develop
• 0e35146 Bumping version to 1.35.52
• a839e28 Update endpoints model
• 9286129 Update to latest models
• Additional commits viewable in compare view

Updates buildkite-test-collector from 0.1.8 to 0.1.9

Release notes

Sourced from buildkite-test-collector's releases.

v0.1.9

What's Changed

• Add TEST_ANALYTICS_COMMIT_MESSAGE by @​petemounce in buildkite/test-collector-python#32

New Contributors

• @​petemounce made their first contribution in buildkite/test-collector-python#32

Full Changelog: buildkite/test-collector-python@v0.1.8...v0.1.9

Commits

• cc149ba v0.1.9
• 9f5e674 Add TEST_ANALYTICS_COMMIT_MESSAGE (#32)
• See full diff in compare view

Updates lm-eval[api] from 0.4.4 to 0.4.5

Release notes

Sourced from lm-eval[api]'s releases.

v0.4.5

lm-eval v0.4.5 Release Notes

New Additions

Prototype Support for Vision Language Models (VLMs)

We're excited to introduce prototype support for Vision Language Models (VLMs) in this release, using model types hf-multimodal and vllm-vlm. This allows for evaluation of models that can process text and image inputs and produce text outputs. Currently we have added support for the MMMU (mmmu_val) task and we welcome contributions and feedback from the community!

New VLM-Specific Arguments

VLM models can be configured with several new arguments within --model_args to support their specific requirements:

• max_images (int): Set the maximum number of images for each prompt.
• interleave (bool): Determines the positioning of image inputs. When True (default) images are interleaved with the text. When False all images are placed at the front of the text. This is model dependent.

hf-multimodal specific args:

• image_token_id (int) or image_string (str): Specifies a custom token or string for image placeholders. For example, Llava models expect an "<image>" string to indicate the location of images in the input, while Qwen2-VL models expect an "<|image_pad|>" sentinel string instead. This will be inferred based on model configuration files whenever possible, but we recommend confirming that an override is needed when testing a new model family
• convert_img_format (bool): Whether to convert the images to RGB format.

Example usage:

• lm_eval --model hf-multimodal --model_args pretrained=llava-hf/llava-1.5-7b-hf,attn_implementation=flash_attention_2,max_images=1,interleave=True,image_string=<image> --tasks mmmu_val --apply_chat_template

• lm_eval --model vllm-vlm --model_args pretrained=llava-hf/llava-1.5-7b-hf,max_images=1,interleave=True --tasks mmmu_val --apply_chat_template

Important considerations

1. Chat Template: Most VLMs require the --apply_chat_template flag to ensure proper input formatting according to the model's expected chat template.
2. Some VLM models are limited to processing a single image per prompt. For these models, always set max_images=1. Additionally, certain models expect image placeholders to be non-interleaved with the text, requiring interleave=False.
3. Performance and Compatibility: When working with VLMs, be mindful of potential memory constraints and processing times, especially when handling multiple images or complex tasks.

Tested VLM Models

We have currently most notably tested the implementation with the following models:

• llava-hf/llava-1.5-7b-hf
• llava-hf/llava-v1.6-mistral-7b-hf
• Qwen/Qwen2-VL-2B-Instruct
• HuggingFaceM4/idefics2 (requires the latest transformers from source)

New Tasks

Several new tasks have been contributed to the library for this version!

New tasks as of v0.4.5 include:

• Open Arabic LLM Leaderboard tasks, contributed by @​shahrzads @​Malikeh97 in #2232
• MMMU (validation set), by @​haileyschoelkopf @​baberabb @​lintangsutawika in #2243

... (truncated)

Commits

• 0845b58 Bump version to v0.4.5 (#2389)
• 2576a8c Fix Llava-1.5-hf ; Update to version 0.4.5 (#2388)
• 1ed1f9e max_images are passed on to vllms limit_mm_per_prompt (#2387)
• ab2c46c HF: switch conditional checks to self.backend from AUTO_MODEL_CLASS (#2353)
• 4cec66e [API] tokenizer: add trust-remote-code (#2372)
• aa457ed Fix float limit override (#2325)
• fe3040f LingOly - Fixing scoring bugs for smaller models (#2376)
• 8f61936 Solution for CSAT-QA tasks evaluation (#2385)
• bfdcdbe Hotfix! (#2383)
• 5e0bc28 fix tests (#2380)
• Additional commits viewable in compare view

Updates rpds-py from 0.20.0 to 0.20.1

Release notes

Sourced from rpds-py's releases.

v0.20.1

What's Changed

• Fix hashing overflow issues (#86) by @​FlickerSoul in crate-py/rpds#87

Full Changelog: crate-py/rpds@v0.20.0...v0.20.1

Commits

• 1b5852d Bump to 0.20.1.
• 06d98b9 Merge pull request #87 from FlickerSoul/hash
• edc02d6 Set --profile=dev in tests to catch issues like #86.
• 53d1976 Merge pull request #95 from crate-py/dependabot/cargo/pyo3-0.22.5
• f63b5e5 Bump pyo3 from 0.22.3 to 0.22.5
• 9a5e177 Merge pull request #93 from crate-py/dependabot/github_actions/wntrblm/nox-20...
• 85f1129 Bump wntrblm/nox from 2024.04.15 to 2024.10.09
• e52d37f Merge pull request #92 from crate-py/pre-commit-ci-update-config
• d65f637 [pre-commit.ci] pre-commit autoupdate
• d962da1 Merge pull request #91 from crate-py/dependabot/cargo/pyo3-0.22.3
• Additional commits viewable in compare view

Updates torchvision from 0.20 to 0.20.1

Commits

• 3ac97aa Update version.txt to 0.20.1 (#8694)
• See full diff in compare view

Updates yarl from 1.17.0 to 1.17.1

Release notes

Sourced from yarl's releases.

1.17.1

Miscellaneous internal changes

• Improved performance of many :class:~yarl.URL methods -- by :user:bdraco.

  Related issues and pull requests on GitHub: #1396, #1397, #1398.

• Improved performance of passing a dict or str to :py:meth:~yarl.URL.extend_query -- by :user:bdraco.

  Related issues and pull requests on GitHub: #1401.

---

Changelog

Sourced from yarl's changelog.

1.17.1

(2024-10-30)

Miscellaneous internal changes

• Improved performance of many :class:~yarl.URL methods -- by :user:bdraco.

  Related issues and pull requests on GitHub: :issue:1396, :issue:1397, :issue:1398.

• Improved performance of passing a dict or str to :py:meth:~yarl.URL.extend_query -- by :user:bdraco.

  Related issues and pull requests on GitHub: :issue:1401.

---

Commits

• f7b83d7 Release 1.17.1 (#1400)
• 5c2af26 Small cleanups to encode_url (#1405)
• 0113797 Fix grammar in #1396 changelog fragment (#1404)
• fb25462 Add missing type for _val to _InternalURLCache TypedDict (#1403)
• 979e06a Additional small cleanups to as a followup to removing self._val (#1402)
• 71d171e Add a fast path to get_str_query for dict and str (#1401)
• 8ce8d67 Small cleanups as a followup to removing self._val (#1399)
• 07798bf Avoid creation of tuples to create new URL objects (#1398)
• 3f2c26f Replace self._val with separate properties (#1397)
• 41af4a3 Complete removal of SplitResult as an internal (#1396)
• Additional commits viewable in compare view

Updates lm-format-enforcer from 0.10.6 to 0.10.9

Release notes

Sourced from lm-format-enforcer's releases.

v0.10.9

• Added new sample: Extracting structured data from vision models

v0.10.8

• Fixed support for tokenizers with larger vocabularies than the models. Enables visual models such as Llama3.2
• JsonSchemaParser: Fixed additionalProperies and oneOf edge cases

v0.10.7

• [135] Updated Haystack V2 integration with latest API

Changelog

Sourced from lm-format-enforcer's changelog.

v0.10.9

• Added new sample: Extracting structured data from vision models

v0.10.8

• Fixed support for tokenizers with larger vocabularies than the models. Enables visual models such as Llama3.2
• JsonSchemaParser: Fixed additionalProperies and oneOf edge cases

v0.10.7

• [135] Updated Haystack V2 integration with latest API

Commits

• cd03b6d v0.10.9
• 86d03d9 Updating vision sample to be easier to run
• 343b790 Adding visual LLM data extraction example
• ad0175d v0.10.8
• c61f00c Allow passing custom vocab_size to build_token_enforcer_tokenizer_data, allow...
• 0a174f7 Fixing additionalProperties related crash
• aaa924f Fixes in JsonSchema oneOf parsing, better multilingual testing (#144)
• f649926 feat: use the vllm Mistral tokenizer wrapper as tokenizer (#142)
• ef0c854 v0.10.7
• 083cdd0 Updating Haystack V2 API integration
• See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore <dependency name> major version will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)
• @dependabot ignore <dependency name> minor version will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)
• @dependabot ignore <dependency name> will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)
• @dependabot unignore <dependency name> will remove all of the ignore conditions of the specified dependency
• @dependabot unignore <dependency name> <ignore condition> will remove the ignore condition of the specified dependency and ignore conditions

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[khluu]

Reverted torch-vision to 0.20 since 0.20.1 depends on torch 2.5.1. Also, we might want to consider pinning lm-eval