Update from upstream#1
Merged
Merged
Conversation
* Fix Catalyst Build * Fix SwiftLint
…#417) * Update VLM READMEs to reflect current model support - Update MLXVLM README with all 10 currently supported models from VLMRegistry - Add missing model types: qwen2_5_vl, qwen3_vl, idefics3, gemma3, smolvlm - Update VLMEval README to reflect it uses SmolVLM2, not Qwen2-VL-2B - Change 'have been tried' to 'are supported' for accuracy * Update VLM READMEs to update processing capabilities * Update MLXVLM README
* fastvlm: start modeling * Weights now load * Multimodal projector * Pre-processing * Preparing inputs * Embeddings * Match Python prompt * Remove warnings * Comments * format
* Add LoRA layer keys from config
Note: this is a source breaking change to adapt to a change on the mlx-lm side. Previously you would write:
```swift
public func loraLinearLayers() -> LoRALinearLayers {
model.layers.map { ($0.selfAttn, ["q_proj", "v_proj"]) }
}
```
now this should read:
```swift
public var loraLayers: [Module] {
model.layers
}
```
…xplore#408) * Add EmbedderTool command line utility
* re-port sanitize, fix ml-explore#431
There was a problem hiding this comment.
Pull request overview
This PR syncs updates from upstream, introducing significant enhancements to the MLX Swift Examples repository. The update includes a new embedder-tool for text embedding operations, dependency version updates, and architectural improvements to the LoRA (Low-Rank Adaptation) training system.
- Adds comprehensive embedder-tool with REPL, search, and indexing capabilities
- Updates swift-transformers (1.0.0 → 1.1.1) and mlx-swift (0.25.5 → 0.29.1)
- Refactors LoRA architecture from function-based to property-based API
Reviewed changes
Copilot reviewed 81 out of 82 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| Tools/embedder-tool/* | New command-line tool for text embeddings with search, indexing, and REPL commands |
| Tools/llm-tool/LoraCommands.swift | Updates LoRA command integration to use new adapter-based API; fixes spelling error |
| Libraries/MLXVLM/Models/FastVLM.swift | Adds new FastVLM vision-language model implementation |
| Libraries/MLXLMCommon/Adapters/LoRA/* | Refactors LoRA from function-based to property/adapter-based architecture |
| Libraries/MLXLLM/Models/*.swift | Updates all model implementations to use new loraLayers property |
| Package.swift, Package.resolved | Updates dependency versions for swift-transformers and mlx-swift |
| mlx-swift-examples.xcodeproj/* | Adds embedder-tool target and scheme configuration |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| public let eosTokenId: Int | ||
| public let multimodalProjectorType: String | ||
| public let multimodalProjectorHiddenSize: Int | ||
| public let tokenizerModelMaxLangth: Int |
There was a problem hiding this comment.
The spelling "tokenizerModelMaxLangth" contains a typo - it should be "tokenizerModelMaxLength" with "Length" fully spelled out.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.