|
1 | | -Example iOS app that uses LLM Swift APIs in extension/llm. Priority P0 |
| 1 | +# etLLM Demo App |
| 2 | + |
| 3 | +Get hands-on with running LLMs — exported via ExecuTorch — natively on your iOS device! |
| 4 | + |
| 5 | +<table> |
| 6 | + <tr> |
| 7 | + <td> |
| 8 | + <video src="https://github.com/user-attachments/assets/6ab7b299-cbe5-4412-bb37-cdb12738860a" muted autoplay loop playsinline></video> |
| 9 | + </td> |
| 10 | + <td> |
| 11 | + <video src="https://github.com/user-attachments/assets/9620b7ea-1f09-460d-a120-bfbec896f486" muted autoplay loop playsinline></video> |
| 12 | + </td> |
| 13 | + <td> |
| 14 | + <video src="https://github.com/user-attachments/assets/a9fd4af9-0ca9-4667-a56e-45c91d3e25dd" muted autoplay loop playsinline></video> |
| 15 | + </td> |
| 16 | + <td> |
| 17 | + <video src="https://github.com/user-attachments/assets/f00d0dee-5d51-476b-9dba-70031a21089d" muted autoplay loop playsinline></video> |
| 18 | + </td> |
| 19 | + </tr> |
| 20 | + <tr> |
| 21 | + <td> |
| 22 | + <video src="https://github.com/user-attachments/assets/b1981754-8660-4d6a-86fb-bf7c5010a36b" muted autoplay loop playsinline></video> |
| 23 | + </td> |
| 24 | + <td> |
| 25 | + <video src="https://github.com/user-attachments/assets/0ad0f182-9c54-4042-9789-5e0d954e5844" muted autoplay loop playsinline></video> |
| 26 | + </td> |
| 27 | + <td> |
| 28 | + <video src="https://github.com/user-attachments/assets/16b1bbf6-74b3-4ee9-aed7-2638a72274d7" muted autoplay loop playsinline></video> |
| 29 | + </td> |
| 30 | + </tr> |
| 31 | +</table> |
| 32 | + |
| 33 | + |
| 34 | +## Requirements |
| 35 | +- [Xcode](https://apps.apple.com/us/app/xcode/id497799835?mt=12/) 15.0 or later |
| 36 | +- A development provisioning profile with the [`increased-memory-limit`](https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_developer_kernel_increased-memory-limit) entitlement. |
| 37 | + |
| 38 | +## Models |
| 39 | + |
| 40 | +Download already exported LLMs along with tokenizers from [HuggingFace](https://huggingface.co/executorch-community) or export your own empowered by XNNPACK, Core ML or MPS backends. |
| 41 | + |
| 42 | +## Build and Run |
| 43 | + |
| 44 | +1. Open the Xcode project: |
| 45 | + ```bash |
| 46 | + open llm/apple/etLLM.xcodeproj |
| 47 | + ``` |
| 48 | + |
| 49 | +2. Click the Play button to launch the app in the Simulator. |
| 50 | + |
| 51 | +3. To run on a device, ensure you have it set up for development and a provisioning profile with the `increased-memory-limit` entitlement. Update the app's bundle identifier to match your provisioning profile with the required capability. |
| 52 | +
|
| 53 | +4. After successfully launching the app, copy the exported ExecuTorch model (`.pte`) and tokenizer (`.model`, `.json`, `.bin`, etc.) files to the etLLM folder. Four models are currently supported at the moment - Gemma3, LLaMa, LLaVA, Qwen3, Phi4, SmolLM3, and Voxtral. Please ensure that your model `.pte` file starts with a corresponding prefix (e.g. `llama`, `qwen3`, `phi4`, etc.) so that the app selects the correct model type. |
| 54 | +
|
| 55 | + - **For the Simulator:** Drag and drop both files onto the Simulator window and save them in the `On My iPhone > etLLM` folder. |
| 56 | + - **For a Device:** Open a separate Finder window, navigate to the Files tab, drag and drop both files into the etLLM folder, and wait for the copying to finish. |
| 57 | +
|
| 58 | +6. Follow the app's UI guidelines to select the model and tokenizer files from the local filesystem and issue a prompt. |
| 59 | + |
| 60 | +For more details check out the [Using ExecuTorch on iOS](https://docs.pytorch.org/executorch/1.0/using-executorch-ios.html) page. |
0 commit comments