MediaPipe samples on Kotlin Multiplatform (Android and iOS).
Migration and Feature List:
| Feature | Package | Additional Setup or information | Original Android Sample URL |
|---|---|---|---|
| llm inference | me.xx2bab.mediapiper.llm | Refer to below section to set up Gemma models. | https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/llm_inference/android |
| object detection | me.xx2bab.mediapiper.objectdetection | Please visit the /object-detection branch to get the Object Detection KMP demo, models are embedded in both projects already. (We could not merge it to main at this moment due to its framework conflict with LLM, more information) | https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/object_detection/android-jetpack-compose |
- Download 2 models, gemma-1.1-2b-it-gpu-int4 and gemma-1.1-2b-it-cpu-int4, from Kaggle respectively.
- Execute below command to push models to Android device respectively (must be physical device)
adb push /the/path/to/gemma-2b-it-gpu-int8.bin data/local/tmp/llm/ - Move models to iOS project respectively by dragging the model to the "iosApp" target and select "create folder reference".
- Open the project via Android Studio, create a
local.properties, put the android sdk path and cocoapods executable path.
sdk.dir=/Users/xxxx/Library/Android/sdk
kotlin.apple.cocoapods.bin=/opt/homebrew/lib/ruby/gems/3.3.0/bin/pod
- Get it run via the Android Studio Run button.
- Generate a dummy Framework
# cd to the root dir of this project and run below command
./gradlew :app:generateDummyFramework- Install the pod
# cd to /iosApp and run below command
pod install- Open the
/iosAppproject (iosApp.xcworkspace) via XCode, run it via the XCode Run button or Android Studio Run button with the KMP Plugin support.
Once you modified the code, just reproduce step 3 and that's it!